Deterministic query cycles for parallel front-end

[zetanumbers]

It's a bit messy currently, so I will need to tidy things up. Anyway this is a working implementation of parallel query cycle detection. Also I suspect code like saving thread ids in query info is redundant and needs to be removed.

The mechanism is similar to cycle detection in single-threaded mode. From the top active query downwards towards subqueries we traverse the deadlocked query graph until we visit some query a second time, thus finding a cycle. With multi-thread front-end enabled one query may now have more than one active subqueries, aka we used one of parallel interfaces parallel!, join, par_for_each, etc. As such we have to traverse the "leftmost" active subquery to recover the sequential behavior of these parallel interfaces in single-threaded mode. New BranchKey saves into implicit context information about what join (or scope) branch in order we took while executing a query, which we then use in break_query_cycle. I reasonably assume only "nesting" functions like join and scope are used to spawn subtasks, but not spawn or something else.

However we then have to guarantee the query stack from single-threaded mode is included in the active query graph. This is true for join functions as their first task will be completed on the same thread and same will be tried for the second task unless stolen which is fine for us. But scope function does not necessary follow this wisdom, and the order of executed tasks there does not seem to be right. TODO: fix scope

Fixes #142064
Fixes #142063
Fixes #127971

Probably fixes other issues with query cycles if those aren't fixed already.

[rustbot]

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

[rustbot]

r? @eholk

rustbot has assigned @eholk.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rust-log-analyzer]

The job tidy failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

checking C++ file formatting
spellchecking files
building external tool typos from package [email protected]
finished building tool typos
error: `begining` should be `beginning`
    ╭▸ compiler/rustc_query_system/src/query/job.rs:349:84
    │
349 │             assert!(stack.start.is_none(), "found two active queries at a thread's begining");
    ╰╴                                                                                   ━━━━━━━━
rerun tidy with `--extra-checks=spellcheck --bless` to fix typos
tidy [extra_checks]: checks with external tool 'typos' failed
tidy [extra_checks]: FAIL
tidy: The following check failed: extra_checks
Bootstrap failed while executing `test src/tools/tidy tidyselftest --extra-checks=py,cpp,js,spellcheck`
Command `/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools-bin/rust-tidy /checkout /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo /checkout/obj/build 4 /node/bin/yarn --extra-checks=py,cpp,js,spellcheck` failed with exit code 1
Created at: src/bootstrap/src/core/build_steps/tool.rs:1612:23
Executed at: src/bootstrap/src/core/build_steps/test.rs:1320:29

Command has failed. Rerun with -v to see more details.
Build completed unsuccessfully in 0:02:45
  local time: Wed Dec 10 14:08:26 UTC 2025
  network time: Wed, 10 Dec 2025 14:08:27 GMT
##[error]Process completed with exit code 1.
##[group]Run echo "disk usage:"

[Kivooeo]

Hi, first of all, thank you for your work on this. I can see a lot has been done to address the issue!

Unable to speak about entire implementation since don't know much about this part, but one note on this particular file: the logic seems a bit unclear to me due to the magic numbers and bit manipulations. Even though the file isn't large, adding a few explanatory comments would be a big help for clarity

[Kivooeo]

Also, we'll need a regression test to check it doesn't ICE with these changes

[matthiaskrgr]

@bors try @rust-timer queue

[Kivooeo]

@matthiaskrgr there is a compilation error in this PR, not sure if this can be benchmarked?

Ok this is surprising

[rust-bors]

☀️ Try build successful (CI)
Build commit: 306a768 (306a768e2f03217e62aee6655739249be1d8aa95, parent: 377656d3dd3f9c23a9c8713e163f4365a5261a84)

[rust-timer]

Finished benchmarking commit (306a768): comparison URL.

Overall result: ❌ regressions - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.2%, 1.8%]	155
Regressions ❌ (secondary)	0.7%	[0.2%, 1.9%]	138
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.0%	[-0.0%, -0.0%]	1
All ❌✅ (primary)	0.5%	[0.2%, 1.8%]	155

Max RSS (memory usage)

Results (secondary -1.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.3%	[2.3%, 2.3%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.1%	[-3.5%, -0.9%]	3
All ❌✅ (primary)	-	-	0

Cycles

Results (primary 2.2%, secondary 2.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.2%	[2.0%, 2.4%]	3
Regressions ❌ (secondary)	3.6%	[2.8%, 5.3%]	4
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-4.6%	[-4.6%, -4.6%]	1
All ❌✅ (primary)	2.2%	[2.0%, 2.4%]	3

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 471.42s -> 474.532s (0.66%)
Artifact size: 389.04 MiB -> 389.08 MiB (0.01%)