Tracking Issue for Parallel Rustc Front-end #113349
Labels
C-tracking-issue
Category: An issue tracking the progress of sth. like the implementation of an RFC
WG-compiler-parallel
Working group: Parallelizing the compiler
Comments
SparrowLii
added
the
C-tracking-issue
nnethercote
added
the
WG-compiler-parallel
|
The |
|
It's been a long time since there was any update. Has all work been paused？ |
|
Them main work now is to fix deadlock and ice bugs in the issue list I recently submitted a PR for an enhanced compiler test to reproduce the issue and successfully located three deadlock bugs. Unfortunately, it takes timeto figure out the cause of these bugs, which is what is making the updates slow. I expect it will take another 1 to 2 months to fix the remaining bugs, after which the stabilization will begin. |
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
Nov 12, 2024
…=cjgillot Delete the `cfg(not(parallel))` serial compiler Since it's inception a long time ago, the parallel compiler and its cfgs have been a maintenance burden. This was a necessary evil the allow iteration while not degrading performance because of synchronization overhead. But this time is over. Thanks to the amazing work by the parallel working group (and the dyn sync crimes), the parallel compiler has now been fast enough to be shipped by default in nightly for quite a while now. Stable and beta have still been on the serial compiler, because they can't use `-Zthreads` anyways. But this is quite suboptimal: - the maintenance burden still sucks - we're not testing the serial compiler in nightly Because of these reasons, it's time to end it. The serial compiler has served us well in the years since it was split from the parallel one, but it's over now. Let the knight slay one head of the two-headed dragon! rust-lang#113349 Note that the default is still 1 thread, as more than 1 thread is still fairly broken. cc `@onur-ozkan` to see if i did the bootstrap field removal correctly, `@SparrowLii` on the sync parts
As well as I understand there is no such option anymore. Please, update post |
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
Feb 5, 2025
tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc` tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc` This is continuation of rust-lang#132282 . I'm pretty sure I did everything right. In particular, I searched all occurrences of `Lrc` in submodules and made sure that they don't need replacement. There are other possibilities, through. We can define `enum Lrc<T> { Rc(Rc<T>), Arc(Arc<T>) }`. Or we can make `Lrc` a union and on every clone we can read from special thread-local variable. Or we can add a generic parameter to `Lrc` and, yes, this parameter will be everywhere across all codebase. So, if you think we should take some alternative approach, then don't merge this PR. But if it is decided to stick with `Arc`, then, please, merge. cc "Parallel Rustc Front-end" ( rust-lang#113349 ) r? SparrowLii `@rustbot` label WG-compiler-parallel
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
Feb 6, 2025
tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc` tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc` This is continuation of rust-lang#132282 . I'm pretty sure I did everything right. In particular, I searched all occurrences of `Lrc` in submodules and made sure that they don't need replacement. There are other possibilities, through. We can define `enum Lrc<T> { Rc(Rc<T>), Arc(Arc<T>) }`. Or we can make `Lrc` a union and on every clone we can read from special thread-local variable. Or we can add a generic parameter to `Lrc` and, yes, this parameter will be everywhere across all codebase. So, if you think we should take some alternative approach, then don't merge this PR. But if it is decided to stick with `Arc`, then, please, merge. cc "Parallel Rustc Front-end" ( rust-lang#113349 ) r? SparrowLii `@rustbot` label WG-compiler-parallel
github-actions bot
pushed a commit
to rust-lang/miri
that referenced
this issue
Feb 7, 2025
tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc` tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc` This is continuation of rust-lang/rust#132282 . I'm pretty sure I did everything right. In particular, I searched all occurrences of `Lrc` in submodules and made sure that they don't need replacement. There are other possibilities, through. We can define `enum Lrc<T> { Rc(Rc<T>), Arc(Arc<T>) }`. Or we can make `Lrc` a union and on every clone we can read from special thread-local variable. Or we can add a generic parameter to `Lrc` and, yes, this parameter will be everywhere across all codebase. So, if you think we should take some alternative approach, then don't merge this PR. But if it is decided to stick with `Arc`, then, please, merge. cc "Parallel Rustc Front-end" ( rust-lang/rust#113349 ) r? SparrowLii `@rustbot` label WG-compiler-parallel
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
Feb 9, 2025
…, r=<try>
[perf run] parallel: rustc_data_structures: sync: implement RwLock as enum { Sync(parking_lot::RwLock), NoSync(RefCell) }
Please, somebody, run perf.
parallel: rustc_data_structures: sync: implement `RwLock` as `enum { Sync(parking_lot::RwLock), NoSync(RefCell) }`
Hopefully this will make single threaded front end faster.
I carefully split changes into commits. Commit messages are self-explanatory. Squashing is not recommended.
cc "Parallel Rustc Front-end" rust-lang#113349
r? SparrowLii
`@rustbot` label: +WG-compiler-parallel
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
Feb 10, 2025
…, r=<try>
parallel: rustc_data_structures: sync: implement RwLock as enum { Sync(parking_lot::RwLock), NoSync(RefCell) }
parallel: rustc_data_structures: sync: implement `RwLock` as `enum { Sync(parking_lot::RwLock), NoSync(RefCell) }`
Hopefully this will make single threaded front end faster.
I carefully split changes into commits. Commit messages are self-explanatory. Squashing is not recommended.
cc "Parallel Rustc Front-end" rust-lang#113349
r? SparrowLii
`@rustbot` label: +WG-compiler-parallel
github-actions bot
pushed a commit
to rust-lang/rustc-dev-guide
that referenced
this issue
Feb 10, 2025
tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc` tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc` This is continuation of rust-lang/rust#132282 . I'm pretty sure I did everything right. In particular, I searched all occurrences of `Lrc` in submodules and made sure that they don't need replacement. There are other possibilities, through. We can define `enum Lrc<T> { Rc(Rc<T>), Arc(Arc<T>) }`. Or we can make `Lrc` a union and on every clone we can read from special thread-local variable. Or we can add a generic parameter to `Lrc` and, yes, this parameter will be everywhere across all codebase. So, if you think we should take some alternative approach, then don't merge this PR. But if it is decided to stick with `Arc`, then, please, merge. cc "Parallel Rustc Front-end" ( rust-lang/rust#113349 ) r? SparrowLii `@rustbot` label WG-compiler-parallel
workingjubilee
added a commit
to workingjubilee/rustc
that referenced
this issue
Feb 10, 2025
…22, r=SparrowLii rustc_middle: parallel: TyCtxt: remove "unsafe impl DynSend/DynSync" rustc_middle: parallel: TyCtxt: remove "unsafe impl DynSend/DynSync" We don't need to "short circuit trait resolution", because DynSend and DynSync are auto traits and thus coinductive cc "Parallel Rustc Front-end" rust-lang#113349 r? SparrowLii `@rustbot` label: +WG-compiler-parallel (rustbot sometimes ignores me and doesn't attach labels on my behalf. rustbot banned me?)
matthiaskrgr
added a commit
to matthiaskrgr/rust
that referenced
this issue
Feb 10, 2025
…22, r=SparrowLii rustc_middle: parallel: TyCtxt: remove "unsafe impl DynSend/DynSync" rustc_middle: parallel: TyCtxt: remove "unsafe impl DynSend/DynSync" We don't need to "short circuit trait resolution", because DynSend and DynSync are auto traits and thus coinductive cc "Parallel Rustc Front-end" rust-lang#113349 r? SparrowLii ``@rustbot`` label: +WG-compiler-parallel (rustbot sometimes ignores me and doesn't attach labels on my behalf. rustbot banned me?)
rust-timer
added a commit
to rust-lang-ci/rust
that referenced
this issue
Feb 10, 2025
Rollup merge of rust-lang#136731 - safinaskar:parallel-2025-02-08-07-22, r=SparrowLii rustc_middle: parallel: TyCtxt: remove "unsafe impl DynSend/DynSync" rustc_middle: parallel: TyCtxt: remove "unsafe impl DynSend/DynSync" We don't need to "short circuit trait resolution", because DynSend and DynSync are auto traits and thus coinductive cc "Parallel Rustc Front-end" rust-lang#113349 r? SparrowLii ``@rustbot`` label: +WG-compiler-parallel (rustbot sometimes ignores me and doesn't attach labels on my behalf. rustbot banned me?)
jhpratt
added a commit
to jhpratt/rust
that referenced
this issue
Feb 13, 2025
…2-11-07-54, r=SparrowLii Parallel-compiler-related cleanup Parallel-compiler-related cleanup I carefully split changes into commits. Commit messages are self-explanatory. Squashing is not recommended. cc "Parallel Rustc Front-end" rust-lang#113349 r? SparrowLii `@rustbot` label: +WG-compiler-parallel
jhpratt
added a commit
to jhpratt/rust
that referenced
this issue
Feb 13, 2025
…2-11-07-54, r=SparrowLii Parallel-compiler-related cleanup Parallel-compiler-related cleanup I carefully split changes into commits. Commit messages are self-explanatory. Squashing is not recommended. cc "Parallel Rustc Front-end" rust-lang#113349 r? SparrowLii ``@rustbot`` label: +WG-compiler-parallel
rust-timer
added a commit
to rust-lang-ci/rust
that referenced
this issue
Feb 13, 2025
Rollup merge of rust-lang#136858 - safinaskar:parallel-cleanup-2025-02-11-07-54, r=SparrowLii Parallel-compiler-related cleanup Parallel-compiler-related cleanup I carefully split changes into commits. Commit messages are self-explanatory. Squashing is not recommended. cc "Parallel Rustc Front-end" rust-lang#113349 r? SparrowLii ``@rustbot`` label: +WG-compiler-parallel
antoyo
pushed a commit
to rust-lang/rustc_codegen_gcc
that referenced
this issue
Feb 14, 2025
tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc` tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc` This is continuation of rust-lang/rust#132282 . I'm pretty sure I did everything right. In particular, I searched all occurrences of `Lrc` in submodules and made sure that they don't need replacement. There are other possibilities, through. We can define `enum Lrc<T> { Rc(Rc<T>), Arc(Arc<T>) }`. Or we can make `Lrc` a union and on every clone we can read from special thread-local variable. Or we can add a generic parameter to `Lrc` and, yes, this parameter will be everywhere across all codebase. So, if you think we should take some alternative approach, then don't merge this PR. But if it is decided to stick with `Arc`, then, please, merge. cc "Parallel Rustc Front-end" ( rust-lang/rust#113349 ) r? SparrowLii `@rustbot` label WG-compiler-parallel
antoyo
pushed a commit
to rust-lang/rustc_codegen_gcc
that referenced
this issue
Feb 14, 2025
…4, r=SparrowLii Parallel-compiler-related cleanup Parallel-compiler-related cleanup I carefully split changes into commits. Commit messages are self-explanatory. Squashing is not recommended. cc "Parallel Rustc Front-end" rust-lang/rust#113349 r? SparrowLii ``@rustbot`` label: +WG-compiler-parallel
github-actions bot
pushed a commit
to rust-lang/rustc-dev-guide
that referenced
this issue
Feb 17, 2025
…4, r=SparrowLii Parallel-compiler-related cleanup Parallel-compiler-related cleanup I carefully split changes into commits. Commit messages are self-explanatory. Squashing is not recommended. cc "Parallel Rustc Front-end" rust-lang/rust#113349 r? SparrowLii ``@rustbot`` label: +WG-compiler-parallel
flip1995
pushed a commit
to flip1995/rust-clippy
that referenced
this issue
Feb 20, 2025
tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc` tree-wide: parallel: Fully removed all `Lrc`, replaced with `Arc` This is continuation of rust-lang/rust#132282 . I'm pretty sure I did everything right. In particular, I searched all occurrences of `Lrc` in submodules and made sure that they don't need replacement. There are other possibilities, through. We can define `enum Lrc<T> { Rc(Rc<T>), Arc(Arc<T>) }`. Or we can make `Lrc` a union and on every clone we can read from special thread-local variable. Or we can add a generic parameter to `Lrc` and, yes, this parameter will be everywhere across all codebase. So, if you think we should take some alternative approach, then don't merge this PR. But if it is decided to stick with `Arc`, then, please, merge. cc "Parallel Rustc Front-end" ( rust-lang/rust#113349 ) r? SparrowLii `@rustbot` label WG-compiler-parallel
workingjubilee
added a commit
to workingjubilee/rustc
that referenced
this issue
Mar 4, 2025
…lel, r=compiler-errors Revert "compiler/rustc_data_structures/src/sync/worker_local.rs: delete "unsafe impl Sync"" Revert "compiler/rustc_data_structures/src/sync/worker_local.rs: delete "unsafe impl Sync"" This reverts commit 0240690. See rust-lang#136858 (comment) cc "Parallel Rustc Front-end" rust-lang#113349 r? SparrowLii `@rustbot` label: +WG-compiler-parallel `@rustbot` label: +C-cleanup
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is a tracking issue for Parallel Rustc Front-end
This feature will improve compilation speed by parallelizing the process of rustc front end (before the codegen which has been parallelized).
The feature was previously called parallel queries (and its tracking issue can be found here), developed by the former parallel rustc working group. Thanks to their work, developers can currently use parallel rustc by setting
parallel_compiler=trueinconfig.tomlwhen building the compiler.The parallel rustc working group has rebooted (here's the associated MCP), which will make parallel front-end the default option in rustc. Here's the (draft) landing strategy for this feature.
Below is the task list.
Solve the problem of single-thread performance reduction
DynSendandDynSyncauto trait for parallel compiler #107586Shardedan enum and specialize it for the single thread case #114860Shardedfrom query state #114894Shardedand refactorLock#115388Shardedonly for multiple threads #115162Optimize multi-threading performance
Enable parallel compilation by default in nightly rustc
Bugs or test failures of parallel rustc
tcxin deadlock handler when parallel compilation #98570evaluate_trait_predicate_recursivelyin rustdoc when proving Send/Sync #106930attempted to read from stolen value: rustc_middle::thir::Thir:-Zthir-unsafeck=yes#111520CurrentGcxtype to let the deadlock handler accessTyCtxt#115220ParallelGuardtype to handle unwinding in parallel sections #115144deadlock detected#111521no ImplicitCtxt stored in tls#111522value must be in cache after waiting#111528Resource temporarily unavailablewhen limiting max memory #115021assertion failed: found_cycle#115223Testing
Documentation
About tracking issues
Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.
The text was updated successfully, but these errors were encountered: