Run main rust-analyzer tests in rust-lang/rust CI

[jieyouxu]

Part of #147370.
MCP: rust-lang/compiler-team#923

This PR prepares rust-analyzer crates with in-rust-tree cargo featues where needed, and and updates bootstrap to run the main rust-analyzer tests in rust-lang/rust CI, not just the proc-macro-srv crate tests.

This supersedes the earlier attempt at #136779. I was honestly expecting more failures in this PR, but looking back at the previous attempt, that makes sense because we no longer run i686-mingw (32-bit windows-gnu) which had a bunch of these failures. In the earlier attempt I also disabled the i686-mingw-related failures for i686-msvc since I didn't feel like digging into 32-bit msvc at the time. Try results from this PR shows that it's most likely limited to 32-bit windows-gnu specifically.

rust-analyzer test remarks

• I actually had to remove the CARGO_WORKSPACE_DIR expect-test-hack in order for expect-test to be able to find the test expectation HTML files (for syntax_highlighting tests in ide). When I added the hack, ironically, it made expect-test unable to find the expectation files. I think this was because previously the path was of the proc-macro-srv crate specifically, now we point to the root r-a workspace?
• The cfg-related differences on aarch64-apple-darwin might've been fixed? I can't tell, but we don't seem to be observing the differences now.
• I'm not sure why config::{generate_config_documentation, generate_package_json_config} no longer fails. Perhaps they were fixed to no longer try to write to source directory?

Review remarks

• Commit 1 updates r-a crates that are involved in tests needing artifacts from rustc_private compiler crates to use the in-rust-tree cargo feature. I briefly tried to use a plain --cfg=in_rust_tree, but quickly realized it was very hacky, and needed invasive bootstrap changes. The cargo feature approach seems most "natural"/well-supported to both bootstrap and cargo.
• Commit 2 updates bootstrap to not only run the proc-macro-srv tests, but the whole r-a tests.
• Commit 3 restricts r-a main tests to non-32-bit targets we test in CI, since (1) r-a repo does not run tests against 32-bit platforms, and (2) there are some target pointer width sensitive hash differences causing tests to fail. Notably, this means that we also no longer run r-a proc-macro-srv tests against 32-bit targets, but we don't expect that crate to be have target pointer width differences. Discussed this in #t-compiler/rust-analyzer > 32-bit tests?.

---

// try-job: aarch64-gnu
// try-job: aarch64-apple
// try-job: x86_64-mingw-1
// try-job: i686-msvc-1
// try-job: x86_64-msvc-1
// try-job: aarch64-msvc-1

[jieyouxu]

@bors try

[jieyouxu]

Huh.
@bors try

[rust-bors]

☀️ Try build successful (CI)
Build commit: 867471f (867471f2085391d229c964771d52579ba984c324, parent: e2c96cc06bdbdbc6f59c7551194d6a742260d6ff)

[Kivooeo]

I'd consider this a win?

[jieyouxu]

r? @Kobzol (for bootstrap side)
r? @Veykril (for r-a side)

FYI @rust-lang/rust-analyzer

[jieyouxu]

Random CI Jobs ™️ go
@bors try jobs=dist-i586-gnu-i586-i686-musl,i686-msvc-1,i686-gnu-2

[rust-bors]

☀️ Try build successful (CI)
Build commit: 2554967 (2554967f93483df371abc1c8d3eeacede37b951d, parent: f5209000832c9d3bc29c91f4daef4ca9f28dc797)

[jieyouxu]

@rustbot ready

[Kobzol]

@bors r+

[bors]

📌 Commit 15e4969 has been approved by Kobzol

It is now in the queue for this repository.

[bors]

⌛ Testing commit 15e4969 with merge eb171a2...

[bors]

☀️ Test successful - checks-actions
Approved by: Kobzol
Pushing eb171a2 to main...

[github-actions]

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing ce63e5d (parent) -> eb171a2 (this PR)

Test differences

Show 15001 test diffs

Stage 0

• completions::dot::tests::macro_expansion_resilient: [missing] -> pass (J0)
• completions::item_list::trait_impl::tests::macro_generated_assoc_item: [missing] -> pass (J0)
• completions::keyword::tests::if_completion_in_format: [missing] -> pass (J0)
• completions::keyword::tests::test_else_edit_after_if: [missing] -> pass (J0)
• config::tests::proc_macro_srv_null: [missing] -> pass (J0)
• goto_definition::tests::goto_def_for_match_nested_in_if: [missing] -> pass (J0)
• handlers::add_missing_impl_members::tests::nested_macro_should_not_cause_crash: [missing] -> pass (J0)
• handlers::apply_demorgan::tests::demorgan_method_call_receiver: [missing] -> pass (J0)
• handlers::convert_closure_to_fn::tests::with_existing_params_trailing_comma: [missing] -> pass (J0)
• handlers::convert_nested_function_to_closure::tests::convert_nested_function_to_closure_works: [missing] -> pass (J0)
• handlers::convert_two_arm_bool_match_to_matches_macro::tests::not_applicable_non_two_arm_match: [missing] -> pass (J0)
• handlers::desugar_doc_comment::tests::quotes_and_escapes: [missing] -> pass (J0)
• handlers::elided_lifetimes_in_path::tests::fn_: [missing] -> pass (J0)
• handlers::expand_glob_import::tests::expanding_glob_reexport: [missing] -> pass (J0)
• handlers::expand_glob_import::tests::test_expanding_multiple_variants_at_once: [missing] -> pass (J0)
• handlers::extract_variable::tests::extract_var_in_match_guard: [missing] -> pass (J0)
• handlers::extract_variable::tests::extract_var_name_from_function: [missing] -> pass (J0)
• handlers::generate_delegate_trait::test::assoc_items_attributes_mutably_cloned: [missing] -> pass (J0)
• handlers::generate_new::tuple_tests::generate_new_not_applicable_if_fn_exists: [missing] -> pass (J0)
• handlers::generate_single_field_struct_from::tests::unit: [missing] -> pass (J0)
• handlers::inline_call::tests::inline_call_closure_body: [missing] -> pass (J0)
• handlers::inline_local_variable::tests::test_used_in_while_expr: [missing] -> pass (J0)
• handlers::introduce_named_lifetime::tests::test_function_not_applicable_without_self_and_multiple_unnamed_param_lifetimes: [missing] -> pass (J0)
• handlers::missing_fields::tests::test_fill_struct_fields_shorthand: [missing] -> pass (J0)
• handlers::missing_unsafe::tests::needs_parentheses_for_unambiguous: [missing] -> pass (J0)
• handlers::missing_unsafe::tests::no_missing_unsafe_diagnostic_with_legacy_safe_intrinsic: [missing] -> pass (J0)
• handlers::promote_local_to_const::tests::multiple_uses: [missing] -> pass (J0)
• handlers::pull_assignment_up::tests::pull_assignment_up_let_stmt_not_applicable: [missing] -> pass (J0)
• handlers::remove_dbg::tests::test_remove_trailing_comma_dbg: [missing] -> pass (J0)
• handlers::replace_filter_map_next_with_find_map::tests::replace_with_find_map: [missing] -> pass (J0)
• handlers::replace_qualified_name_with_use::tests::does_not_replace_assoc_item_path: [missing] -> pass (J0)
• handlers::trait_impl_missing_assoc_item::tests::trait_with_default_value: [missing] -> pass (J0)
• handlers::type_mismatch::tests::type_mismatch_pat_smoke_test: [missing] -> pass (J0)
• handlers::unwrap_return_type::tests::unwrap_result_return_type_break_split_tail: [missing] -> pass (J0)
• handlers::wrap_return_type::tests::wrap_return_type_in_result_simple_with_tail_block_like_match_return_expr: [missing] -> pass (J0)
• hover::tests::raw_keyword_different_editions: [missing] -> pass (J0)
• hover::tests::test_hover_enum_has_impl_action: [missing] -> pass (J0)
• macro_expansion_tests::builtin_derive_macro::test_clone_expand: [missing] -> pass (J0)
• macro_expansion_tests::mbe::test_match_is_not_greedy: [missing] -> pass (J0)
• nameres::tests::reexport_across_crates: [missing] -> pass (J0)
• render::tests::no_parens_in_call: [missing] -> pass (J0)
• render::tests::no_parens_in_use_item: [missing] -> pass (J0)
• syntax_editor::tests::test_replace_root_with_dependent: [missing] -> pass (J0)
• tests::fn_param::first_param: [missing] -> pass (J0)
• tests::generated::doctest_remove_underscore_from_used_variables: [missing] -> pass (J0)
• tests::macros::infer_type_value_non_legacy_macro_use_as: [missing] -> pass (J0)
• tests::never_type::diverging_place_match2: [missing] -> pass (J0)
• tests::record::record_pattern_field_in_macro: [missing] -> pass (J0)
• tests::regression::new_solver::regression_20975: [missing] -> pass (J0)
• tests::runner::ok::record_field_pat_leading_or: [missing] -> pass (J0)
• tests::simple::infer_while_let: [missing] -> pass (J0)
• tests::special::completes_after_colon_without_trigger: [missing] -> pass (J0)
• typing::on_enter::tests::indents_fn_body_block: [missing] -> pass (J0)

Stage 1

• completions::dot::tests::test_tuple_field_inference: [missing] -> pass (J1)
• completions::keyword::tests::test_completion_await_impls_future: [missing] -> pass (J1)
• consteval::tests::locals: [missing] -> pass (J1)
• fixup::tests::for_no_iter_pattern: [missing] -> pass (J1)
• goto_definition::tests::goto_def_for_super_assoc_ty_in_path: [missing] -> pass (J1)
• handlers::auto_import::tests::associated_trait_function: [missing] -> pass (J1)
• handlers::convert_bool_to_enum::tests::field_in_mod_properly_indented: [missing] -> pass (J1)
• handlers::convert_tuple_return_type_to_struct::tests::function_basic: [missing] -> pass (J1)
• handlers::destructure_struct_binding::tests::record_private_fields_appends_rest: [missing] -> pass (J1)
• handlers::extract_function::tests::non_tail_expr_of_tail_expr_loop: [missing] -> pass (J1)
• handlers::extract_struct_from_enum_variant::tests::test_extract_struct_one_field_named: [missing] -> pass (J1)
• handlers::extract_variable::tests::extract_var_regular_parameter: [missing] -> pass (J1)
• handlers::generate_getter_or_setter::tests_getter::test_convert_reference_type: [missing] -> pass (J1)
• handlers::generic_args_prohibited::tests::in_record_pat: [missing] -> pass (J1)
• handlers::inline_call::tests::inline_call_wrapped_in_parentheses: [missing] -> pass (J1)
• handlers::json_is_not_rust::tests::nested_structs: [missing] -> pass (J1)
• handlers::merge_imports::tests::test_double_comma: [missing] -> pass (J1)
• handlers::missing_unsafe::tests::multiple_target_feature_enable: [missing] -> pass (J1)
• handlers::qualify_path::tests::not_applicable_in_non_start_use: [missing] -> pass (J1)
• handlers::qualify_path::tests::not_applicable_when_no_imports_found: [missing] -> pass (J1)
• handlers::replace_if_let_with_match::tests::test_replace_match_with_if_bool: [missing] -> pass (J1)
• handlers::toggle_async_sugar::tests::desugar_with_use: [missing] -> pass (J1)
• handlers::unmerge_imports::tests::unmerge_import_with_attributes: [missing] -> pass (J1)
• handlers::unwrap_return_type::tests::unwrap_result_return_type_simple_with_nested_if: [missing] -> pass (J1)
• hover::tests::float_literal: [missing] -> pass (J1)
• hover::tests::test_hover_struct_doc_attr: [missing] -> pass (J1)
• imports::insert_use::tests::merge_last_fail1: [missing] -> pass (J1)
• lsp::utils::tests::empty_completion_disjoint_tests: [missing] -> pass (J1)
• math: [missing] -> pass (J1)
• references::tests::proc_macro: [missing] -> pass (J1)
• syntax_helpers::format_string_exprs::tests::format_str_parser: [missing] -> pass (J1)
• syntax_highlighting::tests::test_rainbow_highlighting: [missing] -> pass (J1)
• tests::flyimport::trait_method_fuzzy_completion_aware_of_fundamental_boxes: [missing] -> pass (J1)
• tests::generated::doctest_desugar_async_into_impl_future: [missing] -> pass (J1)
• tests::item_list::in_trait_assoc_const_missing_body: [missing] -> pass (J1)
• tests::method_resolution::method_resolution_trait_before_autoderef: [missing] -> pass (J1)
• tests::never_type::diverging_place_match8: [missing] -> pass (J1)
• tests::overlapping_possible_matches: [missing] -> pass (J1)
• tests::runner::err::path_item_without_excl: [missing] -> pass (J1)
• tests::runner::err::static_where_clause: [missing] -> pass (J1)
• tests::runner::ok::marco_pat: [missing] -> pass (J1)
• tests::runner::ok::where_clause: [missing] -> pass (J1)
• tests::simple::generic_default_depending_on_other_type_arg_forward: [missing] -> pass (J1)
• tests::traits::dyn_trait: [missing] -> pass (J1)
• tests::traits::infer_async: [missing] -> pass (J1)
• tests::traits::infer_await: [missing] -> pass (J1)
• tests::traits::lifetime_bounds: [missing] -> pass (J1)

(and 14838 additional test diffs)

Additionally, 63 doctest diffs were found. These are ignored, as they are noisy.

Job group index

• J0: x86_64-gnu-llvm-20-3, x86_64-gnu-llvm-21-3
• J1: aarch64-apple, aarch64-gnu, aarch64-gnu-llvm-20-2, aarch64-msvc-2, x86_64-gnu, x86_64-gnu-llvm-20-1, x86_64-gnu-llvm-21-1, x86_64-gnu-nopt, x86_64-gnu-stable, x86_64-mingw-2, x86_64-msvc-2

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard eb171a227f9e5de5d376b6edb56b174bc8235fb3 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

1. aarch64-apple: 8697.0s -> 15116.9s (+73.8%)
2. aarch64-gnu-llvm-20-2: 2165.6s -> 2949.3s (+36.2%)
3. x86_64-gnu-llvm-21-1: 3167.6s -> 4229.1s (+33.5%)
4. x86_64-msvc-2: 7256.8s -> 9095.2s (+25.3%)
5. x86_64-gnu-llvm-20-1: 3209.5s -> 3988.4s (+24.3%)
6. aarch64-msvc-2: 5223.0s -> 6112.4s (+17.0%)
7. x86_64-gnu-nopt: 7388.3s -> 8510.0s (+15.2%)
8. x86_64-gnu-distcheck: 7046.4s -> 8051.0s (+14.3%)
9. aarch64-gnu: 6443.9s -> 7352.3s (+14.1%)
10. x86_64-gnu-llvm-21-2: 5653.0s -> 4868.8s (-13.9%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

[rust-timer]

Finished benchmarking commit (eb171a2): comparison URL.

Overall result: ❌ regressions - no action needed

@rustbot label: -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.4%	[0.4%, 0.4%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.4%	[0.4%, 0.4%]	1

Max RSS (memory usage)

Results (primary -2.8%, secondary 1.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.0%	[1.0%, 1.0%]	1
Improvements ✅ (primary)	-2.8%	[-2.8%, -2.8%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-2.8%	[-2.8%, -2.8%]	1

Cycles

Results (secondary 3.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.5%	[3.5%, 3.5%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 476.217s -> 476.004s (-0.04%)
Artifact size: 389.30 MiB -> 389.34 MiB (0.01%)