Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ci): prepare.sh should install the toolchain #22572

Merged
merged 15 commits into from
Mar 4, 2025
Merged

Conversation

pront
Copy link
Member

@pront pront commented Mar 3, 2025

Summary

This https://github.com/vectordotdev/vector/actions/runs/13635254034/job/38112313325 fails on master.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

How did you test this PR?

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the "no-changelog" label to this PR.

Checklist

  • Please read our Vector contributor resources.
    • make check-all is a good command to run locally. This check is
      defined here. Some of these
      checks might not be relevant to your PR. For Rust changes, at the very least you should run:
      • cargo fmt --all
      • cargo clippy --workspace --all-targets -- -D warnings
      • cargo nextest run --workspace (alternatively, you can run cargo test --all)
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run dd-rust-license-tool write to regenerate the license inventory and commit the changes (if any). More details here.

References

Ref: rust-lang/rustup#4217

@github-actions github-actions bot added the domain: ci Anything related to Vector's CI environment label Mar 3, 2025
pront

This comment was marked as outdated.

@pront pront force-pushed the pront/fix-rust-prepare branch 2 times, most recently from e6715bb to 4ecdcc5 Compare March 3, 2025 17:04
@pront pront force-pushed the pront/fix-rust-prepare branch from 4ecdcc5 to 4a2d3fb Compare March 3, 2025 17:06
@pront pront added the no-changelog Changes in this PR do not need user-facing explanations in the release changelog label Mar 3, 2025
@pront pront marked this pull request as ready for review March 3, 2025 17:08
@pront pront requested a review from a team as a code owner March 3, 2025 17:08
@pront
Copy link
Member Author

pront commented Mar 3, 2025

@pront pront enabled auto-merge March 3, 2025 17:08
@datadog-vectordotdev
Copy link

datadog-vectordotdev bot commented Mar 3, 2025

Datadog Report

Branch report: pront/fix-rust-prepare
Commit report: 715f989
Test service: vector

✅ 0 Failed, 7 Passed, 0 Skipped, 25.44s Total Time

@pront pront requested a review from jszwedko March 3, 2025 17:56
@pront pront added this pull request to the merge queue Mar 3, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 3, 2025
@pront pront enabled auto-merge March 3, 2025 18:43
@pront pront changed the title fix(ci): prepare.sh override toolchain not installed fix(ci): prepare.sh should install the toolchain Mar 3, 2025
@pront pront force-pushed the pront/fix-rust-prepare branch from 1ca3720 to a30b56f Compare March 3, 2025 21:37
@pront pront added this pull request to the merge queue Mar 3, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 3, 2025
@pront pront added this pull request to the merge queue Mar 3, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 4, 2025
@@ -101,6 +101,7 @@ jobs:
- run: sudo -E bash scripts/ci-free-disk-space.sh
- run: sudo -E bash scripts/environment/bootstrap-ubuntu-24.04.sh
- run: bash scripts/environment/prepare.sh
- run: sudo -E ~/.cargo/bin/rustup target add x86_64-unknown-linux-gnu
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the latest failure is because you are using sudo to add the target resulting in the local user not being able to use it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With Rustup 1.28 (https://blog.rust-lang.org/2025/03/02/Rustup-1.28.0.html), rustup will no longer automatically install the active toolchain if it is not installed.

Also, rust-lang/rustup#4217 (related to the previous fix)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aaaah, I see. Should we add this to the bootstrap or prepare script instead?

Copy link
Member Author

@pront pront Mar 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is already part of the prepare script. Not for the bootstrap, because that script is executed with sudo -E and we want an installation for non-root users too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused why you had to add it here, too, then since it is right after the prepare.sh script which also runs it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you referring to https://github.com/vectordotdev/vector/blob/master/.github/workflows/k8s_e2e.yml#L104?

I basically followed the errors and added it. The prepare.sh script runs rustup show active-toolchain || rustup toolchain install and that doesn't install the x86_64-unknown-linux-gnu target.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 I think I'm getting turned around but I'd suggest that we try to move the direct commands into one of the bootstrap or prepare scripts to DRY it up rather than having it specified in different workflows. In the k8s_e2e case it has:

      - run: bash scripts/environment/prepare.sh
      - run: ~/.cargo/bin/rustup target add x86_64-unknown-linux-gnu

So here it seems like you could move the target add to the prepare.sh script.

Copy link
Member Author

@pront pront Mar 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline, we can leave as is for now because it is a special target that is not required by all jobs.

Also, per https://x.com/rustlang/status/1896954416101732709?s=46&t=x164rvMb1LGppUzviMH60w we might be able to just delete all these one-off target add lines.

@pront pront added this pull request to the merge queue Mar 4, 2025
Merged via the queue into master with commit e1dce44 Mar 4, 2025
43 checks passed
@pront pront deleted the pront/fix-rust-prepare branch March 4, 2025 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: ci Anything related to Vector's CI environment no-changelog Changes in this PR do not need user-facing explanations in the release changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants