-
Notifications
You must be signed in to change notification settings - Fork 154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix TestLogIngestionFleetManaged, TestDebLogIngestFleetManaged #5375
Conversation
Pinging @elastic/elastic-agent (Team:Elastic-Agent) |
This pull request does not have a backport label. Could you fix it @belimawr? 🙏
NOTE: |
This commit enables the work directory used by the integration tests framework to be kept in the filesystem if the test fails. The full path of the test directory is printed when the test fails.
Co-authored-by: Paolo Chilà <paolo.chila@elastic.co>
Co-authored-by: Paolo Chilà <paolo.chila@elastic.co>
The namespace generated by the integration tests framework was not unique among different tests and test runs, so sometimes collisions would occurs causing some tests to be flaky.
- Remove debug logs - Make the deb respect the AGENT_KEEP_INSTALLED env var - Add errors that only happen on deb to allow list
06ad52d
to
be00df9
Compare
/test |
|
Force merging this PR to get main back to an healthy state. Here the TestEvenLogFile failed on windows with this error which is unrelated to those change :
|
* [integration tests] Keep work directory if test fails This commit enables the work directory used by the integration tests framework to be kept in the filesystem if the test fails. The full path of the test directory is printed when the test fails. * Update pkg/testing/fixture.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/fixture.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Remove log in error branch * [Integration Tests] Generate namespace based on UUIDv4 The namespace generated by the integration tests framework was not unique among different tests and test runs, so sometimes collisions would occurs causing some tests to be flaky. * Add debug logs * run mage fmt * Fix TestDebLogIngestFleetManaged - Remove debug logs - Make the deb respect the AGENT_KEEP_INSTALLED env var - Add errors that only happen on deb to allow list * Improve logs and test error message --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> Co-authored-by: Julien Lind <julien.lind@elastic.co> (cherry picked from commit 6695324)
* [integration tests] Keep work directory if test fails This commit enables the work directory used by the integration tests framework to be kept in the filesystem if the test fails. The full path of the test directory is printed when the test fails. * Update pkg/testing/fixture.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Update pkg/testing/fixture.go Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * Remove log in error branch * [Integration Tests] Generate namespace based on UUIDv4 The namespace generated by the integration tests framework was not unique among different tests and test runs, so sometimes collisions would occurs causing some tests to be flaky. * Add debug logs * run mage fmt * Fix TestDebLogIngestFleetManaged - Remove debug logs - Make the deb respect the AGENT_KEEP_INSTALLED env var - Add errors that only happen on deb to allow list * Improve logs and test error message --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> Co-authored-by: Julien Lind <julien.lind@elastic.co> (cherry picked from commit 6695324)
TestLogIngestionFleetManaged was failing because the namespace generated by the integration tests framework was not unique among different tests and test runs, so sometimes collisions would occurs causing some tests to be flaky. TestDebLogIngestFleetManaged was failing because it also has got Beats logging connection errors before receiving the configuration from Elastic-Agent, now this message is also in the allow list. When testing .deb the AGENT_KEEP_INSTALLED environment variable is respected. When an integration test fails, the work directory created by the framework is now kept and its path is printed. createTempDir register a test cleanup function to remove the folder it created, however, on Windows, this folder sometimes fails to be removed because there are still open file handlers for the files within the folder. We fix this problem by retrying to remove the folder with a maximum overall wait time of 3 seconds. This is a very similar approach to what Go's t.TempDir does. Fix the flakiness from TestUpgradeHandler* tests by re-working the mockUpgradeManager, now it accepts a function for its Upgrade method and their implementation is goroutine safe TestEnvWithDefault Now TestEnvWithDefault unsets all environment variables it sets, allowing it to be run multiple times using -count. TestContainerCMDEventToStderr TestContainerCMDEventToStderr did not call agentFixture.Prepare early enough leading to an empty STATE_PATH env var, so all state information was in /usr/share/elastic-agent, which could make subsequent tests to fail because they could read /usr/share/elastic-agent/state/container-paths.yml and use a state path different than the one set in the test.
[Integration Tests] Generate namespace based on UUIDv4
What does this PR do?
TestLogIngestionFleetManaged was failing because the namespace generated by the integration tests framework was not unique among different tests and test runs, so sometimes collisions would occurs causing some tests to be flaky.
TestDebLogIngestFleetManaged was failing because it also has got Beats logging connection errors before receiving the configuration from Elastic-Agent, now this message is also in the allow list.
When testing .deb the AGENT_KEEP_INSTALLED environment variable is respected.
When an integration test fails, the work directory created by the framework is now kept and its path is printed.
Why is it important?
It fixes flakiness in our tests
Checklist
I have made corresponding changes to the documentationI have made corresponding change to the default configuration filesI have added tests that prove my fix is effective or that my feature worksI have added an entry in./changelog/fragments
using the changelog toolI have added an integration test or an E2E test## Disruptive User ImpactHow to test this PR locally
Run an integration test:
mage -v integration:single TestLogIngestionFleetManaged
Related issues
Questions to ask yourself