Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better error message when experiment is terminated by user #10158

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

JHolba
Copy link
Contributor

@JHolba JHolba commented Feb 26, 2025

Issue
Resolves #9960

Approach
Short description of the approach

(Screenshot of new behavior in GUI if applicable)

  • PR title captures the intent of the changes, and is fitting for release notes.
  • Added appropriate release note label
  • Commit history is consistent and clean, in line with the contribution guidelines.
  • Make sure unit tests pass locally after every commit (git rebase -i main --exec 'just rapid-tests')

When applicable

  • When there are user facing changes: Updated documentation
  • New behavior or changes to existing untested code: Ensured that unit tests are added (See Ground Rules).
  • Large PR: Prepare changes in small commits for more convenient review
  • Bug fix: Add regression test for the bug
  • Bug fix: Create Backport PR to latest release

@JHolba JHolba self-assigned this Feb 26, 2025
@JHolba JHolba added the release-notes:improvement Automatically categorise as improvement in release notes label Feb 26, 2025
@JHolba JHolba marked this pull request as ready for review February 26, 2025 14:36
@JHolba JHolba changed the title Change error message shown when experiment is terminated by user Better error message when experiment is terminated by user Feb 26, 2025
Copy link

codspeed-hq bot commented Feb 26, 2025

CodSpeed Performance Report

Merging #10158 will not alter performance

Comparing JHolba:fix_error_on_terminate_experiment (7d90b91) with main (3965659)

Summary

✅ 25 untouched benchmarks

@JHolba JHolba force-pushed the fix_error_on_terminate_experiment branch from 9c2e575 to c0158cd Compare February 26, 2025 14:50
@@ -610,7 +615,7 @@ async def run_ensemble_evaluator_async(
await evaluator._server_started
if not (await self.run_monitor(ee_config, ensemble.iteration)):
await evaluator_task
return []
return _UserCancelled()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it a good practice to return Exceptions and raise it somewhere else?
Maybe it would be better to just do the extra logic already here, and raise it directly?

Suggested change
return _UserCancelled()
self.active_realizations = [False for _ in self.active_realizations]
raise _UserCancelled()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with @jonathan-eq on this one. Besides, it might be that await evaluator_task raises first if something fails (?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it now raises instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a nicer error message if it fails while waiting for evaluator_task

@@ -94,6 +94,10 @@ def delete_runpath(run_path: str) -> None:
shutil.rmtree(run_path)


class _UserCancelled(Exception):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want this exception to be private, if we intend for it to be raised up to a different module?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made it public. I guess it could be useful, but it was just added to control the exception handling inside base_run_model better

Copy link
Contributor

@jonathan-eq jonathan-eq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some questions 🤔

@JHolba JHolba force-pushed the fix_error_on_terminate_experiment branch 2 times, most recently from fc25259 to 9946c2c Compare February 28, 2025 13:23
@JHolba JHolba force-pushed the fix_error_on_terminate_experiment branch from 9946c2c to 7d90b91 Compare February 28, 2025 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-notes:improvement Automatically categorise as improvement in release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Terminate experiment produces unexpected ValueError
3 participants