SPD-409 Scrubbing added #17
Conversation
lnauta
force-pushed
the
master
branch
2 times, most recently
from
d33aa49 to
ec8d177
Compare
|
Please merge with master and resolve conflicts. |
Done! |
There was a problem hiding this comment.
According to user story 2:
"As a Picas user, I want the tokens that exceed a specified expected execution time to be automatically resubmitted up to X times, so that I do not need to manually reset them every time."
However, the current implementation of the maximum execution time (either through max_time or stop_function) is different. Currently, the maximum execution time does not apply to the execution time of one token but all tokens together, because the Timer is initialized before the iterator loop. Also, a token is not forced to stop if max_time is exceeded, instead it is checked only after the token is already finished if max_time is exceeded.
If I understand the user story correctly, we need to use e.g. the stopit library: https://theautomatic.net/2021/11/27/how-to-stop-long-running-code-in-python/
examples/scrub-example.py
Outdated
There was a problem hiding this comment.
Having the scrub_count functionality in a example files (scrub-example.py and scrub-timer-example.py) creates duplicate code (i.e. more difficult to maintain) and makes it more difficult for the users as they would need to adapt more code themselves.
I think it's better to have the scrub_count functionality in actors.py together with the stop functionality (RunActor.run). Also the scrub_count needs to be increased in case of the handler function is called (i.e. when Picas is killed).
|
I added an initial attempt for the I have also added the scrubbing to the handler. As for moving it to the actor class, can you make a suggestion on the implementation? |
Find suggestion here: #23 |
* Catch TimeoutExeption * Move scrubbing to RunActor class * Simplify max_time implementation * Fix tests * Check max_total_time for all iterators * Added unit tests --------- Co-authored-by: hailihu@gmail.com <Haili Hu>
|
While testing, I ran into this issue: #24. It is an already existing issue, in this case triggered by an Exception when a token exceeds the |
Suggestion to fix this error here: #25. |
…as DB Views (#25) Co-authored-by: hailihu@gmail.com <Haili Hu>
Has been merged into SPD-409 |
The scrubbing options have been added through examples where the Actor is customised.