refactor: add image scheduler abstraction #115
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change introduces an abstraction for an ImageScheduler that is responsible for selecting the next image to process from a collection of pending requests. Instead of reading directly from the image request queue the ModelRunner loops will ask the ImageScheduler for the next image whenever there are no regions to process. The current code has been refactored as a first-in-first-out (FIFO) scheduler which will let us maintain the current behavior using the new abstraction. Future PRs will introduce new scheduling options based on lessons learned from the ongoing load testing.
New unit tests were added to verify behavior of the FIFO scheduler and the ModelRunner updates. In addition these updates were deployed into a development account and integration test images were run to confirm basic function was not impacted.
Checklist
Before you submit a pull request, please make sure you have the following:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.