Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: add image scheduler abstraction #115

Merged
merged 2 commits into from
Feb 11, 2025

Conversation

edparris
Copy link
Contributor

This change introduces an abstraction for an ImageScheduler that is responsible for selecting the next image to process from a collection of pending requests. Instead of reading directly from the image request queue the ModelRunner loops will ask the ImageScheduler for the next image whenever there are no regions to process. The current code has been refactored as a first-in-first-out (FIFO) scheduler which will let us maintain the current behavior using the new abstraction. Future PRs will introduce new scheduling options based on lessons learned from the ongoing load testing.

New unit tests were added to verify behavior of the FIFO scheduler and the ModelRunner updates. In addition these updates were deployed into a development account and integration test images were run to confirm basic function was not impacted.

Checklist

Before you submit a pull request, please make sure you have the following:

  • Code changes are compact and well-structured to facilitate easy review
  • Changes are documented in the README.md and other relevant documentation pages
  • PR title and description accurately reflect the changes and are detailed enough for historical tracking
  • PR contains tests that cover all new code and the code has been manual tested
  • All new dependencies are declared (if any), and no unnecessary libraries are added
  • Performance impacts (if any) of the changes are evaluated and documented
  • Security implications of the changes (if any) are reviewed and addressed
  • I have read the Contributing Guidelines and agree to follow the Code of Conduct

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

RanbirAulakh
RanbirAulakh previously approved these changes Jan 29, 2025
@edparris edparris force-pushed the feature/image-scheduler-abstraction branch 2 times, most recently from 4a67bf8 to d750b38 Compare January 30, 2025 19:25
@edparris edparris force-pushed the feature/image-scheduler-abstraction branch from d750b38 to 5d4fb12 Compare January 30, 2025 22:05
Copy link
Collaborator

@drduhe drduhe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@edparris edparris merged commit 0c8379c into main Feb 11, 2025
2 checks passed
@edparris edparris deleted the feature/image-scheduler-abstraction branch February 11, 2025 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants