Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ODSC-68321] Create grouped model with multiple verified models #1064

Open
wants to merge 3 commits into
base: feature/multi_model_deployment
Choose a base branch
from

Conversation

VipulMascarenhas
Copy link
Member

Description

This PR adds the functionality to support group model creation when more than one model is selected to be deployed. The PR does not cover the deployment part, that needs to be covered by a separate PR.

Usage

The create model API is not accessible directly to the user, we need to use the create deployment API to test this.

ads aqua deployment create 
--instance_shape VM.GPU.A10.2 
--display_name "Model grouping deployment"  
--model_info '[{"model_id": "ocid1.datasciencemodel.oc1.iad.<ocid>", 
                           "gpu_count": 1}, 
                         {"model_id": "ocid1.datasciencemodel.oc1.iad.<ocid2>", 
                          "gpu_count": 1}]'

The new model is called "model_group_YYYYMMDD", and has the freeform tags

multimodel: true
OCI_AQUA: active

The custom metadata added in this model are:

deployment-container: odsc-vllm-serving
model_group_count: 2
model-id-0: ocid1.datasciencemodel.oc1.iad.<ocid>
model-name-0: tiiuae/falcon-7b
artifact_location-0: oci://service-managed-models@<namespace>/service_models/falcon-7b/898df13/artifact
model-id-1:  ocid1.datasciencemodel.oc1.iad.<ocid>
model-name-1: phi-2
artifact_location-1: oci://service-managed-models@<namespace>/service_models/phi-2/b10c3eb/artifact

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Feb 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant