-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deployment of agent container is always attempted #195
Comments
@mvanniasingheTT which models are compatible for using with the agent container? I assume its all the LLMs? |
All the llama models except the small ones like 3B should work. |
I don't mean work in terms of accuracy. I mean do they raise errors, if any? I expect the small LLMs like 3.1-1B to still deploy properly but are kind of buggy. |
Yes, that's correct. All the LLMs, even the small ones, still deploy properly. |
Describe the bug
In
app/api/docker_control/views.py
we also attempt to deploy the agent container for every model. This does not make sense for non-LLM models, causing the API call to "fail" but the actual model deployment is successful.To Reproduce
Steps to reproduce the behavior:
Expected behavior
We should launch the agent container depending on the model type. This is an exact scenario where implementing #167 will fix all of our problems. I think the solution from #167 should be when deciding whether to deploy the agent container.
The text was updated successfully, but these errors were encountered: