You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We need to integrate CentML as a new remote inference provider within the llama-stack framework. This integration should allow users to seamlessly utilize CentML's available models (meta-llama/Llama-3.3-70B-Instruct and meta-llama/Llama-3.1-405B-Instruct-FP8) for various inference tasks such as chat completions, text completions, and embeddings.
💡 Why is this needed? What if we don't build it?
CentML offers high-performance, scalable inference capabilities for large language models. Integrating it into llama-stack broadens the range of accessible models for users, catering to those who require robust and efficient inference solutions.
If we don't built it, users seeking CentML's specific advantages would be unable to leverage llama-stack, potentially driving them to alternative frameworks that already support CentML.
Other thoughts
No response
The text was updated successfully, but these errors were encountered:
V2arK
changed the title
CentML inference Provier support
CentML inference Provider support
Jan 17, 2025
V2arK
changed the title
CentML inference Provider support
CentML inference provider support
Jan 17, 2025
🚀 Describe the new functionality needed
We need to integrate CentML as a new remote inference provider within the llama-stack framework. This integration should allow users to seamlessly utilize CentML's available models (meta-llama/Llama-3.3-70B-Instruct and meta-llama/Llama-3.1-405B-Instruct-FP8) for various inference tasks such as chat completions, text completions, and embeddings.
💡 Why is this needed? What if we don't build it?
CentML offers high-performance, scalable inference capabilities for large language models. Integrating it into llama-stack broadens the range of accessible models for users, catering to those who require robust and efficient inference solutions.
If we don't built it, users seeking CentML's specific advantages would be unable to leverage llama-stack, potentially driving them to alternative frameworks that already support CentML.
Other thoughts
No response
The text was updated successfully, but these errors were encountered: