Merge pull request #363 from weaviate/read-model-names-openai

Connect to custom OpenAI Server and retrieve models

CHANGELOG.md

@@ -2,7 +2,11 @@
 
 All notable changes to this project will be documented in this file.
 
-## [2.2.0] Investing in Testing (Suffering will never end)
+## [2.1.1] More Bugs!
+
+## Added
+
+- Dynamic model name retrieval for OpenAI Generator based on OpenAI URL and API Key (https://github.com/weaviate/Verba/issues/123) (https://github.com/weaviate/Verba/issues/362)
 
 ## [2.1.0] Fixing Bugs and Adding Friends
 

README.md

@@ -448,4 +448,8 @@ You can learn more about Verba's architecture and implementation in its [technic
   - Verba is designed and optimized for single user usage only. There are no plans on supporting multiple users or role based access in the near future.
 
 - **Does Verba offer a API endpoint to use externally?**
+
   - No, right now Verba does not offer any useful API endpoints to interact with the application. The current FastAPI setup is optimized for the internal communication between the frontend and backend. It is not recommended to use it as a API endpoint. There are plans to add user-friendly
+
+- **How to connect to your custom OpenAI Server?**
+  - Set your custom OpenAI API Key and URL in the `.env` file, this will allow Verba to start up and retrieve the models from your custom OpenAI Server. `OPENAI_BASE_URL` is set to `https://api.openai.com/v1` by default.

goldenverba/components/generation/GroqGenerator.py

@@ -184,7 +184,6 @@ def get_models(url: str, api_key: str) -> List[str]:
         return models
 
     except Exception as e:
-        msg.info(f"Couldn't connect to Groq ({url})")
         return DEFAULT_MODEL_LIST
 
 

goldenverba/components/generation/OpenAIGenerator.py

@@ -3,8 +3,10 @@
 from goldenverba.components.interfaces import Generator
 from goldenverba.components.types import InputConfig
 from goldenverba.components.util import get_environment, get_token
+from typing import List
 import httpx
 import json
+from wasabi import msg
 
 load_dotenv()
 
@@ -20,7 +22,9 @@ def __init__(self):
         self.description = "Using OpenAI LLM models to generate answers to queries"
         self.context_window = 10000
 
-        models = ["gpt-4o", "gpt-3.5-turbo"]
+        api_key = get_token("OPENAI_API_KEY")
+        base_url = os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1")
+        models = self.get_models(api_key, base_url)
 
         self.config["Model"] = InputConfig(
             type="dropdown",
@@ -118,3 +122,24 @@ def prepare_messages(
         )
 
         return messages
+
+    def get_models(self, token: str, url: str) -> List[str]:
+        """Fetch available embedding models from OpenAI API."""
+        default_models = ["gpt-4o", "gpt-3.5-turbo"]
+        try:
+            if token is None:
+                return default_models
+
+            import requests
+
+            headers = {"Authorization": f"Bearer {token}"}
+            response = requests.get(f"{url}/models", headers=headers)
+            response.raise_for_status()
+            return [
+                model["id"]
+                for model in response.json()["data"]
+                if not "embedding" in model["id"]
+            ]
+        except Exception as e:
+            msg.info(f"Failed to fetch OpenAI models: {str(e)}")
+            return default_models

setup.py

@@ -2,7 +2,7 @@
 
 setup(
     name="goldenverba",
-    version="2.1.0",
+    version="2.1.1",
     packages=find_packages(),
     python_requires=">=3.10.0,<3.13.0",
     entry_points={