PDF/Image Upload with ollama/llama3 without external services

[jowin202]

Hello,

I would like to analyse my pdfs/images/textfiles with ollama and llama3. Therefore, I successfully installed Librechat instance with ollama and llama3 package. Then I deactivated all external services with api keys (google, bing, openai, ...), such that my local ollama instance is my very only endpoint and no connection to the internet and external services. There is llama3 installed at ollama docker container and the "nomic-embed-text" from the tutorial Use RAG with Ollama Local Embedding in this link:

https://www.librechat.ai/docs/configuration/rag_api

Log outputs of rag_api container when trying to upload a file before following the tutorial:

2024-07-08 13:59:03,313 - root - ERROR - Error code: 401 - {'error': {'message': 'Incorrect API key provided: user_pro*ided. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}
2024-07-08 13:59:03,315 - root - INFO - Request POST http://rag_api:8000/embed - 200
which is strange, because I deactivated openai api

which is strange, because I haven't activated any openai support or connection

Log outputs of rag_api container when trying to upload a file after following the tutorial:

with the .env file adjusted like in the given tutorial

RAG_API_URL=http://host.docker.internal:8000
EMBEDDINGS_PROVIDER=ollama
OLLAMA_BASE_URL=http://host.docker.internal:11434
EMBEDDINGS_MODEL=nomic-embed-text

I get

2024-07-08 13:56:09,992 - root - ERROR - Error raised by inference endpoint: HTTPConnectionPool(host='host.docker.internal', port=11434): Max retries exceeded with url: /api/embeddings (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x74a916479a50>: Failed to establish a new connection: [Errno -2] Name or service not known'))
2024-07-08 13:56:09,993 - root - INFO - Request POST http://rag_api:8000/embed - 200

In both cases, the GUI give an "Error processing file" and the docker logs of the librechat container gives

2024-07-08 14:04:28 error: Error embedding file File embedding failed.
2024-07-08 14:04:28 error: [/files] Error processing file: File embedding failed.

Furthermore, I enabled this in librechat.yml to allow file uploads

fileConfig:
  endpoints:
    default:
      fileLimit: 5
      fileSizeLimit: 50
      totalSizeLimit: 500
      supportedMimeTypes:
        - image/.*
        - application/pdf
        - text/plain

and the rag_api successfully installed from the docker-compose.override.yml file, as described in the tutorial.

I don't have any ideas why file upload does not work. However, most of the tutorials are written for bing and openai and not for the local AIs.

thank you in advance

[Kelvinm]

I had a similar error. I fixed it using this page specifically: https://www.librechat.ai/docs/configuration/rag_api

also i made a point of installing poppler tools and running pdftotext on the pdf instead of trying to load the pdf into it. It was very quick, but it worked as intended.