Hard crash while using Gemini 2.0 Thinking #5552
Replies: 3 comments 5 replies
-
|
Latest dependency is now 591a019 and coincidentally, I updated google dependencies. Try latest. I can't reproduce this on the same commit anyway.
|
Beta Was this translation helpful? Give feedback.
-
Reproduced on 591a019. Here is the logs - a number of attempts that ends up with 500 Internal Server Error immediately and do not cause any harm, then the fatal crash while streaming the response UI becomes unresponsive while backend is down, there's how it looks:
|
Beta Was this translation helpful? Give feedback.
-
|
I just get the censor right away and no additional messages. What are the resources of your machine? 2025-01-30T01:24:37.547Z debug: [BaseClient] userMessage
{
messageId: "3b45698a-a374-463c-bfb8-4a0de1f18310",
parentMessageId: "00000000-0000-0000-0000-000000000000",
conversationId: "4dc2951e-7280-4e09-a033-324b78f6d819",
sender: "User",
text: "расскажи мне длинную историю",
isCreatedByUser: true,
tokenCount: 20,
}
2025-01-30T01:24:37.570Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-01-30T01:24:37.686Z error: [handleAbortError] AI response error; aborting request: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1be... [truncated]
2025-01-30T01:24:37.686Z warn: AI Response error for conversation 4dc2951e-7280-4e09-a033-324b78f6d819 likely caused by Google censor/filter
2025-01-30T01:24:37.706Z debug: [AskController] Request closed |
Beta Was this translation helpful? Give feedback.
-
I really don't think it's resource-related in any way Rather, it seems to be somewhat rare unhandled case of filter catching up too late to immediately stop the response so it then kills the stream. Right now it took me ~80 attempts to replicate it again, most of the time it's indeed censor right away |
Beta Was this translation helpful? Give feedback.
-
|
Here is a prompt that replicates it somewhat reliably: It also crashes with |
Beta Was this translation helpful? Give feedback.
-
|
Ok I will try this and see |
Beta Was this translation helpful? Give feedback.
-
|
@danny-avila this needs to be handled. it makes the instance to stop/restart in case of any malformed data received by GenAI, due to network issues for example. |
Beta Was this translation helpful? Give feedback.
-
|
I still can't replicate this but adding a try/catch block over the stream generation should fix this. The following prompt indeed gets cut off due to google censoring, but I may not be receiving the error as the abort controller may be triggering before upstream SDK can bubble up the error Tell me a story in two paragraphs, the first paragraph should be in English, the second paragraph should be in Cyrillic alphabetRelevant upstream code: |
Beta Was this translation helpful? Give feedback.
-
What happened?
Model is gemini-2.0-flash-thinking-exp-01-21, right now it's quite unstable for me and returns 500 Internal Server Error for almost any request. The immediate 500 itself is being handled correctly.
However, if it's starts responding, and then fails while streaming the response, it seems to crash whole LibreChat instance.
I haven't found any correlation of that happening with the GOOGLE_SAFETY settings.
LibreChat Version: d60a149 (latest
maincommit at the time of writing)Steps to Reproduce
gemini-2.0-flash-thinking-exp-01-21to GOOGLE_MODELSWhat browsers are you seeing the problem on?
No response
Relevant log output
Service:
Error log:
{"level":"error","message":"There was an uncaught error: [GoogleGenerativeAI Error]: Failed to parse stream","stack":"Error: [GoogleGenerativeAI Error]: Failed to parse stream\n at /root/LibreChat/node_modules/@google/generative-ai/dist/index.js:711:46\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5)"}Screenshots
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions