embedding api request in parallelize cause crash

[joebnb]

when request embedding api in dify with parallelize will cause loaded embedding model eject.

and if in headless mode would make a configuration that if model was unload, lms will reload recent used model automatic

[yagil]

Hi @joebnb does it crash (with an error), or does it just unload unexpectedly (NO error)?

If you get an error, can you please share it here?

[joebnb]

2025-02-25 13:58:04 [ERROR] 
[Server Error] Error: The model has crashed without additional information. (Exit code: 6)
2025-02-25 13:58:04 [ERROR] 
[Server Error] Error: The model has crashed without additional information. (Exit code: 6)
2025-02-25 13:58:04 [ERROR] 
[Server Error] Error: The model has crashed without additional information. (Exit code: 6)
2025-02-25 13:58:04 [ERROR] 
[Server Error] Error: The model has crashed without additional information. (Exit code: 6)
2025-02-25 13:58:04 [ERROR] 
[Server Error] Error: The model has crashed without additional information. (Exit code: 6)
2025-02-25 13:58:04 [ERROR] 
[Server Error] Error: The model has crashed without additional information. (Exit code: 6)
2025-02-25 13:58:04 [ERROR] 
[Server Error] Error: The model has crashed without additional information. (Exit code: 6)
2025-02-25 13:58:04 [ERROR] 
[Server Error] Error: The model has crashed without additional information. (Exit code: 6)
2025-02-25 13:58:04 [ERROR] 
[Server Error] Error: The model has crashed without additional information. (Exit code: 6)
2025-02-25 13:58:04 [ERROR] 
[Server Error] Error: Model has unloaded or crashed.

maybe it's caused by model self,but if there have a reload function would better,lmstudio-community/granite-embedding-278m-multilingual-GGUF i use this model to encoding Chinese documents, if use lm-commnuity/text-embedding-nomic-embed-text-v1.5 works fine

[joebnb]

only nomic-embed works,other embedding model all will cause crash when handling batch encoding

[yagil]

Thanks @joebnb would you be able to share one of these documents or at least that very last string before the crash?

[joebnb]

Thanks @joebnb would you be able to share one of these documents or at least that very last string before the crash?

sorry, accidentally closed this issue. i can refer that document tomorrow, but i think it's not about some special characters,because these model crashed at different progress, some crash at start of embedding,some almost finish. anyway i will offer a document on here when i have time

[yagil]

Reopened the issue. And good to know you saw it in different places

[joebnb]

Reopened the issue. And good to know you saw it in different places

sorry,that is English version,this is Chinese

https://static1.squarespace.com/static/5f92a5011169d63a9f7a2533/t/617ae29e892f820731bfce40/1635443358975/Lowders+Classic+Recipe+Training+Manual+-+%E4%B8%AD%E6%96%87%E7%89%88+%286+Aug+2020%29.pdf