Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Share more about the file system design tailored to the characteristics of LLM tasks #40

Open
zhitaoli-6 opened this issue Mar 2, 2025 · 1 comment

Comments

@zhitaoli-6
Copy link

Dear community,

Firstly thanks for your contribution to the open source.

The Fire-Flyer File System (3FS) is a high-performance distributed file system designed to address the challenges of AI training and inference workloads.

I understand 3FS is mainly designed for AI or LLM workloads and it has high performance by leveraging modern SSDs and RDMA networks. However, from the README and design notes, I can't see the design dedicated to AI workloads clearly.

I can see some parts as following:

Image

Image

Could you share more about the difference between your file system design compared with a general distributed file system?

@Deeperfinder
Copy link

+1, don't know how to use it with LLM reasoning framework, maybe write a quick start?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants