Popular repositories Loading

1. LookaheadDecoding LookaheadDecoding Public

   [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding

   Python 1.2k 72

2. FastVideo FastVideo Public

   FastVideo is a lightweight framework for accelerating large video diffusion models.

   Python 1.2k 69

3. Consistency_LLM Consistency_LLM Public

   [ICML 2024] CLLMs: Consistency Large Language Models

   Python 377 17

4. Dynasor Dynasor Public

   Simple extension on vLLM to help you speed up reasoning model without training.

   Python 94 10

5. MuxServe MuxServe Public

   Jupyter Notebook 50 4

6. vllm-ltr vllm-ltr Public

   [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank

   Python 40 7