Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
RyanMarten authored Jan 30, 2025
1 parent 3f6a677 commit 10e5095
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,13 @@ Evalchemy is a unified and easy-to-use toolkit for evaluating language models, f

## 🎉 What's New

#### [2025.01.30] New Reasoning Benchmarks and API Model Support

We've added AIME24, AMC23, MATH500, LiveCodeBench, GPQA, HumanEvalPlus, MBPPPlus, BigCodeBench, MultiPL-E, and CRUXEval benchmarks as part of our [Open Thoughts](https://github.com/open-thoughts/open-thoughts) project.

We've also integrated [Curator](https://github.com/bespokelabsai/curator/) as `--model` option to make evaluating even more API based models possible, including all those supported by [LiteLLM](https://docs.litellm.ai/docs/providers).


#### [2025.01.28] New Model Support
- [vLLM models](https://blog.vllm.ai/2023/06/20/vllm.html): High-performance inference and serving engine with PagedAttention technology
```bash
Expand Down

0 comments on commit 10e5095

Please sign in to comment.