Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hard code max token counts for supported models #9675

Merged
merged 1 commit into from
Mar 22, 2024
Merged

Hard code max token counts for supported models #9675

merged 1 commit into from
Mar 22, 2024

Conversation

nathansobo
Copy link
Contributor

Previously, we relied on tiktoken-rs to provide the max token count, but it doesn't have a value for "gpt-4-turbo".

Release Notes:

  • Fixed miscalculation of the remaining token count for gpt-4-turbo.

@nathansobo
Hard code max token counts for supported models
093a2bd 
Previously, we relied on tiktoken-rs to provide the max token count,
but it doesn't have a value for "gpt-4-turbo".
@cla-bot cla-bot bot added the cla-signed The user has signed the Contributor License Agreement label Mar 21, 2024
rgbkrk
rgbkrk approved these changes Mar 21, 2024
View reviewed changes
Copy link
Member

@rgbkrk rgbkrk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok this looks great.

@nathansobo nathansobo merged commit 6d5787c into main Mar 22, 2024
9 checks passed
@nathansobo nathansobo deleted the token-counts branch March 22, 2024 02:30
@JosephTLyons
Copy link
Collaborator

Fixes: #9565

@JosephTLyons JosephTLyons mentioned this pull request Mar 22, 2024
Closed
1 task
@JosephTLyons
Copy link
Collaborator

Can we remove the tiktoken dependency entirely?

@as-cii
Copy link
Member

as-cii commented Mar 22, 2024

/cherry-pick v0.128.x

gcp-cherry-pick-bot bot pushed a commit that referenced this pull request Mar 22, 2024
@nathansobo
Hard code max token counts for supported models (#9675)
96b5e56
@gcp-cherry-pick-bot gcp-cherry-pick-bot bot mentioned this pull request Mar 22, 2024
Merged
as-cii pushed a commit that referenced this pull request Mar 22, 2024
@gcp-cherry-pick-bot @nathansobo
Hard code max token counts for supported models (cherry-pick #9675) (#…
646b493 
…9685)

Cherry-picked Hard code max token counts for supported models (#9675)

Co-authored-by: Nathan Sobo <nathan@zed.dev>
@as-cii
Copy link
Member

as-cii commented Mar 22, 2024

@JosephTLyons Unfortunately we can't because we still need to show the count of the tokens we will send.

@JosephTLyons
Copy link
Collaborator

Oh right! Makes sense.

pjlast pushed a commit to pjlast/zed that referenced this pull request Mar 26, 2024
@nathansobo @pjlast
Hard code max token counts for supported models (zed-industries#9675)
3a6d9c7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rgbkrk rgbkrk rgbkrk approved these changes

Assignees
No one assigned
Labels
cla-signed The user has signed the Contributor License Agreement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@nathansobo @JosephTLyons @as-cii @rgbkrk