make_sampler
creates sampler chain with all sampling parameters
#1330
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #1219
Apply samplers in this order: top-k --> top-p --> min-p --> temp
The sampler methods were refactored to return the transformed logits instead of a token. Now, we can can chain them together to apply a series of sampling methods.
The method names of the samplers were changed to indicate that the implementation has changed, i.e., each method returns logits instead of a token. I am not sure about the API stability requirements, so let me know if you have a preference on the method name.
The temperature adjustment was removed from each sampler application, and instead it's applied only once as the very last step.