Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added string pooling feature #344

Merged
merged 3 commits into from
Feb 15, 2025
Merged

Added string pooling feature #344

merged 3 commits into from
Feb 15, 2025

Conversation

sandrohanea
Copy link
Owner

Improve String Pooling in Whisper.net

Summary

This PR enhances the string pooling mechanism in Whisper.net to optimize memory usage and reduce allocations when handling strings, particularly in high-performance scenarios. The changes focus on refining how strings are pooled, retrieved, and returned, ensuring efficient reuse.

Changes

  • Implemented StringPool to manage and reuse strings, reducing memory pressure.
  • Optimized UTF8 string conversion using pooled strings to avoid unnecessary allocations.
  • Introduced ReturnString to correctly reinsert strings into the pool for future reuse.
  • Integrated StringPool usage across WhisperProcessor where applicable.
  • Ensured thread safety and performance by structuring the pool using SortedList<int, Stack<string>>.
  • Added example of how to use default string pool.

Benefits

  • Lower GC Pressure: Reduces the number of string allocations, minimizing garbage collection overhead.
  • Improved Performance: Reuses pre-allocated strings where possible, speeding up frequent string operations.

Notes

  • This PR only modifies the string pooling logic; no other functional changes were made.
  • The default behavior remains unchanged unless explicitly configured to use the string pool.

Inspired by #339

@sandrohanea sandrohanea merged commit af98fb3 into main Feb 15, 2025
25 checks passed
@sandrohanea sandrohanea deleted the sandro/stringpooling branch February 15, 2025 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant