You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the Tokenizer::with_vocabulary constructor returns a std::io::Error error, abusing the io::ErrorKind::Other variant for situations where the vocabulary data format is invalid.
We should probably provide a custom crate-level Error type and use that instead, with one variant for I/O-errors and further variants for invalid data formats.
The text was updated successfully, but these errors were encountered:
Currently, the
Tokenizer::with_vocabulary
constructor returns astd::io::Error
error, abusing theio::ErrorKind::Other
variant for situations where the vocabulary data format is invalid.We should probably provide a custom crate-level
Error
type and use that instead, with one variant for I/O-errors and further variants for invalid data formats.The text was updated successfully, but these errors were encountered: