Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
PI-2526 Remove repeated punctuation strings
to prevent model errors due to "too many tokens": ``` Input validation error: `inputs` must have less than 512 tokens Given: 566 ``` caused by strings such as: ``` --------------------------------------------------------- Comment added by name on 01/02/2023 at 12:34 Report Edited: 01/02/2023 at 12:34 --------------------------------------------------------- Comment added by name on 01/02/2023 at 12:34 Report Edited: 01/02/2023 at 12:34 ... ```
- Loading branch information