py314-tokenizers (python/py-tokenizers) Updated: 3 weeks ago Add to my watchlist
Fast and Customizable TokenizersTokenizers provides an implementation of today's most used tokenizers, with a focus on performance and versatility. Includes BPE, WordPiece, and Unigram tokenizer implementations.
Version: 0.22.2 License: Apache-2
GitHub
Statistics for selected duration
2026-Mar-06 to 2026-Apr-05
| Total Installations | 1 |
|---|---|
| Requested Installations | 0 |
Loading Chart 
Loading Chart
Loading Chart
Loading Chart
| Variants | Count |
|---|
Monthly Statistics
Can remain cached for up to 24 hours
Loading Chart
Percentage of installations per version per month
Loading Chart