py-tokenizers (python/py-tokenizers) Updated: 1 year, 3 months ago Add to my watchlist
Fast and Customizable TokenizersTokenizers provides an implementation of today's most used tokenizers, with a focus on performance and versatility.
Version: 0.13.3 License: Apache-2
GitHub
| Maintainers | No Maintainer |
| Categories | python |
| Homepage | https://github.com/huggingface/tokenizers |
| Platforms | any |
| Variants | - |
Subport(s) (5)
"py-tokenizers" depends on
lib (2)
build (4)
Ports that depend on "py-tokenizers"
No ports
Port Health:
Loading Port Health
Installations (30 days)
0
Requested Installations (30 days)
0
Livecheck results
py-tokenizers seems to have been updated (port version: 0.13.3, new version: 0.20.3)
livecheck ran: 6 hours ago