v 0.5.2 Updated: 2 years, 2 months ago
Fast BPE tokeniser for use with OpenAI's models.
https://github.com/openai/tiktoken