v 0.5.2 Updated: 2 years ago
Fast BPE tokeniser for use with OpenAI's models.
https://github.com/openai/tiktoken