py310-tokenizers (python/py-tokenizers) Updated: 3 weeks, 1 day ago Add to my watchlist
Fast and Customizable TokenizersTokenizers provides an implementation of today's most used tokenizers, with a focus on performance and versatility. Includes BPE, WordPiece, and Unigram tokenizer implementations.
Version: 0.22.2 License: Apache-2
GitHub
97 build(s) found
| Builder | Build Number | Start Time | Elapsed Time | Watcher | Build Status |
|---|---|---|---|---|---|
| 10.14 | 255660 | 2026-04-03 6:44:35 | 0:05:09 | 87735 | build successful |
| 10.15 | 231938 | 2026-04-02 19:16:29 | 0:05:31 | 76810 | build successful |
| 10.13 | 282444 | 2026-04-01 9:31:40 | 0:00:11 | 97035 | failed install-dependencies |
| 10.8 | 232820 | 2026-03-27 5:16:16 | 0:00:09 | 76520 | failed install-dependencies |
| 10.6.x86_64 | 266090 | 2026-03-26 17:04:58 | 0:00:12 | 79128 | failed install-dependencies |
| 10.11 | 316224 | 2026-03-25 23:47:50 | 0:00:12 | 106225 | failed install-dependencies |
| 10.10 | 325460 | 2026-03-25 23:41:38 | 0:00:08 | 105941 | failed install-dependencies |
| 10.11 | 316172 | 2026-03-25 22:04:42 | 0:00:17 | 106213 | failed install-dependencies |
| 10.7 | 247592 | 2026-03-25 17:22:18 | 0:00:14 | 78335 | failed install-dependencies |
| 10.7 | 247553 | 2026-03-25 16:36:13 | 0:00:08 | 78323 | failed install-dependencies |
| 10.6 | 228684 | 2026-03-25 6:00:15 | 0:00:27 | 78979 | failed install-dependencies |
| 10.9 | 333678 | 2026-03-25 3:35:04 | 0:00:13 | 105776 | failed install-dependencies |
| 10.9 | 333651 | 2026-03-25 3:14:00 | 0:00:10 | 105775 | failed install-dependencies |
| 10.12 | 326537 | 2026-03-25 0:33:44 | 0:00:09 | 105033 | failed install-dependencies |
| 10.6 | 228344 | 2026-03-24 6:45:15 | 0:00:36 | 78910 | failed install-dependencies |
| 10.6.x86_64 | 265464 | 2026-03-22 13:20:13 | 0:00:12 | 78864 | failed install-dependencies |
| 10.12 | 326037 | 2026-03-22 3:49:47 | 0:00:09 | 104859 | failed install-dependencies |
| 10.6 | 227871 | 2026-03-22 1:24:20 | 0:00:32 | 78762 | failed install-dependencies |
| 10.8 | 231292 | 2026-03-20 23:50:56 | 0:00:12 | 76115 | failed install-dependencies |
| 10.9 | 331987 | 2026-03-19 22:04:55 | 0:00:08 | 105303 | failed install-dependencies |
| 13.arm64 | 99615 | 2026-03-18 6:08:37 | 0:01:53 | 40569 | build successful |
| 10.12 | 325232 | 2026-03-18 2:33:23 | 0:00:14 | 104538 | failed install-dependencies |
| 10.6.x86_64 | 264474 | 2026-03-17 21:35:23 | 0:00:14 | 78515 | failed install-dependencies |
| 15 | 51302 | 2026-03-17 13:42:24 | 0:01:56 | 16515 | build successful |
| 14.arm64 | 72938 | 2026-03-16 23:29:22 | 0:02:32 | 28662 | build successful |
| 10.6 | 226712 | 2026-03-16 17:56:57 | 0:00:21 | 78431 | failed install-dependencies |
| 10.11 | 314787 | 2026-03-16 15:12:18 | 0:00:09 | 105768 | failed install-dependencies |
| 26.arm64 | 30008 | 2026-03-15 22:44:38 | 0:01:38 | 5444 | build successful |
| 14 | 77894 | 2026-03-15 18:37:02 | 0:03:01 | 29023 | build successful |
| 11 | 203451 | 2026-03-15 16:12:06 | 0:07:00 | 66168 | build successful |
| 15.arm64 | 39051 | 2026-03-15 16:11:29 | 0:01:32 | 15708 | build successful |
| 15 | 48937 | 2026-02-07 12:49:23 | 0:02:03 | 15526 | failed install-port |
| 15.arm64 | 36619 | 2026-02-07 12:46:33 | 0:02:10 | 14721 | failed install-port |
| 10.6.x86_64 | 260407 | 2026-02-04 20:34:14 | 0:00:11 | 77409 | failed install-dependencies |
| 10.6 | 223148 | 2026-02-04 14:26:28 | 0:00:30 | 77333 | failed install-dependencies |
| 26.arm64 | 27496 | 2026-02-03 15:29:25 | 0:01:42 | 4323 | failed install-port |
| 15.arm64 | 31436 | 2025-11-22 9:54:37 | 0:02:22 | 12509 | failed install-port |
| 26.arm64 | 21476 | 2025-11-22 1:52:28 | 0:04:11 | 2217 | failed install-port |
| 15 | 43252 | 2025-11-21 20:24:03 | 0:02:14 | 13317 | failed install-port |
| 10.6 | 215208 | 2025-11-21 2:14:40 | 0:00:20 | 75193 | failed install-dependencies |
| 10.6.x86_64 | 252276 | 2025-11-21 2:11:53 | 0:00:06 | 75259 | failed install-dependencies |
| 26.arm64 | 14004 | 2025-10-01 22:59:13 | 0:01:34 | 707 | failed install-port |
| 26.arm64 | 10895 | 2025-09-28 12:17:26 | 0:03:13 | 589 | failed install-port |
| 15 | 38002 | 2025-08-20 4:35:55 | 0:02:18 | 10895 | failed install-port |
| 15.arm64 | 26376 | 2025-08-20 3:16:56 | 0:01:35 | 10083 | failed install-port |
| 10.6 | 207939 | 2025-08-19 12:14:48 | 0:00:21 | 72733 | failed install-dependencies |
| 10.6.x86_64 | 244614 | 2025-08-19 4:32:12 | 0:00:08 | 72789 | failed install-dependencies |
| 15.arm64 | 13454 | 2025-03-14 10:43:01 | 0:01:00 | 5802 | failed install-port |
| 15.arm64 | 7519 | 2025-03-06 1:37:04 | 0:01:14 | 3953 | failed install-port |
| 10.6 | 191184 | 2025-01-11 3:11:36 | 0:00:20 | 67272 | failed install-dependencies |
| 10.6.x86_64 | 226845 | 2025-01-11 3:11:28 | 0:00:16 | 67323 | failed install-dependencies |
| 15 | 25217 | 2025-01-11 3:10:41 | 0:01:58 | 5460 | failed install-port |
| 15 | 17583 | 2024-10-20 11:54:43 | 0:02:09 | 2341 | failed install-port |
| 15 | 13263 | 2024-10-14 4:28:13 | 0:02:15 | 1375 | failed install-port |
| 10.6 | 177194 | 2024-08-07 12:58:22 | 0:00:22 | 61711 | failed install-dependencies |
| 10.6.x86_64 | 212822 | 2024-08-07 12:52:45 | 0:00:16 | 61756 | failed install-dependencies |
| 10.6.x86_64 | 194257 | 2024-03-25 2:43:43 | 0:00:22 | 56581 | failed install-dependencies |
| 10.6 | 158548 | 2024-03-25 2:43:20 | 0:00:28 | 56534 | failed install-dependencies |
| 14.arm64 | 8795 | 2024-01-19 14:38:02 | 0:02:03 | 4221 | build successful |
| 14 | 6614 | 2023-11-03 19:42:06 | 0:04:35 | 1246 | build successful |
| 10.6 | 134016 | 2023-08-30 23:27:32 | 0:00:18 | 48701 | failed install-dependencies |
| 10.6.x86_64 | 169368 | 2023-08-30 23:23:24 | 0:00:13 | 48740 | failed install-dependencies |
| 11.arm64 | 100435 | 2023-08-24 18:33:40 | 0:01:40 | 36112 | build successful |
| 12 | 78938 | 2023-08-19 12:37:51 | 0:04:48 | 24914 | build successful |
| 11 | 124200 | 2023-08-19 10:59:31 | 0:06:01 | 36115 | build successful |
| 10.13 | 202061 | 2023-08-19 7:03:43 | 0:13:35 | 66451 | build successful |
| 10.14 | 176754 | 2023-08-19 6:59:50 | 0:11:05 | 57210 | build successful |
| 10.8 | 145025 | 2023-08-19 6:17:56 | 0:12:12 | 45970 | build successful |
| 10.7 | 156628 | 2023-08-19 6:15:58 | 0:08:53 | 47912 | build successful |
| 10.15 | 152336 | 2023-08-19 6:02:07 | 0:08:51 | 46198 | build successful |
| 10.11 | 232610 | 2023-08-19 5:12:09 | 0:09:00 | 75606 | build successful |
| 10.6 | 131882 | 2023-08-19 5:02:26 | 0:02:35 | 48130 | failed install-port |
| 10.6.x86_64 | 167429 | 2023-08-19 4:49:44 | 0:05:18 | 48169 | failed install-port |
| 10.9 | 242706 | 2023-08-19 4:39:27 | 0:07:42 | 75183 | build successful |
| 10.10 | 236220 | 2023-08-19 4:28:52 | 0:14:09 | 75494 | build successful |
| 10.12 | 241175 | 2023-08-19 3:44:52 | 0:06:06 | 74399 | build successful |
| 13.arm64 | 32244 | 2023-08-19 3:17:46 | 0:01:38 | 10524 | build successful |
| 13 | 45114 | 2023-08-19 2:54:32 | 0:05:12 | 10667 | build successful |
| 12.arm64 | 103422 | 2023-08-19 2:49:45 | 0:01:46 | 24918 | build successful |
| 13.arm64 | 4365 | 2023-03-01 8:36:51 | 0:01:46 | 1994 | build successful |
| 13 | 7379 | 2022-12-27 11:45:27 | 0:05:22 | 1060 | build successful |
| 10.7 | 111158 | 2022-09-01 15:35:19 | 0:05:58 | 34562 | build successful |
| 10.7 | 110609 | 2022-08-28 21:09:43 | 0:00:12 | 34280 | failed install-dependencies |
| 10.11 | 192323 | 2022-08-25 14:28:22 | 0:06:02 | 61978 | build successful |
| 10.6 | 85458 | 2022-08-25 14:20:15 | 0:08:04 | 34502 | build successful |
| 10.10 | 193261 | 2022-08-25 12:09:11 | 0:06:47 | 61858 | build successful |
| 10.8 | 100389 | 2022-08-25 11:53:23 | 0:05:53 | 32331 | build successful |
| 11.arm64 | 63791 | 2022-08-25 7:25:47 | 0:01:34 | 22486 | build successful |
| 12 | 39510 | 2022-08-25 3:23:28 | 0:09:12 | 11307 | build successful |
| 10.6.x86_64 | 119241 | 2022-08-25 1:04:38 | 0:17:55 | 34524 | build successful |
| 12.arm64 | 61523 | 2022-08-24 23:19:54 | 0:03:17 | 11312 | build successful |
| 10.9 | 198582 | 2022-08-24 22:00:54 | 0:08:53 | 61550 | build successful |
| 10.14 | 137322 | 2022-08-24 21:29:33 | 0:09:05 | 43636 | build successful |
| 10.13 | 160622 | 2022-08-24 21:18:32 | 0:13:58 | 52867 | build successful |
| 11 | 82371 | 2022-08-24 20:20:21 | 0:06:59 | 22531 | build successful |
| 10.15 | 110203 | 2022-08-24 19:50:56 | 0:11:04 | 32629 | build successful |
| 10.12 | 196636 | 2022-08-24 19:27:00 | 0:09:04 | 60777 | build successful |