Jonathan Tow
|
a7a1fb8a83
|
update(tokenizer): convert to GPT2Tokenizer (#7)
- update(tokenizer): convert to `GPT2Tokenizer` (85625532dc8753c206eecc8a76323783a7b64744)
|
2024-03-01 07:31:02 +00:00 |
|
Jonathan Tow
|
9f2def929c
|
fix: add default special tokens
|
2024-01-19 22:54:48 +00:00 |
|
Jonathan Tow
|
ca32832c44
|
fix: add default special tokens
|
2024-01-19 22:52:04 +00:00 |
|
Jonathan Tow
|
192936bf86
|
fix: revert zephyr tokenizer config change
|
2024-01-19 22:43:56 +00:00 |
|
Max
|
507254d7bd
|
fix: add chat template to tokenizer config
|
2024-01-19 22:01:18 +00:00 |
|
jon-tow
|
3aeae29673
|
init: release
|
2024-01-18 23:46:52 +00:00 |
|