Adds tokenizer_config.json file

This commit is contained in:
Lysandre 2024-02-16 09:38:02 +00:00 committed by system
parent 38cc92ec43
commit 224ef41937
No known key found for this signature in database
GPG Key ID: 6A528E38E0733467

1
tokenizer_config.json Normal file

@ -0,0 +1 @@
{"model_max_length": 1024}