projekt-glebokie/models/gpt2/special_tokens_map.json
Andrzej Preibisz a5fb14928c GPT-2
2023-02-12 20:03:40 +01:00

7 lines
131 B
JSON

{
"bos_token": "<|endoftext|>",
"eos_token": "<|endoftext|>",
"pad_token": "<|endoftext|>",
"unk_token": "<|endoftext|>"
}