description: Roberta trained from scratch with 50 GB filtered data (checkpoint 94000 from 202000)
tags:
- roberta
params:
epochs: 1