description: Roberta trained from scratch with 50 GB filtered data (checkpoint 94000 from 202000)
tags:
  - roberta
params:
  epochs: 1