580 B
580 B
Text corbus
based on Wikipedia EN Latest Articles -- enwiki-latest-pages-articles.xml.bz2 created with wiki.sh script
Code
- include only article text in between marks
- exclude lines without alphabet letters
- clear excessive white spaces
- remove special characters
- remove unintentional empty lines
Stats
No lines
wc -l
Size
du -sh enwiki-latest-corpus.txt.bz2
Head of file
bzcat enwiki-latest-corpus.txt.bz2 | head -n 5
Random lines from file
bzcat enwiki-latest-corpus.txt.bz2 | shuf -n 5