research data . Dataset . 2019

Magi Practical Web Article Corpus

Ji, Yichao; Liu, Xinyang; Ma, Kui; Zhao, Xuezhi; Sun, Qiao;
Open Access Chinese
  • Published: 01 Jun 2019
  • Publisher: Zenodo
Magi Practical Web Article Corpus (MPWAC) contains 10 million Chinese web articles consisting of more than 10 billion words. Articles have been shuffled to mitigate the negative influence on mini-batch based training. Each article has been refined such that only the main body of the text is left, without advertisements or other noises. This corpus is extracted in May 2019, by crawlers located inside Mainland China. The merged file should be processed with Gzip stream. It is encoded in UTF-8, and has the format of '0xFE [URL] 0xFF [TEXT] ...'.
free text keywords: Magi, Natural Language Processing, Information Retrieval, Web Crawl, Corpus, Chinese, NLP
  • Digital Humanities and Cultural Heritage
Download fromView all 2 versions
Open Access
Dataset . 2019
Providers: Datacite
Open Access
Dataset . 2019
Providers: ZENODO
1 research outcomes, page 1 of 1
Any information missing or wrong?Report an Issue