Hi there!<br /><br />

These scripts found here will facilitate organizing and cleaning up the corpus used in the paper:<br /><br />

&nbsp;&nbsp;  "Tradition and Modernity in 20th Century Chinese Poetry"<br />
&nbsp;&nbsp;  presented at the 2013 NAACL Workshop on Computational Linguistics for Literature<br />
&nbsp;&nbsp;  authors Rob Voigt and Dan Jurafsky (robvoigt, jurafsky @stanford.edu)<br />
  &nbsp;&nbsp;http://stanford.edu/~jurafsky/traditionandmodernity.pdf<br />
<br /><br />
First, download and unzip this file:
 <br /><br />
&nbsp;&nbsp;  http://nlp.stanford.edu/robvoigt/chpoetry/shigeku_corpus_builder.zip<br />
<br /><br />
To get the corpus from scratch, run the following in the directory where you unzipped the above:
<br /><br />
&nbsp;&nbsp;  wget http://www.shigeku.com/shiku/xs/xz/zhongguoshige.zip<br />
&nbsp;&nbsp;  ./build_corpus.sh
<br /><br />
This will result in the creation of a 'poems' directory, containing all the<br />
poems in the corpus in UTF-8 encoding, numbered and named by author, as well <br />
as a 'meta' directory, containing a small amount of metadata for each poet<br />
(active decade, poetic 'school' affiliation, name in Chinese characters).<br />
<br />
The poem files are named according to the following convention:<br />
<br />
&nbsp;&nbsp;  poems/[author]_[poem#]<br />
<br />
and the metadata files are named according to the following convention:<br />
<br />
&nbsp;&nbsp;  meta/[author].meta<br />
<br />
Please let us know if you have any questions!