diff options
Diffstat (limited to 'corpus/support/README')
| -rw-r--r-- | corpus/support/README | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/corpus/support/README b/corpus/support/README new file mode 100644 index 00000000..fdbd523e --- /dev/null +++ b/corpus/support/README @@ -0,0 +1,2 @@ +Run ./tokenize.sh to tokenize text +Edit eng_token_patterns and eng_token_list to add rules for things not to segment |