diff options
author | Chris Dyer <cdyer@allegro.clab.cs.cmu.edu> | 2014-03-10 18:40:13 -0400 |
---|---|---|
committer | Chris Dyer <cdyer@allegro.clab.cs.cmu.edu> | 2014-03-10 18:40:13 -0400 |
commit | 284383880f043edb2d67afbe2f64237c466245c1 (patch) | |
tree | 2751eaccdfc82586cf8aad1cfe257dbc3cf27436 /corpus/support/token_list | |
parent | 8c2d8217f068d8f107f95171496a25013d4e35fe (diff) |
few tokenization bugs
Diffstat (limited to 'corpus/support/token_list')
-rw-r--r-- | corpus/support/token_list | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/corpus/support/token_list b/corpus/support/token_list index 228663f6..d38638cf 100644 --- a/corpus/support/token_list +++ b/corpus/support/token_list @@ -156,8 +156,9 @@ Mass. Md. Mfg. Mgr. -Mexican-U.S. -Mich. +Mio. +Mrd. +Bio. Minn. Mo. Mon. @@ -187,6 +188,7 @@ Rd. Rev. R.J. C.L +Rs. Rte. Sat. W.T |