diff options
| author | Patrick Simianer <p@simianer.de> | 2014-06-14 14:43:14 +0200 |
|---|---|---|
| committer | Patrick Simianer <p@simianer.de> | 2014-06-14 14:43:14 +0200 |
| commit | 2783f837303ae07c4a1d676302bca779abbb1296 (patch) | |
| tree | e388dda12d6d31285b32663b937a8d55ecc909c5 /nonbreaking_prefixes/nonbreaking_prefix.sl | |
| parent | 85ea0fc5e3ae7ea646cc6e843d01939b4d8e4dbf (diff) | |
steal tokenizer from moses' scripts
Diffstat (limited to 'nonbreaking_prefixes/nonbreaking_prefix.sl')
| -rw-r--r-- | nonbreaking_prefixes/nonbreaking_prefix.sl | 78 |
1 files changed, 78 insertions, 0 deletions
diff --git a/nonbreaking_prefixes/nonbreaking_prefix.sl b/nonbreaking_prefixes/nonbreaking_prefix.sl new file mode 100644 index 0000000..230062c --- /dev/null +++ b/nonbreaking_prefixes/nonbreaking_prefix.sl @@ -0,0 +1,78 @@ +dr
+Dr
+itd
+itn
+št #NUMERIC_ONLY#
+Št #NUMERIC_ONLY#
+d
+jan
+Jan
+feb
+Feb
+mar
+Mar
+apr
+Apr
+jun
+Jun
+jul
+Jul
+avg
+Avg
+sept
+Sept
+sep
+Sep
+okt
+Okt
+nov
+Nov
+dec
+Dec
+tj
+Tj
+npr
+Npr
+sl
+Sl
+op
+Op
+gl
+Gl
+oz
+Oz
+prev
+dipl
+ing
+prim
+Prim
+cf
+Cf
+gl
+Gl
+A
+B
+C
+D
+E
+F
+G
+H
+I
+J
+K
+L
+M
+N
+O
+P
+Q
+R
+S
+T
+U
+V
+W
+X
+Y
+Z
|