diff options
| author | Patrick Simianer <p@simianer.de> | 2016-07-05 11:01:46 +0200 |
|---|---|---|
| committer | Patrick Simianer <p@simianer.de> | 2016-07-05 11:01:46 +0200 |
| commit | 2b1d7f881c19c4d4b5afae194e02d3300c7675d0 (patch) | |
| tree | 5a06ee7de98640a39244b57bb369697176b44ebf /preprocess-no-lower | |
| parent | 69949dda35c3ea21d8e926e5f0a596a0a0f61c6a (diff) | |
mv
Diffstat (limited to 'preprocess-no-lower')
| -rwxr-xr-x | preprocess-no-lower | 9 |
1 files changed, 9 insertions, 0 deletions
diff --git a/preprocess-no-lower b/preprocess-no-lower new file mode 100755 index 0000000..afd87e9 --- /dev/null +++ b/preprocess-no-lower @@ -0,0 +1,9 @@ +#!/bin/bash + +pushd `dirname $0` > /dev/null +P=`pwd -P` +popd > /dev/null + +LANG=$1 +$P/no-non-printables | sed "s|[-,\.]\{4,\}|...|g" | $P/htmlentities 2>htmlentities.$LANG.err | $P/normalize-punctuation 2>normalize-punctuation.$LANG.err | $P/tokenizer-no-escape.perl -a -b -threads 1 -l $LANG 2>tokenizer.$LANG.err + |