scripts
=======

A number of NLP related scripts.
Some scripts require my zipf gem,
see https://github.com/pks/zipf

compound-splitter.perl and tokenizer.no-escape.perl
taken from the moses [1] toolkit.


[1] https://github.com/moses-smt/mosesdecoder