blob: 44dccc6f84639024ce7d9ea9282563c292af7465 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
First column = tune iteration, second column = tune BLEU
Choosing the average of a few random sentence's model->hope direction, With Vlad's default --bleu_weight=1, nothing improved, and convergenced seemed to be slower, which is the opposite of what we intended. 5 random directions and the orthogonal directions were still used (fewer than normal random directions; the oracle directions replaced some of them).
dev.bleu.baseline.txt - usual search directions (orth+random)
dev.bleu.oracle_mira_direction.txt - as above, but fewer random directions, 10 model->hope directions
note that the search directions are recomputed after each MERT iteration.
/home/jgraehl/tune/urdu/vest-dth-oracle*
~/tune/urdu$ ls -d vest-dth*
vest-dth vest-dth-oracle vest-dth-oracle-bleuwt=0.1 vest-dth-oracle-bleuwt=10 vest-dth-oracle-bleuwt=100
oraclewt devtest-estimated-BLEU
0 22.01 (orth. dirs, 10 random dirs)
0.1 22.13 (orth. dirs, 10+10 model->hope and fear->hope random directions, 5 purely random dirs)
1 22.06 "
10 21.85 "
100 22.08 "
no evidence of any advantage using oracle directions (TODO: decode mt09 as test set, check for any advantage when using many features)
|
