The order you do it in
The lorry brings the crates. The clerk chalks a word on each one. What comes next?
Loading lab…
- Collect, label, split, learn, measure. In that order.
- In order: measure on the kept-back tray. Score 4 of 8.
- Swap: measure runs before split. Score 24 of 24.
- Same program, same crates. The order changed the count.
The rule
Next comes the split. The kept-back crates go onto a tray for testing. A count that flatters looks better than the program really is.
Collect, label, split, learn, measure — in that order. Measure on the examples it learned from, and the count flatters. Measure on the kept-back tray, and it means something.
The 24 crates are split. Eight go onto a tray. The program learns from the other 16. Then the tray is measured. Measured on the learn pile instead, the program copies their words.
| Order | Measured on | Score |
|---|---|---|
| Correct order | Kept-back tray, 8 crates | 4 of 8 |
| Measure before split | Everything it holds, 24 crates | 24 of 24 |
Skip the label step and the crates carry no word. Then there is nothing to compare each guess with. The measuring step cannot run. Not being able to measure is not the same as scoring 0.