On fractures kept out of every decision, pretraining still wins, and it is not just extra training.
Since 26 October, 88 of the 170 real fractures have been sealed. No model is tuned on them, and each finished model is scored on them once.
Training from scratch three times longer helps, but it only ties a pretrained model trained a third as long. Fine-tuning on the 82 working cases then gives the best result so far: 6.6 mm.
How it was testedQuestions and pass marks written before each run (Routes 17 and 18). Two independent training runs per model. Both gains in the caption have 95% intervals above zero in both runs.
Show the numbers
| Model | Read-out | Runs 1 / 2, mm |
|---|---|---|
| From scratch, 20,000 training updates | one pass | 9.03 / 9.12 |
| From scratch, 60,000 updates | one pass | 7.96 / 8.00 |
| Pretrained, 20,000 updates | one pass | 7.82 / 7.87 |
| Pretrained, 60,000 updates | one pass | 7.21 / 7.33 |
| Pretrained with a refinement pass, 20,000 updates | three passes | 7.31 / 7.10 |
| … then fine-tuned on the 82 working cases | three passes | 6.72 / 6.51 |
| Paired comparison, same 88 cases | Run 1 [95%] | Run 2 [95%] |
|---|---|---|
| Pretrained ahead of scratch, both 60,000 | 0.75 [0.43, 1.05] | 0.67 [0.35, 0.97] |
| Scratch at 60,000 against pretrained at 20,000 | −0.14 [−0.37, 0.10] | −0.13 [−0.34, 0.10] |
| Fine-tuning on real cases adds | 0.59 [0.18, 1.08] | 0.59 [0.20, 1.04] |
Each model is read out the way its pre-registration named: one pass, or three refinement passes for the refinement models. Compare rows within the same read-out where it matters.