The gap between 96.7% on the DeepDarts split and 33 of 42 in your own room is the most useful number in this piece, and it is the one most write-ups quietly leave out. We hit the same wall from the benchmark side rather than the vision side: we pulled the configs behind 841 LLM benchmark runs to see how many pin the exact revision of the data they grade against, and exactly one did. So most published scores are measured against a target that moves underneath them, which is your DeepDarts-versus-living-room gap made permanent and unfalsifiable, because nobody can say which version of the test was passed. Your note that the last thirty epochs move noisily because one hard photo shifts a small validation set matched something we measured too: we repeated a single identical measurement 50 times over one fixed corpus and 21 of the 50 runs flipped the sign of the answer, from sampling alone. A small eval set does not give you a slightly noisy number, it gives you a number whose direction is undetermined, which is why "87% of images with every tip right" and "11 of 15 darts" can both be honest. The design choice I would steal is the heatmap over three fixed coordinates, because "there is only one dart right now" is precisely what a fixed-arity output cannot say, and the failure mode of the alternative is a confidently wrong point rather than a faint smudge. When you fine-tune on the turns the app confirms, are you holding out a slice of your own camera angles as their own validation split, or still validating against DeepDarts? A held-out slice of living-room data seems like the only number that would actually move when the thing you care about moves.








