Restart Lab
A set-piece simulator built so its own results can be checked. It plays a corner out thousands of times, scores every chance with an expected-goals model trained on real matches, and will not call a routine a winner until it has beaten a random search of the same size.
What it had to do
Football clubs spend a great deal of time on set pieces and have very little evidence about which routines actually work. The obvious approach, counting what happened in past matches, runs out of data almost immediately, because any specific routine against any specific defence has been tried a handful of times at most.
So the problem was not really about football. It was about building a tool that could answer a question the data was too sparse to answer directly, without manufacturing confidence the evidence could not support.
What I chose
Simulation over statistics. A deterministic physics engine flies the ball with real spin, drag and bounce, moves every attacker and defender, and resolves the delivery into a shot, a clearance or a scramble. That turns a handful of real attempts into thousands of synthetic ones.
Every simulated chance is then graded by an expected-goals model trained on StatsBomb's open World Cup and Euros data, so the scoring is anchored to real football and not to my own judgement.
The decision I care most about is the last one. An optimiser searches for better routines, but nothing it proposes counts unless it beats a random search on the same budget. Optimisers are good at finding noise and reporting it as a result, so the baseline is not optional anywhere in the pipeline.
How it works
Three steps, one loop. Simulate the set piece, measure each chance with the xG model and report it with confidence intervals, then search the space of deliveries, runs and timings across thousands of trials.
Two consoles sit on top. The Scenario Workbench builds a set piece from real squads and compares two routines under common random numbers, so the comparison isn't polluted by luck. Optimization Studies reads a completed search: convergence against the baseline, a parallel-coordinates view of every trial, and a SHAP explanation of what separated the winners.
Results are reproducible on purpose. Every stored result carries the engine build id, and the same scenario and seed produce a byte-identical outcome whatever the batch size, so a stale result is detectable instead of silently wrong.
What it cost
Speed, first. The engine is slower than I want it to be, and the throughput kernel, a Numba port of the inner loop checked to 1e-9 against the readable NumPy reference, arrived after the platform instead of before it. The live demo works, but it is slower than it should be.
On results, the machinery does the job it was built for. The study committed to the repository, England attacking an Argentine zonal six-two, puts its best confirmed routine at 6.66% mean xG per corner against a library baseline of 1.04%, over 400 confirmation simulations. The 95% intervals do not overlap, and the winning routine stays the winner under every ±10% perturbation of player attributes. It also had to out-run an equal-budget random search before any of that counted, which is the harder of the two gates.
That is one fixture and twenty-four trials. The absolute numbers are worth precisely as much as the simulator that produced them, which is why the interface reports an interval and never a single figure, and why the banner still refuses to crown anything outside the conditions it was tested in.
What I'd change
I would build the throughput kernel first. Almost every limitation downstream traces back to how fast a single simulation runs: how many trials a study can afford, how tight the confidence intervals get, how much of the space the optimiser can explore. I optimised the interesting part before the boring part, and the boring part turned out to set the ceiling.