Ayto

01The problem

A club badminton player has no cheap way to find out how fast they smash: the number comes from a radar gun or a multi-camera rig. The repository's one-line description is the ambition — Hawk-Eye for the people who can't afford Hawk-Eye. Ayto is the attempt to get that number from one phone camera pointed at a court.

It is a Python monorepo — shuttle tracking, court calibration, pose and a stroke classifier — with an Expo app on top. This page is about the speed number, because that is where the physics pushes back.

One sentence belongs here rather than at the bottom: every speed figure on this page comes from simulated flights with known ground truth, and none has yet been checked against a real shuttle.

02Frame differences miss the smash

The first version did the obvious thing. Two taps on the court set a pixels-per-metre scale, and speed was how far the shuttle moved between consecutive frames. Every part of that is wrong for a shuttle.

A single scale cannot see motion towards or away from the camera. Worse, drag on a shuttle scales with the square of its speed, so it sheds much of that speed within a few frames of contact. At 30–60 fps, a frame difference measures the shuttle after it has already slowed down. On simulated drag flights, the fastest of the first few frame differences came out 45–86 km/h low. The closest published system, an iPhone at 30 fps with a two-click scale, averaged 66 km/h of error against radar on peak speed and about 10 km/h at the net crossing: the same failure, on real smashes.

Detection had its own version of it. In phone video a shuttle is a motion-blurred blob a few pixels across, which a single-frame box detector cannot separate from clutter; when the old detector found no shuttle, it fell back to the smallest object in the frame. On a shared benchmark a YOLO detector scored F1 68.0 against TrackNetV3's 97.5, so detection is now a TrackNet-style heatmap network over several frames, with YOLO as a fallback. The literature review behind that choice put 45 claims from 26 papers to three adversarial verifiers each; fifteen failed, including both TrackNetV5 papers' claims to beat V3 and V4, so V5 is not used.

03Fit the flight, not the frames

Instead, each flight is treated as a projectile. Flights are cut at hits, found by fusing turns in the shuttle's track with wrist-speed peaks from pose, and the pipeline searches for the launch position, velocity and a per-shot drag constant whose simulated gravity-plus-drag flight, projected through the calibrated camera, lands on the observed pixels. Launch speed is that fitted flight's speed at the hit. Every frame of the flight is evidence about the moment of contact, including the ones taken after the shuttle slowed.

If the fit reprojects worse than 12 px, a cheaper fallback fits the drag law to the first 0.15 s alone. Net-crossing speed is reported alongside, because that is what a radar gun compares with — and radar is not ground truth either: one lab study found it under-read peak smash speed by 9.7% on average against a 500 Hz motion-capture rig.

On simulated flights with 1.5 px of detection noise and drag unknown to ±25%, the fit recovered launch speed to 3.2 km/h median and 7.5 km/h mean error. It costs one to two seconds per flight in pure Python, so it runs after the session; the live overlay is still a frame-difference preview.

04The phone's position is part of the measurement

Calibration is four taps on the court corners, plus the net-post tops when they are visible; a line detector proposes the corners and returns nothing rather than guess. From the corners alone the pipeline recovers a full camera, focal length included. Uncalibrated, it falls back to a rough scale and says so in its output.

The 3D step lifts each detection onto the vertical plane the shuttle flies in, from hitter to next hitter. A camera on the court's centre line sits inside that plane for any shot straight down the middle, so depth is unrecoverable and the code returns no value instead of a wrong one. The plane also has to be right: assuming every shot went straight down the court, for shots angled up to ±29°, gave about 160 km/h of error in simulation against roughly 3 km/h with the true plane. Hence the rule to film from behind a back corner or side-on — where the phone stands is as much a part of the measurement as the code.

05The cleaner that deleted the launch

The October rewrite behind all of this was put through an adversarial review before it was committed. It confirmed 34 defects, 33 of them agreed by at least two of three independent verifiers. The fixes did not make it into that commit.

The worst went straight at the speed number. The trajectory cleaner flags outliers by predicting each frame from a quadratic fitted to its neighbours. At a hit every neighbour lies on one side, so the prediction extrapolates a flight decelerating at tens of g, judged against a threshold set by the calm residuals mid-flight. The first detections after every fast hit were deleted as noise, even when they were perfect, and those are exactly the frames that carry the launch speed. On simulated 324 km/h smashes with 1 px of noise, mean launch error was 127.5 km/h.

Skipping only the edge frames was not enough; the next five were still rejected. It took a cubic local fit and a threshold computed from each frame's own neighbours. Mean error at 324 km/h fell to 16.8 km/h, with clutter rejection unchanged at 80 of 80 across 40 simulated tracks. The tracking and pose fixes carry regression tests named after their findings.

06What it does not do yet

• Nothing has been measured on a real shuttle. Validation against 240 fps slow-motion video is pending.
• The review fixes live on a branch. They pass CI there but are not merged, so main still drops the first frames after a fast hit.
• The app is a shell. Its model loading is simulated and its processing screen shows a demo session; analysis runs in the Python pipeline on recorded video.
• No trained weights are in the repository. There is training code for the detector, the stroke classifier and a learned hit detector, but no models.
• On-device speed is unmeasured. The target is under 15 ms a frame on a Snapdragon 680; nothing has been timed on one.
• Other sports are geometry, not results. Five more have court dimensions and drag constants; the ball drag values are first-order estimates.