Retraction record
Every figure this project withdrew or could not verify, copied from section 4 of the evidence packet (interview-coaching sentences and return figures are left out of the replacement wording; the long tracking row is shortened to three sentences). Each row names the figure, why it fails, the wording that replaces it and a date. The largest withdrawal is the former headline return figure: it was a market-follow artifact, and the honest replacement is calibration: the pregame model trails the devigged close on MLB moneyline and soccer O/U 2.5, and on NBA moneyline the interval includes zero, so we cannot tell. The six withdrawn figures sit in four rows, each marked Retracted. A retracted number appears on this site only inside retraction framing: here and the methodology do-not-claim list. Source: docs/JOB_EVIDENCE_PACKET.md, section 4.
| Status and date | Do not claim | Why it fails | Say instead (replacement) |
|---|---|---|---|
| Retracted 2026-07-23 | "+18.38% ROI on 1,535 walk-forward bets vs real closing lines" (also +15.04% flat, and per-stat splits BLK +26% / STL +17% / etc.) | Market-follow artifact, confirmed at the source-code level. The grader picks bet direction from devig(over_odds, under_odds) — the market's own lean — and never reads the model (the eval CSV has no prediction column); prices at a flat -110 fiction; filters tuned in-sample on the same file. | "The pregame model trails the devigged close on MLB moneyline and soccer O/U 2.5; on NBA moneyline the interval includes zero, so we cannot tell." |
| Retracted 2026-07-23 | "endQ3 in-play Brier 0.1191, inside Pinnacle's range" | Leak-inflated AND mis-sourced. Fed two Q4-derived features (peeks at the predicted quarter); cited file actually reports 0.1354. | "Leak-free walk-forward endQ3 Brier ~0.141, after I removed a Q4 feature leak I found in my own pipeline." |
| Retracted 2026-07-23 | "+54.57% ROI / 78.11% hit on 55,073 in-play bets" | Graded against an L5 line proxy, not real closing lines. A model-quality ceiling, not a tradeable result. | "In play the model trails the Kalshi mid; the L5-proxy backtest is a model-quality ceiling, never a realized result." |
| Retracted 2026-07-23 | "Aggregate CLV +8.94pp" | Circular — computed on the same model-unused, devig-direction corpus. No real Pinnacle-close CLV exists yet; first reading dated Oct 2026 (section 5: as of 2026-10-23 the forward paper series computes no markout or CLV). Full-season backtest shows CLV ≈ 0 vs real closes. | Don't quote a CLV figure. "Real closing-line CLV can't be measured yet; I built the methodology that will measure it." |
| Retired not dated in the packet row | "Re-ID accuracy ~91%, homography RMSE ~4.2px, 28 FPS, ball-valid 87%" | Unbacked. The referenced benchmark test does not exist. | Present as roadmap targets, never achieved metrics. Cite the self-consistency tracking-quality gates instead. |
| Retired not dated in the packet row | "Tracks 10 players at 15 fps" | Observed ~1.6–2.6 tracked players/frame (max 4) on real broadcast footage; ~7–11 fps. | "The detector finds all 10 players; the tracker maintains up to ~5–6 stable slots on the calibration clip; reliable 10-player broadcast tracking is not yet demonstrated." |
| Retired not dated in the packet row | "Position accuracy ±12–18 inches" / "0 ID switches" / "track stability 1.0" | No ground-truth labels exist; evaluate.py returns self_evaluation: True, not validated MOT metrics. | "Outputs court coordinates via homography; positional accuracy and MOT metrics are not yet benchmarked against labeled ground truth." |
| Retired 2026-09-04 | "The tracker reliably follows players" / any claim that tracking OUTPUT is trustworthy | Measured on the research route, 2026-09-04, not asserted: on one WNBA clip through a validated homography, 10.5 pct of on-court same-ID steps are physically impossible (G267/G270). Only about one retained detection in five sits on a player's feet (G287), detector recall is not established, and the impossible-step rate has no established mechanism. Scope is one clip, one camera shot and one labeller; the full measurement record with receipts G267 to G298 is in section 4 of the packet. | "Tracking output quality is now measured rather than assumed, and it is poor." |
| Retired 2026-09-04 | "Court calibration generalises across footage" | Falsified on the research route, 2026-09-04. Seeded calibration works on wnba__wnba_01 (5 px median / 19 px p90) but yields NCAA 0/300, amateur 0 usable across 5 sources, soccer 0/1,195. Causes are camera framing, occlusion and court decoration, not detector quality. Separately, no hand-built signal detects an invalid court: four independent families failed, and G274 produced 0.569 px RMS on a frame containing no court at all. | "Calibration is hand-seeded per clip and does not generalise; sub-pixel reprojection residual is NOT evidence a map is valid. About two thirds of the one measured clip is court-geometry-bearing (upper bound 0.722, no measured floor), so the blocker is calibration validity, not footage availability." |
| Retired not dated in the packet row | "CV features are a predictive moat / edge sportsbooks lack" | Zero measured predictive value today — every CV feature SHAP importance = 0.0 in production prop models; cv_lift_report.json is has_cv_data: false. | "CV-derived features are wired in as a potential future edge; they don't yet move the model (SHAP ~0). Credible thesis and complete plumbing, not a demonstrated advantage." |
| Retired not dated in the packet row | "I trained a deep re-ID model" | The shipped OSNet weights are ImageNet-pretrained, not NBA-fine-tuned; production appearance model is the HSV histogram. | "I reimplemented the OSNet architecture in PyTorch and run it with ImageNet-pretrained weights." |
| Retired not dated in the packet row | "Built over 13 months" | Not supported by git — history spans ~6 months (2026-03-09 to 2026-09-07). | "An intensive solo build (5,504 commits, Mar–Sep 2026)." Drop the 13-month figure entirely. |
| Retired not dated in the packet row | "Solo-built / I wrote 5,504 commits" (implied hand-typed, the 2026-09-07 count) | The large majority of commits are agent-authored under direction. NOTE 2026-09-07: the identity-based recount no longer reproduces -- a history rewrite normalised every author to Neel Shah (5,504/5,504), so the old ~91% GSD Executor split is not re-derivable from git log --format=%an today; the still-derivable trailer figure is 1,333/5,504 = 24.2% carrying Co-Authored-By: Claude. | "Solo human architect/director of an agentic build pipeline. The engineering judgment, ship/reject decisions, and validation methodology are mine." |
| Retired not dated in the packet row | "70 iterations / 29 ships / 41 reverts" as a code-artifact or ledger count | Only ~39 iter files on disk; exactly one revert doc exists; no machine-checkable ledger backs the 29/41 split. | "Ran a documented iteration campaign; most candidates were correctly rejected by the gate, by design." |
| Retired not dated in the packet row | "All ~7,400 tests pass / full suite is green" | Not currently true — ~97–98% pass locally with a documented tail (DB/GPU/optional-dep/version drift), a few real logic-drift failures, and a native pyarrow segfault on Windows. CI enforces only a 30% coverage floor. | "~7,400 tests, ~97–98% passing, with a documented tail tracked in KNOWN_LIMITATIONS. Core betting-math and in-play subsets pass clean." |
| Retired 2026-07-23 | "Run my verify scripts to reproduce the headlines" | AMENDED 2026-07-23: the historical failures (verify_production_mae.py feature mismatch; verify_winprob.py uncommitted cache) were fixed 2026-07-20 (eb95e13b8), and fresh-clone reproducibility is now measured, not assumed: check_all.py passes 52/52 on a bare clone (8 data-dependent modules verify their committed artifacts in a labeled recorded-artifact mode) and runs green in public CI (proof.yml, first green run 2026-07-23; the first run honestly FAILED 46/52 on a path-portability bug, fixed in 03c4ce398 -- the red run stays in Actions history). | "Clone the repo and run check_all.py -- 52 module self-checks pass on a bare clone and in public CI (REPRODUCE.md). The CI trail includes the first red run and its fix." |
| Retired not dated in the packet row | Any "quant alpha / P&L track record" framing | There is no verified leak-free profitable edge. Zero real money placed. | "I built the full quant toolchain — devig, Kelly, CLV, calibration, walk-forward backtesting — and ran the validation that proved the apparent edge was an artifact." |
| Retired 2026-09-03 | "85 trained signals" / "0/85 candidate signals" / "80-artifact intelligence layer" | CORRECTED 2026-09-03 (S56). No artifact holds 85 signals: 85 is the per-row n_trials multiplicity constant, while scripts/platformkit/eval_gate/spa_catalog_report.txt and retro_correction_report.txt both enumerate 60 catalog classes (NBA 16 / soccer 15 / tennis 15 / MLB 14), 0 surviving. The disjoint data/registry/signal_registry.parquet holds 86 rows (72 folded / 14 deferred, coverage_pct null 86/86, no market-relative verdict). data/intelligence/ holds 151 files on disk and the 80-subset is not enumerable by any script. Memo: docs/evidence/harness/SIGNAL_INVENTORY_REDTEAM_2026-09-03.md. | "0 of 60 candidate signal classes survived the multiplicity correction; 86 registry signals carry no market-relative test; the intelligence layer is 151 files on disk." |
| Retired 2026-10-19 | "A forward track record" / "N weeks of forward paper trading" / any forward-series Brier as a model verdict (added 2026-10-19) | Through 2026-10-22 every scoreboard cell is INSUFFICIENT (1 to 6 games against the 30-game floor), 0 of 61 judged games passed qualification, 9 of 13 night receipts are PARTIAL, and the fitted arms scored ticks on two nights only (10-06 and 10-09), none since. Arm A is the venue mid's own Brier, not the model's. Section 5. | "A prospective, receipt-backed paper series has run since 2026-10-06. It has no calibration verdict yet, and the receipts say why: scorer wall time, laptop sleeps and recorder uptime." |
| Retired 2026-10-19 | "Simulated fills" as execution quality or P&L (added 2026-10-19) | fill_realism is COUNTS ONLY on unqualified tape with zero assumed latency. It reports no markout, no fee netting and no money figure. | "The fill-realism step counts which paper quotes would have been touched (188 at a 5 s quote life, 732 at 30 s, 10-08 to 10-22). It is a count, not a result." |
| Retired 2026-10-19 | "The NBA season is pre-registered forward" (added 2026-10-19) | No NBA-specific pre-registration document exists. NBA cells are scored under the general PREREG_2026-10-07.json seal (arm A and B_lag only so far, every cell INSUFFICIENT) and the fitted NBA arms are in warmup until about 20 regular-season games settle. Say "scored under the 2026-10-07 seal, fitted arms warmup", never "a pre-registered NBA season" or any NBA forward verdict. | "NBA is served and scored against the mid from the opener. A fitted NBA arm needs a new dated seal." |
Proof files named in the table are in the public repository when they sit under docs/, kernel/ or scripts/platformkit/eval_gate/; any other path named there is in the private repository.