Demo -- read-only snapshot of the live system as of 2026-07-16T01:45:31Z; no live data, paper units only.
Skip to main content

evidence / Self-refutation & honesty

Leakage-Catching Instruments -- the harnesses built to refute my own results

Leakage is the default failure mode of sports ML. Every result in this repo passes through purpose-built instruments designed to refute it, not confirm it. The single truth-source for any figure is docs/JOB_EVIDENCE_PACKET.md (sections 2C and 2E). For what these instruments actually caught -- the retracted headline numbers -- see the retraction story.

strongest single receipt

In sports forecasting, a good-looking number is almost always a leak until proven otherwise: a feature that peeks at the future, a grader that reads the market instead of the model, a grid search that memorizes its holdout, a single lucky calendar window

the claim

In sports forecasting, a good-looking number is almost always a leak until proven otherwise: a feature that peeks at the future, a grader that reads the market instead of the model, a grid search that memorizes its holdout, a single lucky calendar window. So the machinery that matters is not any metric -- it is the set of instruments that assume every result is wrong and try to break it. This page documents those instruments and the exact committed path for each. What they caught is told in retraction-story.md; this page is the how.

receipts

claimcommitted artifact
Walk-forward CV + per-fold assertionsrc/prediction/walk_forward_backtester.py
Overfit-gap CI gatescripts/run_walk_forward.py --gate
Truncation-invariance testtests/test_ingame_leak_free.py
Multi-corpus calibration gatescripts/validate_calibration_multicorpus.py
Corrective regularizationsrc/prediction/prop_cv_split.py
Shadow logger + settlementsrc/prediction/shadow_logger.py, src/prediction/settlement.py
Ship gate (permutation + ablation + FDR)src/loop/gate.py

reproduce

reproduce on a fresh clone
# Truncation-invariance leak test for streaming features
python -m pytest tests/test_ingame_leak_free.py -q

# Multi-corpus calibration acceptance gate (its tests)
python -m pytest scripts/validate_calibration_multicorpus.py -q

# Walk-forward CI gate: exits nonzero on overfit
python scripts/run_walk_forward.py --gate
no edge claimed

This site reports calibration and sharpness only, never a dollar edge, ROI, or bankroll result. An honest null is a success.