Signal Backtests
How Orrery's price-based signals performed historically, replayed on the top 15 markets by volume. Transparent methodology, no cherry-picking.
History
1M CLOB
Universe
Top 15
Cooldown
1h
Cost model
-2pp
Orrery replays the same live rules minute by minute, then measures 1h, 6h, and 24h follow-through. Flow and resolution-risk need event-time history, so they are excluded for now.
Momentum
326 firings across 7 marketsPrice + 1h direction + 24h direction all aligned, with ≥3pp 24h move. Expected to continue in the same direction.
Expected · 24h, after spread
-5.96pp
raw avg − 2pp typical spread
Median · 24h
-0.65pp
robust to outliers
Win rate · 24h
43%
317 measured
Raw avg · 24h
-3.96pp
before spread / fees
Average move by horizon
- 1h-1.01pp
- 6h-2.73pp
- 24h-3.96pp
Centred at zero. Bar length is proportional to the largest of the three windows, so the decay shape is visible without re-scaling.
By category (3 categories fired)
| Category | Firings | Measured | Win 24h | Avg 24h |
|---|---|---|---|---|
| Geopolitics | 277 | 268 | 38% | -4.59pp |
| Science | 39 | 39 | 64% | -2.54pp |
| Politics | 10 | 10 | 90% | +8.20pp |
Win rate hidden until ≥10 measured firings — thin samples can flip with one outlier.
Per-market breakdown (7 markets fired)
| Market | Firings | Win 24h | Avg 1h | Avg 24h |
|---|---|---|---|---|
| Strait of Hormuz traffic returns to normal by end of May? | 97 | 58% | -0.28pp | +1.78pp |
| Will Trump say "Nuclear" during events with Xi Jinping? | 61 | 33% | -3.73pp | -12.52pp |
| Will Trump say "Iran" during events with Xi Jinping? | 53 | 23% | -1.25pp | -8.97pp |
| Israel x Syria security agreement by June 30? | 52 | 14% | -1.27pp | -3.99pp |
| Trump declassifies new UFO files by May 15? | 39 | 64% | +2.16pp | -2.54pp |
| Iran closes its airspace by May 18? | 14 | 62% | -0.76pp | +0.11pp |
| Will Trump say "Dumbocrat" during Bret Baier interview? | 10 | 90% | -1.71pp | +8.20pp |
Divergence
182 firings across 7 markets1h move running against the 24h trend. Expected to continue short-term in the 1h direction before 24h reasserts.
Expected · 24h, after spread
+2.97pp
raw avg − 2pp typical spread
Median · 24h
+0.50pp
robust to outliers
Win rate · 24h
52%
177 measured
Raw avg · 24h
+4.97pp
before spread / fees
Average move by horizon
- 1h+0.12pp
- 6h+2.58pp
- 24h+4.97pp
Centred at zero. Bar length is proportional to the largest of the three windows, so the decay shape is visible without re-scaling.
By category (3 categories fired)
| Category | Firings | Measured | Win 24h | Avg 24h |
|---|---|---|---|---|
| Geopolitics | 164 | 159 | 57% | +6.09pp |
| Science | 13 | 13 | 15% | +0.98pp |
| Politics | 5 | 5 | — | -21.36pp |
Win rate hidden until ≥10 measured firings — thin samples can flip with one outlier.
Per-market breakdown (7 markets fired)
| Market | Firings | Win 24h | Avg 1h | Avg 24h |
|---|---|---|---|---|
| Strait of Hormuz traffic returns to normal by end of May? | 45 | 33% | +0.06pp | -2.14pp |
| Israel x Syria security agreement by June 30? | 41 | 64% | -0.33pp | +1.83pp |
| Will Trump say "Iran" during events with Xi Jinping? | 37 | 73% | +0.49pp | +14.98pp |
| Will Trump say "Nuclear" during events with Xi Jinping? | 30 | 66% | +1.59pp | +14.84pp |
| Trump declassifies new UFO files by May 15? | 13 | 15% | -0.22pp | +0.98pp |
| Iran closes its airspace by May 18? | 11 | 44% | +0.40pp | +1.83pp |
| Will Trump say "Dumbocrat" during Bret Baier interview? | 5 | 0% | -6.79pp | -21.36pp |
Honest caveats
- Small samples. CLOB returns ~30 days of minute-level history; short-dated markets may have very few firings. Anything below 10 measured firings shows a dash instead of a win rate so we don't overstate confidence.
- Survivorship. This backtest runs on markets currently live — not on markets that already resolved and disappeared. Persistent historical coverage is planned so the full universe can be evaluated once enough live history has accumulated.
- No execution costs. Avg-move figures are mid-price to mid-price. Real entry/exit would pay spread, slippage, and Polymarket fees — so a +1.5pp 24h move is not +1.5pp of profit.
- Signal drift. If we tune the live rule, these numbers change with it. That's the point — the page always reflects the rule that's currently running, not a frozen historical version.