Prediction market backtesting: a rule, a price history, and stated costs

What is replayed

A published price series for the contracts in the bot's universe, stepped at a fixed interval, with the rule evaluated at each step. Positions open when the entry condition is met, and close on a stop, a target or the holding limit — whichever comes first. Open positions at the end of a run are marked at the last price rather than assumed to settle.

The run's bankroll, position size and maximum concurrent positions are part of the rule set, because a rule that only works at a size the market cannot absorb has not been tested, it has been imagined.

Where the assumptions bite hardest

Prediction markets are thin, priced between zero and one, and charged per contract. Those three facts together mean that costs are proportionally heaviest in the middle of the price range — which is where most momentum and mean-reversion rules want to trade. A backtest that ignores fees on this asset class does not have a small error; it has the wrong sign.

Slippage is not modelled at all, which is stated rather than hidden. Adding a slippage assumption without a way to calibrate it would be a guess dressed as a correction.

Availability

The published replay is free to read here. Running your own backtest is in private preview: it is not on any plan, and it is not sold. When that changes, the plan row on this page is generated from the same entitlement map the price list uses, so it will change with it.

The four assumptions that decide the answer

Fill price. This replay fills at the price printed in the published series, which assumes a counterparty was there. On a thin contract that is generous, and it is the single most optimistic thing in the exercise.

Cost. A flat rate is charged on entry and exit. Real fees differ by venue, by whether you took or made, and by the price of the contract, and on this asset class they are large relative to the edges these rules pursue.

Size. Positions are a fixed notional against a fixed bankroll, with a cap on how many can be open. A rule that only works at a size the book cannot absorb has not been tested.

Time. The rule is evaluated at a fixed interval rather than continuously, so a move that happened and reversed between two steps is invisible to it. That cuts both ways: it removes some fictional entries and it misses some real ones.

The mistakes a published run makes harder

Choosing the window after seeing the result. The run's start date is published before it starts, so the period cannot be selected to flatter the rule.

Quietly dropping the losers. The record keeps them. A library of replays that only contains successes tells you about the librarian rather than about the strategy.

Tuning until it works. Every parameter is stated in advance; a changed parameter produces a new run with its own start, not a better version of the old one.

Reading a small sample as a finding. The number of closed trades is printed in the same row as the ratios computed from it, which makes the sample size hard to skip past.