Brier Score Explained

The Brier score is a single number that measures how accurate a set of probability forecasts turned out to be. It ranges from 0 (perfect) to 1 (worst possible), and lower is always better. It's the standard way statisticians and forecasters grade "how good were your probabilities," and it's one half of what powers our Truth Meter.

The formula, in plain terms

For a single forecast, the Brier score is the squared difference between the probability you gave and the actual outcome (1 if it happened, 0 if it didn't):

Brier score = (forecast probability − outcome)²

Across a batch of forecasts, you average that squared difference over every prediction. So if you say "70% chance" on ten different setups and seven of them hit, your Brier score for that batch will be low (good). If you say "70% chance" and only three hit, the score climbs (bad) — the squaring punishes confident forecasts that turn out wrong harder than it punishes cautious ones.

Why squaring matters

Squaring the error does two things: it makes the score always positive (so errors in different directions don't cancel out), and it penalizes overconfidence disproportionately. Saying "95% sure" and being wrong costs you a lot more Brier-score-wise than saying "55% sure" and being wrong — which is exactly the incentive you want if you're trying to keep forecasters honest.

What counts as a "good" Brier score

• 0.0 — perfect forecasting (never realistic over any real sample size)
• ~0.25 — what you'd get from always guessing "50/50" on everything, a useful uninformed baseline
• Below ~0.20 — generally considered meaningfully better than a coin flip
• 1.0 — worst possible (100% confident and wrong, every time)

Context matters: a Brier score is only meaningful relative to a baseline and a large enough sample. A great score on five trades doesn't tell you much; a great score across a few hundred does.

How Trading Tavern uses it

Every probability estimate we publish gets logged, and the outcome gets logged against it. The Brier score — alongside a calibration curve showing whether our "70%" calls actually hit ~70% of the time — is published on the Truth Meter and updated as new outcomes come in. It's not a marketing number; it goes down when we're wrong, in public, same as everyone else's.