What is a calibrated forecast? Why '70%' should mean about 70%
A forecast is calibrated when its stated probabilities match reality: across all the times it said 70%, the event should happen about 70% of the time. Calibration is checked by grouping forecasts into probability bands and comparing the stated chance with the actual rate.
How to check it
Group every forecast by its stated probability (50–60%, 60–70% and so on). For each group, compare the average stated probability with how often the event actually happened. If the two columns are close in every row, the forecast is calibrated. Large gaps mean the model is over- or under-confident.
Why it matters
A probability you cannot trust is just a label. If '90%' really means 75%, any decision that relies on it is mis-sized. Calibration is separate from accuracy: a model can be accurate overall and still badly calibrated at the extremes.
ApolloAlpha's calibration, as published
For the Apollo Axiom 2 model, out-of-sample: 50-60% stated 55.1%, actual 55.2%; 60-70% stated 65.1%, actual 65.2%; 70-80% stated 75.1%, actual 76.6%; 80-90% stated 84.2%, actual 85.8%; 90-100% stated 94.5%, actual 93.1%. The record is slightly under-confident in the middle bands and slightly over-confident at 90% and above, and it is shown that way on purpose. It is a historical test and does not guarantee future results.
Frequently asked questions
Is calibration the same as accuracy?
No. Accuracy counts how often the call was right. Calibration checks whether the stated probabilities match how often events occur.
What does over-confident mean?
The model states a higher probability than the event actually occurs at. For example saying 94.5% when the real rate was 93.1%.
Published 2026-11-07. See the track record and audit for the full method. Statistical model output, not financial advice. Past results do not guarantee future results.