For quants: the full statistics
Apollo Axiom 2, out-of-sample, 2016 to today
Out-of-sample periods differ by market: each market needs history before it can be tested, so January 2016 applies to long-established markets and shorter-history markets start later. These figures show how often the model's calls matched what happened in testing. They are not investment returns, and past accuracy does not guarantee future results. See the track record for method and per-market detail.
Apollo Orion 1 and Apollo Axiom 2, side by side
Improvement = Apollo Axiom 2's relative improvement over Apollo Orion 1 (for the Brier score, the reduction in error), worked out from the rounded figures shown. Same 163 markets and the same weeks for both models, each market scored from its common out-of-sample start. AUC bars start at 0.5 (a coin flip); all other bars start at zero. Historical results, not a guarantee of future results.
How to read the performance statistics
How accurate is the model, really?
It varies by ticker — that's the point. Instead of one headline number, every ticker shows its own accuracy, AUC, Brier score, and full confusion matrix, so you can judge confidence yourself rather than take a marketing claim on faith.
What does the probability on a forecast mean?
It is the model's estimate that the level will hold, for example 80% that last week's low holds as support. It is a probability, not a promise: at 80%, roughly 8 in 10 similar calls should hold and 2 in 10 should break. Calibration checks whether that is true in practice. It is the probability of that defined price event, not the probability of making money: costs, execution prices and gaps affect trading results, and any single forecast can be wrong.
What are ROC and AUC, and how do I read them?
They are two views of the same thing. The ROC curve is the picture: as the confidence cut-off changes, it plots the breaches the model catches (up) against the false alarms it raises (across). The AUC is the score read off that picture, the area under the curve. Put another way, take one week that held and one that broke: AUC is the chance the model gave the held week the higher probability. 0.5 is a coin flip (the diagonal line) and 1.0 is perfect. A curve that bows toward the top-left corner has a high AUC.
What is a Brier score, and how do I read it?
The Brier score measures how close stated probabilities were to what actually happened. 0 is perfect, and always answering 50% scores 0.25, so lower is better and a score well under 0.25 means the probabilities carry real information. It punishes being confidently wrong, which accuracy alone does not.
What is a confusion matrix?
A 2 by 2 table of what the model called against what happened: calls that held and did hold, calls that broke and did break, misses, and false alarms. Large numbers on the correct diagonal are good. We show it for every ticker so you can see where the model is strong and where it is not.
What does calibrated mean?
A forecast is calibrated when its stated probabilities match reality: of all the times it said 70%, about 70% happened. The reliability curve in the dashboard plots exactly this. Points near the diagonal mean the stated odds can be taken at face value.
What does out-of-sample mean?
Out-of-sample means tested on weeks the model had never seen when it made each forecast. Every forecast uses only information available before that week, and our headline track record counts weeks from January 2016 onward. This is what separates a real track record from a backtest that peeked.
What are precision and recall?
Two ways to read the confusion matrix. Precision asks: of the weeks the model called "hold", how many really held? Recall asks: of all the weeks that really held, how many did the model call? High precision means few false alarms; high recall means few misses. The same two measures apply to breaches. Looking at both stops one flattering number from hiding a weakness.
How should I read the correlation table?
The correlation table compares how closely two markets' weekly returns have moved together: +1 moves in step, 0 is unrelated, and -1 moves opposite. It uses ordinary (Pearson) correlation of weekly returns over a chosen window. Short windows are noisy, so the 1-year and longer views are steadier. It is a diversification aid: calls on markets that move together are less independent than they look.