Reading a Backtest Honestly: Metrics That Don't Flatter

Anyone can print a Sharpe ratio. The library call is one line. The hard part — the part that separates a research platform from a spreadsheet — is computing metrics that survive scrutiny. Because the first thing a serious allocator does with your backtest is try to break it, and the second thing is ask how you computed the numbers.

Crypto makes this harder than equities. The data is gappy. It trades 24/7, so "trading days" is a fiction. Funding feeds drop out. A metric formula that's merely standard — copied from a stack-overflow answer — will quietly inflate your results on exactly this kind of data. We sweated the formulas so the numbers are ones you can defend in a meeting, not just ones that look good on a chart.

Annualize from calendar time, not bar count

Here's a subtle inflater that catches almost everyone. To annualize a return, you need to know how many years elapsed. The lazy way is to count bars and divide by bars-per-year. On a continuous series that's fine. On gappy crypto data it's a lie — and a flattering one.

Our compute_metrics derives elapsed years from the calendar span of the equity index, falling back to the bar count only when timestamps are unusable:

# derive elapsed years from the calendar SPAN of the equity index, not the bar
# COUNT. The data layer tolerates gaps, so counting bars understates elapsed time
# and overstates annualized return / Calmar on gappy crypto series.
years = None
idx = eq.index
if isinstance(idx, pd.DatetimeIndex) and len(idx) > 1:
    span = (idx[-1] - idx[0]).total_seconds()
    if span > 0:
        years = span / SECONDS_PER_YEAR
if years is None:
    years = n / bpy
ann_return = (1.0 + total_return) ** (1.0 / years) - 1.0 if years > 0 and (1.0 + total_return) > 0 else None

Why it matters: if your series has gaps, the bar count is lower than the real elapsed time. Divide your total return by too-few years and the annualized number — and the Calmar ratio built on it — comes out too high. By measuring from the first timestamp to the last, the engine annualizes against the time that actually passed. The number gets a little less impressive and a lot more honest.

Sortino the textbook way, not the convenient way

The Sortino ratio rewards strategies that have low downside volatility. The catch is that there are several ways to compute downside deviation, and the convenient one overstates your Sortino. Our implementation uses the textbook definition — and the comment names the wrong way it replaced:

# textbook downside deviation = sqrt(mean of squared NEGATIVE returns over ALL
# observations) annualized. The old std(downside-only, ddof=1) divided by the
# count of losers, overstating downside dev and understating Sortino (and
# inconsistent with how most platforms report it).
neg = rets.clip(upper=0.0)
dvol = float(math.sqrt((neg ** 2).mean() * bpy)) if len(rets) > 0 else None
sortino = (ann_return / dvol) if (ann_return is not None and dvol not in (None, 0.0)) else None

The distinction is whether you average squared negative returns over all observations or only over the losing ones. Averaging over losers only inflates the denominator's meaning and produces a Sortino inconsistent with how reputable platforms report it — which means an allocator who recomputes yours gets a different, lower number and starts wondering what else you fudged. Matching the textbook definition means your Sortino reconciles with theirs. That reconciliation is worth more than the points you'd have gained by cheating.

Refuse to print a number you can't stand behind

Notice what the code does when it can't compute a metric honestly: it returns None, not a plausible-looking placeholder.

if n < 2 or eq.iloc[0] == 0:
    out.update({k: None for k in (
        "total_return", "annualized_return", "annualized_vol", "sharpe", "sortino",
        "max_drawdown", "max_drawdown_bars", "exposure")})

def _safe(x):
    return None if (x is None or (isinstance(x, float) and (math.isnan(x) or math.isinf(x)))) else x

A backtest with one bar has no meaningful Sharpe. Rather than emit a NaN that some downstream chart renders as 0.0 or a garbage spike, the engine emits None — an honest "not computable." A blank is a fact; a fabricated number is a trap.

The metrics come with a data-quality conscience

The most important honesty isn't in the metrics themselves — it's in what ships next to them. Every backtest returns a data report, and the engine populates it with exactly the things that would otherwise silently corrupt your numbers: how many mark-to-market bars were skipped because a held instrument had no price, how many funding accruals were skipped because the rate was missing.

data_report["skipped_mtm_bars"] = skipped_mtm
data_report["skipped_funding_bars"] = skipped_funding
if skipped_mtm:
    data_report.setdefault("warnings", []).append(
        f"{skipped_mtm} mark-to-market bar(s) skipped (a held instrument had no bar at that ts)")

This is the difference between a backtest and a trustworthy backtest. A 3.0 Sharpe means nothing if 30% of the bars were patched over missing data — and without this report, you'd never know. With it, you see the warning, you see the skip counts, and you can decide whether the result is real or an artifact of a thin data window. The platform tells on itself.

Why honest metrics are a growth strategy

There's a short-term temptation to make every backtest look as good as possible — bigger Sharpes sell better, right? It's exactly backwards. Inflated metrics survive precisely until your first sophisticated user recomputes them, finds the gap, and concludes your platform can't be trusted. Then every number you ever show them is discounted to zero.

Honest metrics do the opposite. When a quant recomputes your Sharpe by hand and it matches, when your annualized return reconciles against the calendar, when the data report flags the thin window before they have to ask — that builds the one thing a research platform actually sells: credibility. The platform that flatters you loses you at the first audit. The platform that tells you the truth keeps you for the next decade.

We'd rather show you a slightly lower number you can take to an allocator than a slightly higher one that falls apart under their spreadsheet.

---

Run a backtest and read the metrics — annualized against the calendar, Sortino by the book, with a data-quality report attached. Start a backtest →