Majores Ventures

← All projects

wincurse

Selection-corrected Sharpe ratios for the winning backtest.

Python library and CLI · Python, NumPy, SciPy, pandas, rich · MIT · v0.1.0 · github.com/Thesimpleex/wincurse

Abstract

Gives median-unbiased Sharpe ratio estimates and confidence intervals with valid coverage for the best of many backtested strategy variants. For quantitative researchers, portfolio managers and allocators who are shown the best variant of a parameter sweep.

Coverage of naive versus corrected 95% intervals as the number of backtested variants grows
Figure 1. Coverage of naive versus corrected 95% intervals as the number of backtested variants grows.

1Motivation

Researchers try many correlated variants and report the best one, so its backtest Sharpe ratio is biased upward and its usual confidence interval is too narrow. Existing tools test whether the winner beats zero; wincurse estimates what to expect from it, using the 2024 “Inference on Winners” method (Andrews, Kitagawa and McCloskey).

2Method

  1. Estimate every variant's Sharpe ratio and their joint covariance (delta method with HAC or bootstrap).
  2. Identify the winner and the truncation region implied by having selected it.
  3. Invert the truncated-normal distribution for a median-unbiased estimate and conditional intervals.
  4. Build hybrid intervals bounded by a simultaneous projection set.
  5. Report naive, Deflated Sharpe Ratio and other comparators side by side.

3Results

wincurse analyze on the best of 256 industry-momentum variants: naive and corrected Sharpe ratio, comparators, diagnostics, simulated coverage and an out-of-sample check
Figure 2. wincurse analyze on the best of 256 industry-momentum variants: naive and corrected Sharpe ratio, comparators, diagnostics, simulated coverage and an out-of-sample check.
Naive and corrected intervals for the winner of two backtest grids, with forecast intervals and the realised out-of-sample Sharpe ratio
Figure 3. Naive and corrected intervals for the winner of two backtest grids, with forecast intervals and the realised out-of-sample Sharpe ratio.
wincurse analyze on 340 technical trading rules: the realised out-of-sample Sharpe ratio lies outside every forecast interval
Figure 4. wincurse analyze on 340 technical trading rules: the realised out-of-sample Sharpe ratio lies outside every forecast interval.
Top of the selection-adjusted tear sheet: winner, naive and corrected Sharpe ratio, estimates and intervals
Figure 5. Top of the selection-adjusted tear sheet: winner, naive and corrected Sharpe ratio, estimates and intervals.
Interval ends relative to the winner's estimate as a function of its lead over the runner-up, for the naive, conditional, hybrid and projection intervals
Figure 6. Interval ends relative to the winner's estimate as a function of its lead over the runner-up, for the naive, conditional, hybrid and projection intervals.

4Usage

4.1 Analyze a parameter sweep

Reads a returns matrix of all variants, reports naive versus corrected estimates and intervals, comparators and diagnostics, and writes an HTML tear sheet.

wincurse analyze returns.csv --alpha 0.05 --method hybrid --report tearsheet.html

5Installation

pip install git+https://github.com/Thesimpleex/wincurse

Python 3.11 or later. Not on PyPI; installs from GitHub.

6Specification

Runs
Python 3.11+, CLI
Built with
Python, NumPy, SciPy, pandas, rich
License
MIT
Version
0.1.0

Source code, issues and releases: github.com/Thesimpleex/wincurse. License: MIT.