Time-Series Cross-Validation Without Lookahead Bias

$ through $T_{\min} + (k-1) \cdot T_{\text{test}}$. - Embargo gap: skip the next $L$ days after the training period ends. No data from these days appears in the test set. This prevents any feature in the test set from depending on target values in the training set. - Test period: the next $T_{\text{test}}$ days after the embargo gap.

1. Features are computed using only data available at prediction time. Rolling means and lags are constructed *within* each fold's training window -- never peek past the training cutoff.

1. Optionally use an expanding window (training grows each fold) or a sliding window (fixed-size training set). Expanding window is more common when data is scarce; sliding window is better when you believe the process is non-stationary and old data hurts.

*Part 2: Out-of-Sample Sharpe Ratio and Standard Error*

1. Collect the model's daily out-of-sample returns $\{r_1, r_2, \ldots, r_N\}$ by concatenating the test-period returns across all $K$ folds (each day appears in exactly one test fold).

1. Compute the annualized Sharpe ratio:

$\widehat{SR} = \frac{\bar{r}}{\hat{\sigma}} \cdot \sqrt{252}$

where $\bar{r}$ is the mean daily return and $\hat{\sigma}$ is the sample standard deviation of daily returns.

1. Under mild assumptions (returns roughly i.i.d. and not too heavy-tailed), the standard error of the annualized Sharpe is approximately:

$SE(\widehat{SR}) \approx \frac{1}{\sqrt{N}} \sqrt{1 + \frac{\widehat{SR}^2}{4 \cdot 252}} \cdot \sqrt{252}$

For small Sharpe ratios (common in practice), this simplifies to roughly $\sqrt{252 / N}$. If you have 5 years of daily out-of-sample data ($N \approx 1260$), your SE is about $\sqrt{252/1260} \approx 0.45$. This means a Sharpe of 0.8 is barely two standard errors from zero -- a sobering reality check.

1. If returns exhibit serial correlation (common at daily frequency), inflate the standard error using a Newey-West or HAC adjustment with $q$ lags:

$SE_{\text{HAC}} = SE \cdot \sqrt{1 + 2\sum_{j=1}^{q} \left(1 - \frac{j}{q+1}\right)\hat{\rho}_j}$

where $\hat{\rho}_j$ is the $j$-th autocorrelation of daily returns.

*Part 3: Model Selection with Multiple-Testing Control*

1. The problem: You test $M$ hyperparameter configurations. Even if all models are worthless, the best one will have a positive Sharpe by luck. The expected best Sharpe among $M$ independent null models grows like $\sqrt{2 \ln M}$ (extreme value theory). With $M = 100$, that is about $\sqrt{2 \ln 100} \approx 3.0$ in $t$-statistic terms -- enough to fool most thresholds.

1. Bonferroni correction (simple but conservative): Require the best model's $p$-value to be below $\alpha / M$. Equivalently, require $|t| > \Phi^{-1}(1 - \alpha / (2M))$. For $M = 100$ and $\alpha = 0.05$, you need $|t| > 3.29$.

1. Holm or Hochberg step-down (tighter): Rank the $M$ $p$-values $p_{(1)} \leq p_{(2)} \leq \cdots \leq p_{(M)}$. Reject $p_{(k)}$ if $p_{(k)} \leq \alpha / (M - k + 1)$. This is uniformly more powerful than Bonferroni.

1. Romano-Wolf or White's Reality Check / Hansen's SPA test (best practice): These bootstrap-based methods account for the correlation structure among the $M$ test statistics. Since your hyperparameter grid produces highly correlated strategies, naive Bonferroni is too conservative. These tests simulate the distribution of the maximum Sharpe under the null by bootstrapping (block bootstrap to preserve time-series dependence) and compare your observed best Sharpe to this distribution. If your best model's Sharpe exceeds the 95th percentile of the bootstrap max-Sharpe distribution, you reject the null that all models are worthless.

1. Practical workflow: Run walk-forward CV for all $M$ configurations. Collect each model's out-of-sample returns. Apply Hansen's SPA test (or White's Reality Check) with stationary block bootstrap (block length around $\sqrt{N}$). Report the SPA $p$-value. Only deploy models that survive this filter.

Practical Considerations:

• Embargo length matters. If your longest moving average is 60 days but you also use 5-day lagged returns, the embargo only needs to be 60 days (the max lookback). But if you have features that depend on exponentially weighted averages with long half-lives, be conservative.
• Number of folds vs. test-set size: More folds give you more out-of-sample data points, but if test windows are too short, transaction cost estimates become noisy. A good default: quarterly test windows with at least 2 years of initial training.
• Purging vs. embargo: Some frameworks (e.g., Lopez de Prado's *Advances in Financial Machine Learning*) distinguish between purging (removing training samples whose labels overlap with test samples) and embargo (adding a gap). For daily return forecasting with lagged features, an embargo is usually sufficient. Purging matters more when labels span multiple days (e.g., triple-barrier labels).
• Deflated Sharpe Ratio: Bailey and Lopez de Prado's deflated Sharpe adjusts the observed Sharpe for the number of trials, skewness, and kurtosis. It is a closed-form alternative to the bootstrap tests and easy to implement as a sanity check.

Answer: Use walk-forward (expanding or sliding window) CV with an embargo gap of at least $L$ days (your longest feature lookback) between training and test periods. Compute the annualized out-of-sample Sharpe from concatenated test returns with HAC standard errors. For model selection over $M$ configurations, apply Hansen's Superior Predictive Ability test or White's Reality Check (block-bootstrap-based) to control for multiple testing, since Bonferroni is too conservative when strategies are correlated. Only deploy models whose performance survives this correction.