Questions tagged [backtesting]

The process of evaluating a strategy, theory, or model by applying it to historical data.

94 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
14 votes
0 answers
381 views

Here is an approach for measuring Data Snooping; is it new?

I came up with an approach for measuring data snooping, or overfitting. My question is whether this approach was published and expanded-on already, or is it new? My approach relies on the observation ...
greg's user avatar
  • 300
10 votes
0 answers
446 views

What is the most convenient data structure for backtesting a model of futures options prices?

I have an empirical model for the dynamics of futures prices in a particular market that I have implemented using a long series of the front five contracts. (I account for the roll in my model.) I ...
Drew's user avatar
  • 405
9 votes
0 answers
4k views

VaR model Unconditional Coverage Tests: Is this extension of Kupiec POF test correct?

Background: Kupiec P. in 1995, published paper "Techniques for Verifying the Accuracy of Risk Management Models" on Journal of Derivatives, v3, P73-84, it's a Unconditional Coverage Tests designe for ...
athos's user avatar
  • 2,231
8 votes
1 answer
703 views

Up and Down days in GBPUSD and a Filter

I want to study if the odds of an up or down day in a forex pairs is 50-50. I just count the total number of up and down days in X years and compare it with the total days. The results are very ...
tn240's user avatar
  • 121
7 votes
1 answer
410 views

Significance testing of average returns from Sharpe ratio

I'm aware that one way to do significance testing on a strategy is based on the sampling distribution of its Sharpe (see, e.g., Lo, 2002 and Opdyke, 2008). However, it appears to me that there's ...
Powerfool's user avatar
  • 101
5 votes
1 answer
2k views

How to calculate average entry price for perpetual swap contracts?

I'm trying to calculate the average entry price for perpetual swap contracts for use in back-testing a trading strategy, as per Bitmex's documentation: A Perpetual Contract is a derivative product ...
Natalie Williams's user avatar
5 votes
0 answers
113 views

Backtesting: what happens to real-executed order if mine fills instead?

I'm writing a backtester and using an order-by-order Nasdaq ITCH feed whilst testing it. Let's say the bid-ask spread is 100 @ 9-11 @ 100 My strategy inserts an bid order for 100 @ 10 and this ...
user997112's user avatar
5 votes
0 answers
826 views

VaR backtesting with overlapping time intervals

Of course the issue here is dependence: can it be removed or accounted for (in independence tests too, which of course would be troublesome)? There's a lot of literature on regression in this setting, ...
Quartz's user avatar
  • 1,553
5 votes
0 answers
163 views

Are there canonical test cases for testing of pricing engines

Short intro: We are developing pricing engines for the calculation of market risk in a Solvency II solution, including bonds, callable bonds, cds, options, futures and so on. Are there any canonical ...
Owe Jessen's user avatar
  • 1,144
4 votes
0 answers
162 views

Finding optimal calendar spreads and diagonals

I am looking for some pointers on risk/return profiles of calendar spreads and diagonals with different strikes and expiration dates, preferably based on historical backtests with SPY options. Please ...
vonjd's user avatar
  • 27.4k
4 votes
0 answers
327 views

Trading signal strength: [-1 to 1] or [predicted return]?

In the context of a backtesting engine, is it better to have strategy generate trade signals in the range from -1 to 1 or as exact predicted returns (e.g. -12% or 26%). The difference lies in how to ...
A.L. Verminburger's user avatar
3 votes
0 answers
116 views

Understanding the calibration of High-frequency trading in a limit order book

I am trying understand and replicate this thesis, which is based on, High-frequency trading in a limit order book by (Avellaneda and Stoikov, 2008) and Optimal market making, by Olivier Gueant, 2017, ...
ayamathss1's user avatar
3 votes
0 answers
123 views

How to take into account transaction fee of a backtest from a list of returns?

I have a list of booleans that correspond to buy and sell signals that I would like to backtest. To achieve this, I calculated the return ret of a security and when ...
Florent's user avatar
  • 241
3 votes
0 answers
209 views

PBO algorithm "The Probability of Backtest Overfitting" paper

In this article by Lopez de Prado et al., an algorithm was proposed for assessing the overfit of a trading strategy: The Probability of Backtest Overfitting There is also a package for R: pbo: ...
mr.T's user avatar
  • 155
3 votes
0 answers
186 views

How to backtest multilegged options strategies?

I have got historical data for the index options. Now I am looking at backtesting some of my strategies with this historical data. I would like to backtest strategies like selling a straddle and ...
Abhijeet Singh's user avatar
3 votes
0 answers
155 views

Realistic Modeling of Capacity in Backtesting a trading strategy

Typically the backtest of a quantitative trading strategy assumes a fixed period and fixed capital at the start to backtest a strategy. However, each strategy has a capacity (due to non-linear trading ...
mea43's user avatar
  • 163
3 votes
0 answers
242 views

reference for portfolio / margin calculations in backtesting tool

I have been tasked with writing a backtesting tool from scratch. I understand a lot of trading operations, but I am primarily a researcher. I need to support futures and equities trading. I need to ...
quantresearcher_lambda's user avatar
3 votes
0 answers
1k views

'GARCH - extreme value theory - copula' approach to estimate risk measures in R

I'm reading about this approach of using GARCH-EVT-copula methodology to separate univariate and joint estimation and then estimate for example VaR and ES. I wanted to try something similar, but my ...
Kondo's user avatar
  • 449
3 votes
0 answers
208 views

Strategy Testing

Firstly I am a newbie so I will apologise in advance if this is in the wrong forum. My question concerns testing for an equity trading strategy and I would appreciate any comments as to whether my ...
Chris Wilson's user avatar
3 votes
0 answers
158 views

Adjusting for your own orders in future backtests

I was asked this question in an interview and despite thinking about it for a while, I haven't been able to come up with a good answer. Suppose you have a strategy you are running where at certain ...
Thomas Johnson's user avatar
2 votes
0 answers
53 views

Do you roll back the maturities and schedules when backtesting VaR for portfolios of bonds, options or future contracts?

I want to backtest VaR models which are applied to portfolios of products which have maturities (options and futures) and even schedules (bonds). I have a question which never came up when backtesting ...
leonnis's user avatar
  • 21
2 votes
0 answers
137 views

Backtesting of outperformance of a benchmark using the Deflated Sharpe Ratio

I want to test whether, let's say, strategy A outperforms strategy B. In Marcos López de Prado's book Advances in Financial Machine Learning he presents the following statistics: The Probalistic ...
tuffah12's user avatar
2 votes
0 answers
173 views

How to backtest algorithms in parallel?

I want to backtest a large number of algorithms on the same dataset, one stock ticker. The algorithms are actually variations of one algorithm with various combinations of parameters, amounting to ...
Gascoyne's user avatar
  • 527
2 votes
0 answers
139 views

Mean directional accuracy and zero

I'm trying to use mean directional accuracy to evaluate my directional predictions in back-test, but it can't deal with realised directions which are 0, due to the comparison of the signs of ...
wildbunny's user avatar
  • 629
2 votes
0 answers
545 views

In sample and out of sample in Mean Variance Optimization

Hello to everyone and thanks again for your help, i have find this forum really helpful while working on my final dissertation. However I'm here again because I have loads of doubts regarding the in-...
renato's user avatar
  • 51
2 votes
0 answers
125 views

how to avoid building a strategy that depends on very long trends

When I construct a strategy, it is easy to make subtle dependencies on trends that have existed for a long time. Sometimes it is legit to explicitly take advantage of the trends. For example, it has ...
Tom Bennett's user avatar
2 votes
0 answers
268 views

LGD performing model - LGD estimate vs LGD observed

LGD (Loss Given Default) performing model is developed on through the cycle sample which consists of loans in default. What I want is to compare LGD estimate and LGD observed (realized). LGD observed ...
I. STACK OVERFLOW's user avatar
2 votes
0 answers
17 views

Listed Company's Filing Deadline

Where could I get historical record of listed company's filing deadline that announced by SEC of Asia country & US? I could get the latest deadline from respective SEC websites, but not changes ...
YmLee's user avatar
  • 21
2 votes
0 answers
161 views

How to reset indicators in quantstrat / quantmod?

I am currently trying to back test a strategy on a file which contains minutely data for the whole month. I would like to reset my indicators at 3:30 PM . Is there a way I can reset the indicators ...
Yatharth Narang's user avatar
2 votes
0 answers
747 views

Backtesting Long/Short Market Neutral Z-Score Strategy with Custom Factors and Custom Stock Universe

So I've managed to backtest simple strategies, like MA, RSI and some fundamental ones (P/E ratios etc) but Im stuck at my last strategy. Here is some information: Tools: Excel and Python (also a ...
mortenhaga's user avatar
2 votes
0 answers
743 views

backtest asset allocation strategies

I've searched the site but haven't found an easy way to backtest my personal portfolio allocations. Suppose I want to know the total return for the following portfolio from Jan 1, 2009 to Dec 31, 2014:...
Stephen Grimes's user avatar
2 votes
2 answers
662 views

how to choose a price adjustment, a roll date and a data center for my trading strategy?

I have many doubts about Which roll date and price adjustment should I use. I need to backtest like 50 diferents futures. 6 index(mini sp500, Nikkei 225…), 10 Agriculture (soybean, Oat, Corn….),3 ...
Manuel Botias's user avatar
2 votes
0 answers
150 views

How to compare market values with model values after calibration?

After calibration the G2++ model for interest (with swaption volatilities), I want to statistically test the quality of the calibration by comparing market to model values. What is the best way to ...
user2322's user avatar
2 votes
0 answers
2k views

How to calculate returns of backtested strategy?

Lets say I have some strategy (long/short) backtested for certain period. Strategy has entries/exits only at the end of the day and may have overnight positions hold. Now I would like to compare ...
mde's user avatar
  • 221
2 votes
0 answers
440 views

Comparing Backtests of Value-at-Risk and Expected Shortfall

My goal is to test if ES (CVaR) empirically is a better risk measure than VaR for a set of given variables (assumed underlying distribution, confidence level, sample size) for different asset classes. ...
Chris's user avatar
  • 321
2 votes
0 answers
285 views

Backtesting benchmark / control test design

I am designing a backtesting system to test an algorithm. I'd like to have a benchmark / control test results to compare to the results of a custom algorithm. A limited "set" of stocks is selected ...
Autowealth's user avatar
1 vote
0 answers
104 views

Interest Rate Calibration and Backtesting under Fed's raising rates 2022-2023

With the Fed raising interest rates so fast and so drastically, classical interest rate models such as the Black Karasinski (BK), Hull-White (HW), etc., may have trouble calibrating to current rate ...
RMA's user avatar
  • 11
1 vote
0 answers
138 views

Normalized statistical risk/reward measures to compare different quant trading strategy's returns, eg for backtesting

Want to select a metric or metrics to compare returns of different investment strategies, for quantitative backtesting, strategy selection, and forward measurement. Been reading different approaches ...
user2330237's user avatar
1 vote
0 answers
117 views

Backtesting - treatment of holidays for global (i.e. multi-market) portfolios

Assume a daily trading strategy where each day we rebalance our portfolio weights: Situation A: all constituents of our portfolio are from the same market (e.g. a portfolio of S&P 500 stocks) ...
Metod Jazbec's user avatar
1 vote
0 answers
292 views

How to backtest with fixed-income instruments

I'm running a backtest with the 5-yr and the 30-yr treasury bills going back to 1990, both with a weighting of 25%. How do I use their daily yields to adjust the portfolio through time? I've thought ...
Thomas Hegarty's user avatar
1 vote
0 answers
139 views

Spread betting risk management in backtesting in Python

My background is in Spread Betting. I know how to calculate my position size based on how far away my stop is from my entry, I calculate the amount per pip so that I only ever risk a fixed percentage ...
Oliver P's user avatar
  • 111
1 vote
0 answers
537 views

How to annualise hourly returns?

I have hourly open,high,low,close candles data for a particular asset. I wrote my own algo and some back testing code that replays the data from the past hourly candles to calculate the total return ...
flojdek's user avatar
  • 21
1 vote
0 answers
40 views

Controlling for factors that influence minimum variance optimization

I am trying to compare the performance of two minimum variance optimization (mvpo) methods applied on stocks Hierarchical risk parity (HRP) vs the analytical global minimum variance formula. I feel ...
Lollorn's user avatar
  • 11
1 vote
0 answers
697 views

Easiest possible way to backtest a semi dynamic options strategy

I have a few options strategies Id like to backtest and I have some familiarity with Python. In particular Id like to backtest a "semi-dynamic" long vol. strategy putting on $0$ cost ...
user123124's user avatar
1 vote
0 answers
167 views

Multivariate combinatorial purged cross-validation

Combinatorial purged cross-validation (CPCV) is a technique for backtesting strategies while purging and embargoing observations in a time series. CPCV improves upon classical k-fold and walk-forward ...
develarist's user avatar
  • 3,000
1 vote
0 answers
46 views

Is C(P)CV similar to boostrap?

I am writing to ask if the Combinatorial (Purged) Cross-Validation" method of Marcos Lopez de Prado's "Advances in Financial Machine Learning" book is similar to the idea of bootstrap. If not, what is ...
Lydia's user avatar
  • 41
1 vote
0 answers
1k views

Kupiec Test Backtesting VaR

I am currently analyzing the Kupiec test used for backtesting $VaR$. Suppose that I backtest a $VaR$ system for $n$ days (for example 250), with a confidence interval of $1-\alpha$ (for example a $1-\...
Tommaso Ferrari's user avatar
1 vote
1 answer
320 views

Proper Data Partitioning For Building a Forecasting Model

Goal: A team and I are looking to build a model that performs a predictive action for the state of the market on day T + n, using the data at hand on day ...
David C's user avatar
  • 215
1 vote
0 answers
74 views

forex backtesting spread cost modeling

For professional level forex trading, what is a reasonable estimate of roundtrip transaction costs (I am talking about major pairs, like EUR/USD).
Igor Rivin's user avatar
1 vote
0 answers
168 views

Modelling VIX futures backwardation

I have VIX futures trading algorithm and would like to perform Monte-Carlo simulation of VIX and understand how my algorithm performs on each simulation. In this case, not only VIX should be modeled, ...
Alexandr  Proskurin's user avatar