Questions tagged [backtesting]
The process of evaluating a strategy, theory, or model by applying it to historical data.
94
questions with no upvoted or accepted answers
14
votes
0
answers
381
views
Here is an approach for measuring Data Snooping; is it new?
I came up with an approach for measuring data snooping, or overfitting. My question is whether this approach was published and expanded-on already, or is it new?
My approach relies on the observation ...
10
votes
0
answers
446
views
What is the most convenient data structure for backtesting a model of futures options prices?
I have an empirical model for the dynamics of futures prices in a particular market that I have implemented using a long series of the front five contracts. (I account for the roll in my model.) I ...
9
votes
0
answers
4k
views
VaR model Unconditional Coverage Tests: Is this extension of Kupiec POF test correct?
Background: Kupiec P. in 1995, published paper "Techniques for Verifying the Accuracy of Risk Management Models" on Journal of Derivatives, v3, P73-84, it's a Unconditional Coverage Tests designe for ...
8
votes
1
answer
703
views
Up and Down days in GBPUSD and a Filter
I want to study if the odds of an up or down day in a forex pairs is 50-50. I just count the total number of up and down days in X years and compare it with the total days. The results are very ...
7
votes
1
answer
410
views
Significance testing of average returns from Sharpe ratio
I'm aware that one way to do significance testing on a strategy is based on the sampling distribution of its Sharpe (see, e.g., Lo, 2002 and Opdyke, 2008).
However, it appears to me that there's ...
5
votes
1
answer
2k
views
How to calculate average entry price for perpetual swap contracts?
I'm trying to calculate the average entry price for perpetual swap contracts for use in back-testing a trading strategy, as per Bitmex's documentation:
A Perpetual Contract is a derivative product ...
5
votes
0
answers
113
views
Backtesting: what happens to real-executed order if mine fills instead?
I'm writing a backtester and using an order-by-order Nasdaq ITCH feed whilst testing it.
Let's say the bid-ask spread is 100 @ 9-11 @ 100
My strategy inserts an bid order for 100 @ 10 and this ...
5
votes
0
answers
826
views
VaR backtesting with overlapping time intervals
Of course the issue here is dependence: can it be removed or accounted for (in independence tests too, which of course would be troublesome)? There's a lot of literature on regression in this setting, ...
5
votes
0
answers
163
views
Are there canonical test cases for testing of pricing engines
Short intro: We are developing pricing engines for the calculation of market risk in a Solvency II solution, including bonds, callable bonds, cds, options, futures and so on. Are there any canonical ...
4
votes
0
answers
162
views
Finding optimal calendar spreads and diagonals
I am looking for some pointers on risk/return profiles of calendar spreads and diagonals with different strikes and expiration dates, preferably based on historical backtests with SPY options.
Please ...
4
votes
0
answers
327
views
Trading signal strength: [-1 to 1] or [predicted return]?
In the context of a backtesting engine, is it better to have strategy generate trade signals in the range from -1 to 1 or as exact predicted returns (e.g. -12% or 26%).
The difference lies in how to ...
3
votes
0
answers
116
views
Understanding the calibration of High-frequency trading in a limit order book
I am trying understand and replicate this thesis, which is based on, High-frequency trading in a limit order book by (Avellaneda and Stoikov, 2008) and Optimal market making, by Olivier Gueant, 2017, ...
3
votes
0
answers
123
views
How to take into account transaction fee of a backtest from a list of returns?
I have a list of booleans that correspond to buy and sell signals that I would like to backtest. To achieve this, I calculated the return ret of a security and when ...
3
votes
0
answers
209
views
PBO algorithm "The Probability of Backtest Overfitting" paper
In this article by Lopez de Prado et al., an algorithm was proposed for assessing the overfit of a trading strategy:
The Probability of Backtest Overfitting
There is also a package for R:
pbo: ...
3
votes
0
answers
186
views
How to backtest multilegged options strategies?
I have got historical data for the index options. Now I am looking at backtesting some of my strategies with this historical data. I would like to backtest strategies like selling a straddle and ...
3
votes
0
answers
155
views
Realistic Modeling of Capacity in Backtesting a trading strategy
Typically the backtest of a quantitative trading strategy assumes a fixed period and fixed capital at the start to backtest a strategy. However, each strategy has a capacity (due to non-linear trading ...
3
votes
0
answers
242
views
reference for portfolio / margin calculations in backtesting tool
I have been tasked with writing a backtesting tool from scratch. I understand a lot of trading operations, but I am primarily a researcher. I need to support futures and equities trading.
I need to ...
3
votes
0
answers
1k
views
'GARCH - extreme value theory - copula' approach to estimate risk measures in R
I'm reading about this approach of using GARCH-EVT-copula methodology to separate univariate and joint estimation and then estimate for example VaR and ES. I wanted to try something similar, but my ...
3
votes
0
answers
208
views
Strategy Testing
Firstly I am a newbie so I will apologise in advance if this is in the wrong forum. My question concerns testing for an equity trading strategy and I would appreciate any comments as to whether my ...
3
votes
0
answers
158
views
Adjusting for your own orders in future backtests
I was asked this question in an interview and despite thinking about it for a while, I haven't been able to come up with a good answer.
Suppose you have a strategy you are running where at certain ...
2
votes
0
answers
53
views
Do you roll back the maturities and schedules when backtesting VaR for portfolios of bonds, options or future contracts?
I want to backtest VaR models which are applied to portfolios of products which have maturities (options and futures) and even schedules (bonds). I have a question which never came up when backtesting ...
2
votes
0
answers
137
views
Backtesting of outperformance of a benchmark using the Deflated Sharpe Ratio
I want to test whether, let's say, strategy A outperforms strategy B.
In Marcos López de Prado's book Advances in Financial Machine Learning he presents the following statistics:
The Probalistic ...
2
votes
0
answers
173
views
How to backtest algorithms in parallel?
I want to backtest a large number of algorithms on the same dataset, one stock ticker. The algorithms are actually variations of one algorithm with various combinations of parameters, amounting to ...
2
votes
0
answers
139
views
Mean directional accuracy and zero
I'm trying to use mean directional accuracy to evaluate my directional predictions in back-test, but it can't deal with realised directions which are 0, due to the comparison of the signs of ...
2
votes
0
answers
545
views
In sample and out of sample in Mean Variance Optimization
Hello to everyone and thanks again for your help, i have find this forum really helpful while working on my final dissertation.
However I'm here again because I have loads of doubts regarding the in-...
2
votes
0
answers
125
views
how to avoid building a strategy that depends on very long trends
When I construct a strategy, it is easy to make subtle dependencies on trends that have existed for a long time.
Sometimes it is legit to explicitly take advantage of the trends. For example, it has ...
2
votes
0
answers
268
views
LGD performing model - LGD estimate vs LGD observed
LGD (Loss Given Default) performing model is developed on through the cycle sample which consists of loans in default.
What I want is to compare LGD estimate and LGD observed (realized). LGD observed ...
2
votes
0
answers
17
views
Listed Company's Filing Deadline
Where could I get historical record of listed company's filing deadline that announced by SEC of Asia country & US?
I could get the latest deadline from respective SEC websites, but not changes ...
2
votes
0
answers
161
views
How to reset indicators in quantstrat / quantmod?
I am currently trying to back test a strategy on a file which contains minutely data for the whole month. I would like to reset my indicators at 3:30 PM . Is there a way I can reset the indicators ...
2
votes
0
answers
747
views
Backtesting Long/Short Market Neutral Z-Score Strategy with Custom Factors and Custom Stock Universe
So I've managed to backtest simple strategies, like MA, RSI and some fundamental ones (P/E ratios etc) but Im stuck at my last strategy. Here is some information:
Tools: Excel and Python (also a ...
2
votes
0
answers
743
views
backtest asset allocation strategies
I've searched the site but haven't found an easy way to backtest my personal portfolio allocations. Suppose I want to know the total return for the following portfolio from Jan 1, 2009 to Dec 31, 2014:...
2
votes
2
answers
662
views
how to choose a price adjustment, a roll date and a data center for my trading strategy?
I have many doubts about Which roll date and price adjustment should I use. I need to backtest like 50 diferents futures.
6 index(mini sp500, Nikkei 225…), 10 Agriculture (soybean, Oat, Corn….),3 ...
2
votes
0
answers
150
views
How to compare market values with model values after calibration?
After calibration the G2++ model for interest (with swaption volatilities), I want to statistically test the quality of the calibration by comparing market to model values.
What is the best way to ...
2
votes
0
answers
2k
views
How to calculate returns of backtested strategy?
Lets say I have some strategy (long/short) backtested for certain period. Strategy has entries/exits only at the end of the day and may have overnight positions hold. Now I would like to compare ...
2
votes
0
answers
440
views
Comparing Backtests of Value-at-Risk and Expected Shortfall
My goal is to test if ES (CVaR) empirically is a better risk measure than VaR for a set of given variables (assumed underlying distribution, confidence level, sample size) for different asset classes.
...
2
votes
0
answers
285
views
Backtesting benchmark / control test design
I am designing a backtesting system to test an algorithm.
I'd like to have a benchmark / control test results
to compare to the results of a custom algorithm.
A limited "set" of stocks is selected ...
1
vote
0
answers
104
views
Interest Rate Calibration and Backtesting under Fed's raising rates 2022-2023
With the Fed raising interest rates so fast and so drastically, classical interest rate models such as the Black Karasinski (BK), Hull-White (HW), etc., may have trouble calibrating to current rate ...
1
vote
0
answers
138
views
Normalized statistical risk/reward measures to compare different quant trading strategy's returns, eg for backtesting
Want to select a metric or metrics to compare returns of different investment strategies, for quantitative backtesting, strategy selection, and forward measurement.
Been reading different approaches ...
1
vote
0
answers
117
views
Backtesting - treatment of holidays for global (i.e. multi-market) portfolios
Assume a daily trading strategy where each day we rebalance our portfolio weights:
Situation A: all constituents of our portfolio are from the same market (e.g. a portfolio of S&P 500 stocks)
...
1
vote
0
answers
292
views
How to backtest with fixed-income instruments
I'm running a backtest with the 5-yr and the 30-yr treasury bills going back to 1990, both with a weighting of 25%. How do I use their daily yields to adjust the portfolio through time?
I've thought ...
1
vote
0
answers
139
views
Spread betting risk management in backtesting in Python
My background is in Spread Betting. I know how to calculate my position size based on how far away my stop is from my entry, I calculate the amount per pip so that I only ever risk a fixed percentage ...
1
vote
0
answers
537
views
How to annualise hourly returns?
I have hourly open,high,low,close candles data for a particular asset. I wrote my own algo and some back testing code that replays the data from the past hourly candles to calculate the total return ...
1
vote
0
answers
40
views
Controlling for factors that influence minimum variance optimization
I am trying to compare the performance of two minimum variance optimization (mvpo) methods applied on stocks Hierarchical risk parity (HRP) vs the analytical global minimum variance formula.
I feel ...
1
vote
0
answers
697
views
Easiest possible way to backtest a semi dynamic options strategy
I have a few options strategies Id like to backtest and I have some familiarity with Python. In particular Id like to backtest a "semi-dynamic" long vol. strategy putting on $0$ cost ...
1
vote
0
answers
167
views
Multivariate combinatorial purged cross-validation
Combinatorial purged cross-validation (CPCV) is a technique for backtesting strategies while purging and embargoing observations in a time series. CPCV improves upon classical k-fold and walk-forward ...
1
vote
0
answers
46
views
Is C(P)CV similar to boostrap?
I am writing to ask if the Combinatorial (Purged) Cross-Validation" method of Marcos Lopez de Prado's "Advances in Financial Machine Learning" book is similar to the idea of bootstrap. If not, what is ...
1
vote
0
answers
1k
views
Kupiec Test Backtesting VaR
I am currently analyzing the Kupiec test used for backtesting $VaR$.
Suppose that I backtest a $VaR$ system for $n$ days (for example 250), with a confidence interval of $1-\alpha$ (for example a $1-\...
1
vote
1
answer
320
views
Proper Data Partitioning For Building a Forecasting Model
Goal: A team and I are looking to build a model that performs a predictive action for the state of the market on day T + n, using the data at hand on day ...
1
vote
0
answers
74
views
forex backtesting spread cost modeling
For professional level forex trading, what is a reasonable estimate of roundtrip transaction costs (I am talking about major pairs, like EUR/USD).
1
vote
0
answers
168
views
Modelling VIX futures backwardation
I have VIX futures trading algorithm and would like to perform Monte-Carlo simulation of VIX and understand how my algorithm performs on each simulation. In this case, not only VIX should be modeled, ...