11
$\begingroup$

I am attempting to evaluate and compare the profit factor of different "test runs" of a FOREX trading strategy.

My problem is that, despite an average time between orders of 2hr+, some of these runs can have 20+ orders in a row, every 5 minutes, in the same direction. I need some way to normalize these clusters that occur without just throwing the data out.

I want to treat the cluster as 1 data point by averaging the gain/loss of each trade within the cluster.

I was thinking of doing it with a moving time window in the following manner:

For each order:
Weight = 1/(N+1) 
N =  count of consecutive orders within 30 minutes of the current order.

But I am not sure if that is correct.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

That is definitely not correct. Your test results will suffer from look-ahead bias. At the point in time that you will be entering your order, you will not know yet how many orders will come in the next 30 minutes, thus a proper backtest should not use that information to size the trade.

There are many factors to consider when running a proper backtest (see wikipedia), but the key factor behind all of them is to replicate as closely as possible what is realizable in live trading. In your case, that means that you will have to predict how many trades your strategy is about to enter and size all subsequent trades appropriately. Once you reach your "limit," at which point in real trading you would have run out of capital, your backtest is effectively prevented from entering new trades.

If your strategy isn't able to predict whether trades are about to cluster, you could try an incremental system, whereby the first trade is assigned a proportion $\delta$ of total capital, the second gets $\delta(1-\delta)$, etc.

Improve this answer
$\endgroup$
7
  • $\begingroup$ This is not true. There is no look-ahead bias. You are right that at the point in the when I enter the order I do not know how many orders will come in the next 30 minutes - BUT - I do know how many orders came in the next 30 minutes of SIMILAR orders to the current order - in the past. THAT is what I am evaluating and what I am basing my entry size on - it is a meta test. $\endgroup$
    – Mike Furlender
    Sep 19, 2011 at 0:29
  • $\begingroup$ Mike, it sounds like you are trying to get around the fact that you are allowing look-ahead bias by calling it a "meta" test. I would just be very careful with that. The way your original question is phrased, it is clear that you cannot use the weight scheme you proposed. You cannot even replace N with E[N], because there will be some occasions that N>E[N], and your test will implicitly be allowing you to use more capital than you actually have available. These sort of "meta" tests are nearly worthless for anything but a quick and very rough first pass at the data. $\endgroup$
    – Tal Fishman
    Sep 19, 2011 at 2:08
  • $\begingroup$ How could there be look-ahead bias when everything in the meta test has already occurred? $\endgroup$
    – Mike Furlender
    Sep 19, 2011 at 2:23
  • $\begingroup$ Look-ahead bias refers to using data that would not have been known or available during the period analyzed. It has occurred, but it hadn't occurred yet as of the point in time in your test that you use the information. $\endgroup$
    – Tal Fishman
    Sep 19, 2011 at 2:31
  • $\begingroup$ That's the thing - it HAS occurred at that point in time. Thats why I call it a meta test. $\endgroup$
    – Mike Furlender
    Sep 19, 2011 at 2:42
2
$\begingroup$

you could perhaps cluster the information in a candle stick manner and bin the price data into high, low and closing, instead of throwing some data out and keeping only its closing price.

Using the high, low and closing prices you are able to make 3 separate estimates with regards to Profit and lost. Obviously with a long position, profit calculated by entering at a high price while using a low price for its exit would be the pessimistic estimate of profit.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.