3
$\begingroup$

I have written a model for predicting the winner of UFC fights.

My model calculates the probability of each fighter to win a given match.

I have back tested the model and found it to be very accurate, it predicts the winner around 65% of the time. The model is trained on 3 years of data and then tested out of sample on the past 9 months worth of data.

I am trying to use my model's output (out of sample) and historic book maker odds to come up with an optimal betting strategy. This is based on the Kelly Criterion.

I have 5 parameters that I think will affect the profitability of a betting strategy. They are based around fractions for kelly and handling uncertainty.

What I did is write a program to create ~500k strategies with different weights for each parameter. I then run them through the past 9 months of data to determine their profitability.

From those ~500k I can narrow down the strategies I'm interested in by filtering them by maximum draw down (20%) and minimum ROI (25%), this will bring down the strategies to around 20k.

How can I further narrow down the strategies into an optimal one?

If I take the best strategy (most profit over the 9 months of out of sample data) I worry that it is likely super over fit, and if I take the average of each parameter for each the 20k strategies I worry that this set of parameters may not work well together.

How can I narrow down the ~20k strategies into one that works well and is likely not to be over fit?

Thanks for your help.

Improve this question
$\endgroup$
2
  • $\begingroup$ This would probably be more appropriate is the stats.se. $\endgroup$
    – ch-pub
    Commented Sep 16, 2014 at 19:52
  • $\begingroup$ @nsw you're right, this isn't quant finance. I think it's more linear regression and have provided an answer below along those lines. $\endgroup$
    – Unknown Coder
    Commented Sep 24, 2014 at 15:20

2 Answers 2

Reset to default
2
$\begingroup$

I agree with the previous statement that this is more stats related than anything else (it's not quant finance). But it's still a great question! This sounds awfully similar to linear regression testing with multiple predictor variables; you're basically doing it in a "monte carlo" fashion :)

Depending on how your data is formatted, you could enter it into a program like Minitab and get the regression model from it within seconds. The model (or the resulting equation) would already give you the "best fit" that you are looking for. It would also give you a quantifiable number (like the resulting R-squared value) to give you a measurement of how accurate your strategy would be at any given time.

Improve this answer
$\endgroup$
1
  • $\begingroup$ My data is basically a historic set of matches with bookmaker's odds. So I came up with like 5 parameters and just try every combination of them in a bunch of nested loops, i.e. for x = 0 to 1 (for y = 0 to 0.5 (for z = 0.5 to 1 (and so on...))), I run each over the data to come up with the performance of each. How can I put these into a format that I can read it into minitab/r/matlab? Would I literally just print out [x, y, z, ..., ROI] into a csv file for all of the different variations? $\endgroup$
    – janderson
    Commented Sep 26, 2014 at 14:02
1
$\begingroup$

Choose the most robust (or insensitive) strategy. You are right that the best strategy might be overfit. So look at your parameter space and focus on the area where profitability, for example, changes least when you change the parameter value. Here is a 1D example: simulated profitability

The most profitable strategy is that single point that unfortunately leaves no room for error - miss it and you fall from the abyss. However at the center of the graph, you will find the strategies that might be less profitable, but are much more robust: even if you missed a little bit, you would still be riding high.

To be sure, this gets complicated in five dimensions - but if you look carefully into the data that you've created, you might find these robust areas.

Good luck!

Improve this answer
$\endgroup$
1
  • $\begingroup$ To add, why not use a clustering algorithm on the resulting profitable parameter vectors and then select the point in the center of the largest cluster? $\endgroup$
    – experquisite
    Commented Sep 16, 2014 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.