I have written a model for predicting the winner of UFC fights.
My model calculates the probability of each fighter to win a given match.
I have back tested the model and found it to be very accurate, it predicts the winner around 65% of the time. The model is trained on 3 years of data and then tested out of sample on the past 9 months worth of data.
I am trying to use my model's output (out of sample) and historic book maker odds to come up with an optimal betting strategy. This is based on the Kelly Criterion.
I have 5 parameters that I think will affect the profitability of a betting strategy. They are based around fractions for kelly and handling uncertainty.
What I did is write a program to create ~500k strategies with different weights for each parameter. I then run them through the past 9 months of data to determine their profitability.
From those ~500k I can narrow down the strategies I'm interested in by filtering them by maximum draw down (20%) and minimum ROI (25%), this will bring down the strategies to around 20k.
How can I further narrow down the strategies into an optimal one?
If I take the best strategy (most profit over the 9 months of out of sample data) I worry that it is likely super over fit, and if I take the average of each parameter for each the 20k strategies I worry that this set of parameters may not work well together.
How can I narrow down the ~20k strategies into one that works well and is likely not to be over fit?
Thanks for your help.