0
$\begingroup$

Playing with Random Forest Classifier, I am wondering what could cause in a 80:20 split the test results to perform better than in a 90:10 split?

With 2000+ data points and:

  • with 80:20 split, considering only the test set, the model generates 150 signals with around 55% accuracy
  • with 90:10 split, considering only the test set, the model generates 77 signals with around 49% accuracy

From the images, it seems like the more the model 'sees', the worse it gets.

80:20

90:10

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ My guess is this shows a high sensitivity to the sample data chosen (not the amount of data). You might compare using the first 90% of the data versus using the last 90% of the data for example; same amount of data but different choice of time period. My guess is you get different results. $\endgroup$
    – nbbo2
    Jun 2, 2021 at 20:31
  • $\begingroup$ This could also be due to overfitting. Perhaps your tree has more depth in the 90% sample compared to the 80% sample--but the more depth explains noise that is not present in your test sample. $\endgroup$
    – Jason p
    Jun 16, 2021 at 14:15

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.