Playing with Random Forest Classifier, I am wondering what could cause in a 80:20 split the test results to perform better than in a 90:10 split?

With 2000+ data points and:

  • with 80:20 split, considering only the test set, the model generates 150 signals with around 55% accuracy
  • with 90:10 split, considering only the test set, the model generates 77 signals with around 49% accuracy

From the images, it seems like the more the model 'sees', the worse it gets.



  • 3
    $\begingroup$ My guess is this shows a high sensitivity to the sample data chosen (not the amount of data). You might compare using the first 90% of the data versus using the last 90% of the data for example; same amount of data but different choice of time period. My guess is you get different results. $\endgroup$
    – nbbo2
    Jun 2, 2021 at 20:31
  • $\begingroup$ This could also be due to overfitting. Perhaps your tree has more depth in the 90% sample compared to the 80% sample--but the more depth explains noise that is not present in your test sample. $\endgroup$
    – Jason p
    Jun 16, 2021 at 14:15


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.