I'm building stock selection models, and pick top 5 and bottom 5 stocks. Given the variability in Stochastic gradient decent results, they keep changing. One way to get consistent results is to use the random seed, but I'm looking for if there a better way to deal with this. Also how would you interpret the results, i.e. One set of top 5 versus another set of Top 5 picks (3-4 of them are the same, but may differ in ranking). I'm running enough iterations to know this isn't an issue about convergence.
Have you tried to choose an arbitrary number of model, let say 20, each one having its own seed? Then you run your twenty models and use the median of your 20 results as signal. One advantage of that method is that you can also get a confidence estimate of your prediction thanks to the standard deviation of your 20 results.