The term under which this problems is usually known is cardinality-constrained portfolio optimization. This is a broad and active field of research, with new approaches coming out continuously. In general, research is done in the direction of complex methods using simulated annealing, particle swarm optimizations and genetic algorithms, your mileage may vary using these in practice. Depending on your hardware and on your time to implement techniques, I should caution that $N=10 000$ might be ambitious.
In the following I am listing a few approaches that might be of interest. The articles should also give you a good sense of which computation times and values for $N$ are achievable.
In this paper, the authors propose several heuristic methods: the continuous-relaxation-based method (Heuristic 1), the $l_1$-norm-based solution (Heuristic 2), the integer programming-based solution (Heuristic 3), and the SDP-based solution (Heuristic 4). Utilizing real-life stock data and simulated data sets, the paper assesses the efficiency and accuracy of the proposed techniques. The computational experiments reveal that their SDP-based solution effectively generates high-quality outcomes, outperforming the commercial MIQP solver when time is limited. These methods have apparently been adopted in practice.
This paper examines such a cardinality-constrained index tracking model and introduces an efficient nonmonotone projected gradient (NPG) method to address the problem. Typically, this method solves multiple projected gradient subproblems at each iteration, with each subproblem having a closed-form solution that can be computed in linear time. Under appropriate assumptions, it is established that any accumulation point of the sequence generated by the NPG method is a local minimizer of the cardinality-constrained index tracking issue. The authors also perform empirical tests comparing their approach with the hybrid evolutionary algorithm and the hybrid half thresholding algorithm for index tracking. The results show that their method generally yields sparse portfolios with reduced out-of-sample tracking errors and improved consistency between in-sample and out-of-sample tracking errors.
When addressing large-scale cardinality-constrained Markowitz mean-variance portfolio investment problems, exact solvers may struggle to determine some efficient portfolios within a reasonable time frame. In these instances, information regarding the distance between the best feasible solution found before the optimization process halts and the true efficient solution is not available. In this article, the authors showcase how to provide such information to a decision-maker. Their goal is to employ the concept of lower and upper bounds on objective function values of an efficient portfolio, as developed in their previous works. They demonstrate the proposed approach using a large-scale dataset based on real data. They tackle cases where a top-tier commercial mixed-integer quadratic programming solver fails to deliver efficient portfolios attempted to be derived by Chebyshev scalarization of the bi-objective optimization problem within a given time limit. In this situation, they suggest transforming purely technical information provided by the solver into information that can be utilized in navigating the efficient frontier of the cardinality-constrained Markowitz mean-variance portfolio investment problem.
The authors propose an innovative approach to address cardinality in portfolio selection by introducing a biobjective cardinality/mean-variance problem. This allows investors to analyze the efficient tradeoff between return-risk and the number of active positions. Recent advancements in multiobjective optimization without derivatives enable them to robustly compute (in-sample) the entire cardinality/mean-variance efficient frontier for various datasets and mean-variance models. Their findings indicate that a significant number of efficient cardinality/mean-variance portfolios can outperform (out-of-sample) the naive strategy while maintaining relatively low transaction costs.