A simple way that Ray Dalio suggested is using Risk-Parity portfolio. Even though Modern Portfolio Theory with mean-variance optimization gives great framework for mathematically designing a tradeoff between alpha, risk and costs, some of the drawback of this framework is that one needs to come up with 1) correct alpha and 2) correct estimation of covariance matrix.
Assuming our estimate of covariance matrix is relatively stable, then the only problem that we'd need to deal with is Alpha.
You can have a look at risk-parity optimization where you can set
1) risk contribution of each asset is equal (equal risk contribution optimization)
2) marginal risk contribution of each asset is equal.
In these framework, it removes alpha from your utility function and let the optimizer to choose the optimal portfolio only using historical available data (historical realized volatility). This may not be optimal in realizing the best sharpe, however, it could be a good alternative to the approach that you are taking if you are unsure of your alpha estimation.