Introduction
The answer below measures performance against a benchmark. Alternatively, one can ignore benchmarks and use the Sharp Ratio. I'm not aware of any generally accepted and published ranking for a 'good' SR.
The Level II CFA curriculum contains a number of chapters from the book Active Portfolio Management: A Quantitative Approach for Providing Superior Returns and Controlling Risk by Richard C. Grinold and Ronald N. Kahn that suggest an approach to the problem of measuring portfolio manager performance. It is a formalized version of what you propose in your question. The focus of the book is portfolio management and performance is compared against a benchmark. I've summarized the discussion in the book below.
Definining $\alpha$
The first step is to calculate the residual or risk adjusted return, in the CAPM for a portfolio $p$ you would have
$$R_p(t) = \alpha_p + \beta_p \cdot r_\beta(t) + \varepsilon(t)$$
with $\varepsilon(t)$ a zero mean random variable. Of course, you can also use a multi-factor model but lets keep it simple. The residual returns are defined as
$$\theta_p(t) = \alpha_p(t) + \varepsilon(t).$$
Clearly, $\mathrm{E}[\theta_p(t)] = \mathrm{E}[R_p(t) - R_B(t)] = \alpha_p(t)$ where $R_B(t)$ is the expected benchmark return. They convincingly argue that $\alpha_p(t)$ is insufficient to judge performance and risk also should be taken into account.
Defining risk
As said, performance is compared against a benchmark and so the risk is also measured against the benchmark. Risk is denoted by $\omega_p$ and is simply the standard deviation of the residual return
$$\omega_p = \sqrt{\mathrm{Var}[R_p - R_B]}$$.
The information ratio
The information ratio of a portfolio is
$$\mathrm{IR}_p = \frac{\alpha_p}{\omega_p}$$
and the information ratio of a portfolio manager is
$$\mathrm{IR} = \max\{IR_p|p\}.$$
The information ratio has one very convenient property:
The information ratio is independent of the manager's level of aggressiveness (emphasis adjusted by me).
What's good?
According to Grinold and Kahn an 'Exceptional' manager has IR = 1.00, a 'Very good' one has 0.75 and a 'Good' manager has IR = 0.50 (CFA Institute Level II 2014 Volume 6 Derivatives and Portfolio Management, Reading 58, Table 3).
The Fundamental Law
An alternative way of defining and obtaining(!) the IR is the following:
$$\mathrm{IR} = \mathrm{IC} \cdot \sqrt{\mathrm{BR}}$$
where $\mathrm{IC}$ is the Information Coefficient and $\mathrm{BR}$ is breadth. This tells you how to obtain a good information ratio. In very few words: $\mathrm{IC}$ is the correlation of your bets with the ex post realization and $\mathrm{BR}$ the number of independent bets you can make. For example, the breadth of a stock picker depends on the number of stocks he follows and the number of revisions he makes.
By the way, Grinold and Kahn also elaborate on the implementation of your strategy but that's a whole different chapter.