The CAPM is a single-period model that says $$ \mathbb{E}(R^*_i)=\beta\mathbb{E}(R^*_m) $$ where $R^*_i:=R_i-r_f$ is an asset's excess return, $R^*_i:=R_m-r_f$ is the market's excess return and $\beta:=\frac{\text{Cov}(R^*_i,R^*_m)}{\text{Var}(R^*_m)}=\frac{\text{Cov}(R_i,R_m)}{\text{Var}(R_m)}$. The latter equality holds since $r_f$ is just a constant, and shifting random variables by a constant does not change their covariance or their variances.
Now, the CAPM and $\beta$ are usually estimated from multiple periods of data (think Fama-MacBeth or GMM estimation). There, the risk-free rate $r_{f,t}$ is time varying, and so $\frac{\text{Cov}(R^*_{i,t},R^*_{m,t})}{\text{Var}(R^*_{m,t})}\color{red}{\neq}\frac{\text{Cov}(R_{i,t},R_{m,t})}{\text{Var}(R_{m,t})}$. Which of the two expression defines $\beta$ then?
(For simplicity, let us assume $\beta$, $\mathbb{E}(R^*_{i,t})$ and $\mathbb{E}(R^*_{m,t})$ are constant over time. I think this is a common assumption in the more basic applications; correct me if I am wrong.)
Related threads: "Definitions of Beta" and "Beta using only price returns?".