Error term
The error term tells the difference between the theoretical and the observed values of the dependent variable. As such it is referred to the single observations. In your equation, as you say, $i$ stands for the $i$-th share, therefore the meaning of $\varepsilon_i$ is unclear (as it can't be the $i$-th observation) and the related equations too. Perhaps you should add to the given equation(s) a second index, say $j$, related to the $j$-th observation.
CAPM and alpha
There is a problem with your question in that the CAPM assumes alphas to be zeros. Your claims is trivially true since the average of many zeros will still be zero, but perhaps this is not what you are looking for.
CAPM test
CAPM assumptions can be violated, therefore one might want to test whether in actual markets the share alphas are zeros or not. When alphas are not zero, you may wonder what is the relation between their value with respect to a portfolio and its single constituents.
Portfolio alpha with a linear model
Consider a generic linear model relating a portfolio and the market return in excess of the risk-free rate:
$$
r_{pj} -r_f = \alpha_p + \beta_p (r_{mj} -r_f ) + \varepsilon_{pj}
$$
where the equation is related to the $j$-th observation of the portfolio and market excess return.
The same model with respect to the $i$-th portfolio constituent is:
$$
r_{ij} -r_f = \alpha_i + \beta_i (r_{mj} -r_f ) + \varepsilon_{ij}
$$
The portfolio is equally weighted, therefore (for each $j$-th observation)
\begin{align}
r_{pj}=\frac{1}{n}\sum_j^n r_{ij} \tag{*}\label{*}
\end{align}
By means of some statistical methods one can find the estimated alphas and betas, that is the "best" $\alpha_*, \beta_*$ to minimise the $\varepsilon$-errors between theoretical and observed return values. This is normally found solving:
$$\min_{\alpha_i,\,\beta_i} \sum_{j=1}^n \hat{\varepsilon}_{ij}^{\,2} =
\min_{\alpha_i,\,\beta_i} \sum_{j=1}^n \left(
r_{ij} -r_f - \alpha_i - \beta_i (r_{mj} -r_f ) \right)
$$
and
$$\min_{\alpha_p,\,\beta_p} \sum_{j=1}^n \hat{\varepsilon}_{pj}^{\,2} =
\min_{\alpha_p,\,\beta_p} \sum_{j=1}^n \left(
r_{pj} -r_f - \alpha_p - \beta_p (r_{mj} -r_f ) \right)
$$
For a general linear model, $ y = \alpha + \beta x$, the solution (estimator) is known to be (see for example here):
\begin{align} \tag{**}\label{**}
\hat\beta = \frac{ \sum\limits_{j=1}^{N} (x_{j}-\bar{x})(y_{j}-\bar{y}) }{ \sum\limits_{j=1}^{N} (x_{j}-\bar{x})^2 }
\end{align}
where $\bar{*}$ is the sample mean, e.g. (replacing summation dummy to avoid name clash):
$$
\bar{x}=\frac{1}{N}\sum_h^N x_h
$$
Substituting to \eqref{**} our excess returns, with respect to the $i$-th share beta, we get:
\begin{align}
\hat\beta_i &=
\frac{ \sum\limits_{j=1}^{N} \left( r_{mj} -r_f - \frac{1}{N}\sum\limits_h^N (r_{mh} -r_f) \right)
\left( r_{ij} -r_f - \frac{1}{N}\sum\limits_h^N (r_{ih} -r_f) \right) }
{ \sum\limits_{j=1}^{N} \left( r_{mj} -r_f -\frac{1}{N}\sum\limits_h^N (r_{mh} -r_f) \right)^2 } \notag\\
&=\frac{ \sum\limits_{j=1}^{N} \left( r_{mj} - \frac{1}{N}\sum\limits_h^N r_{mh} \right)
\left( r_{ij} - \frac{1}{N}\sum\limits_h^N r_{ih} \right) }
{ \sum\limits_{j=1}^{N} \left( r_{mj} -\frac{1}{N}\sum\limits_h^N r_{mh} \right)^2 }
\notag
\end{align}
As for the portfolio beta, we have:
\begin{align}
\hat\beta_p
&=\frac{ \sum\limits_{j=1}^{N} \left( r_{mj} - \frac{1}{N}\sum\limits_h^N r_{mh} \right)
\left( r_{pj} - \frac{1}{N}\sum\limits_h^N r_{ph} \right) }
{ \sum\limits_{j=1}^{N} \left( r_{mj} -\frac{1}{N}\sum\limits_h^N r_{mh} \right)^2 }
\notag
\end{align}
Replacing the portfolio return definition from \eqref{*}, we obtain:
\begin{align}
\hat\beta_p
&=\frac{ \sum\limits_{j=1}^{N} \left( r_{mj} - \frac{1}{N}\sum\limits_h^N r_{mh} \right)
\left( \frac{1}{n}\sum\limits_i^n r_{ij}
-\frac{1}{N}\sum\limits_h^N \frac{1}{n}\sum\limits_i^n r_{ih} \right) }
{ \sum\limits_{j=1}^{N} \left( r_{mj} -\frac{1}{N}\sum\limits_h^N r_{mh} \right)^2 }
\notag\\
&=\frac{ \sum\limits_{j=1}^{N} \left( r_{mj} - \frac{1}{N}\sum\limits_h^N r_{mh} \right)
\frac{1}{n}\sum\limits_i^n \left( r_{ij}
-\frac{1}{N}\sum\limits_h^N r_{ih} \right) }
{ \sum\limits_{j=1}^{N} \left( r_{mj} -\frac{1}{N}\sum\limits_h^N r_{mh} \right)^2 }
\notag\\
&= \frac{1}{n}\sum\limits_i^n
\frac{ \sum\limits_{j=1}^{N} \left( r_{mj} - \frac{1}{N}\sum\limits_h^N r_{mh} \right)
\left( r_{ij} -\frac{1}{N}\sum\limits_h^N r_{ih} \right) }
{ \sum\limits_{j=1}^{N} \left( r_{mj} -\frac{1}{N}\sum\limits_h^N r_{mh} \right)^2 } =
\frac{1}{n}\sum\limits_i^n \hat\beta_i \notag
\end{align}
As for alpha the general estimator, this is:
$$
\hat\alpha = \bar{y} - \hat\beta\,\bar{x}
$$
Therefore:
$$
\hat\alpha_i = \frac{1}{N}\sum\limits_h^N (r_{ih} -r_f) - \hat\beta_i\,\frac{1}{N}\sum\limits_h^N (r_{mh} -r_f)
$$
and
$$
\hat\alpha_p = \frac{1}{N}\sum\limits_h^N (r_{ph} -r_f) - \hat\beta_p\, \frac{1}{N}\sum\limits_h^N (r_{mh} -r_f)
$$
Replacing the portfolio return definition from \eqref{*}:
\begin{align}
\hat\alpha_p &= \frac{1}{N}\sum\limits_h^N \left(\frac{1}{n}\sum\limits_i^n r_{ih} -r_f\right)
- \frac{1}{n}\sum\limits_i^n \hat\beta_i \frac{1}{N}\sum\limits_h^N (r_{mh} -r_f) \notag\\
&= \frac{1}{N}\sum\limits_h^N \frac{1}{n}\sum\limits_i^n \left( r_{ih} -r_f\right)
- \frac{1}{n}\sum\limits_i^n \hat\beta_i \frac{1}{N}\sum\limits_h^N (r_{mh} -r_f) \notag\\
&= \frac{1}{n}\sum\limits_i^n \left( \frac{1}{N}\sum\limits_h^N ( r_{ih} -r_f )
- \hat\beta_i \frac{1}{N}\sum\limits_h^N (r_{mh} -r_f)\right)
= \frac{1}{n}\sum\limits_i^n \hat\alpha_i
\notag
\end{align}
This is just to give you a general idea of the problem.
