For any given process for the short rate $\{r_t,, t >0\}$, the price at time $t$ of a zero-coupon bond with maturity $T$, where $t\le T$, is given by
\begin{align*}
P(t, T) = E\left(e^{-\int_t^T r_s ds}\,\big|\, \mathcal{F}_t\right).
\end{align*}
Since, for $t\le T$,
\begin{align*}
\frac{P(t, T)}{e^{\int_0^tr_s ds}} = E\left(e^{-\int_0^T r_s ds}\,\big|\, \mathcal{F}_t\right)
\end{align*}
is a martingale under the risk-neutral measure, we can assume that the dynamics for $r_t$ is already defined in the risk-neutral measure.
For the forward rate $f(t, T)$, we note that $r_t = f(t, t)$ and
\begin{align*}
P(t, T) = e^{-\int_t^T f(t, u)du}. \tag{1}
\end{align*}
We assume that $f(t, T)$ follows, under the risk-neutral measure, the HJM model, that is,
\begin{align*}
df(t, T) = \alpha(t, T) dt + \sigma(t, T) dW_t,
\end{align*}
where $\{W_t, \, t \ge 0\}$ is a standard Brownian motion. From $(1)$,
\begin{align*}
d\ln P(t, T) &= f(t, t) dt -\int_t^T df(t, u) du\\
&=r_t dt - \left(\int_t^T \alpha(t, u) du\right)dt - \left(\int_t^T \sigma(t, u) du\right)dW_t.
\end{align*}
Then
\begin{align*}
\frac{dP(t, T)}{P(t, T)} &= \frac{1}{P(t, T)}d\left(e^{\ln P(t, T)} \right)\\
&=\frac{1}{P(t, T)}\left(e^{\ln P(t, T)} d\ln P(t, T) + \frac{1}{2}e^{\ln P(t, T)} d\langle \ln P, \ln P\rangle_t\right)\\
&=\left(r_t - \int_t^T \alpha(t, u) du +\frac{1}{2}\left(\int_t^T \sigma(t, u) du\right)^2 \right)dt - \left(\int_t^T \sigma(t, u) du\right)dW_t.
\end{align*}
Note that, under the risk-neutral measure, the drift term of $dP(t, T)$ is $r_t$. That is,
\begin{align*}
\int_t^T \alpha(t, u) du = \frac{1}{2}\left(\int_t^T \sigma(t, u) du\right)^2.
\end{align*}
Consequently,
\begin{align*}
\alpha(t, T) = \sigma(t, T)\int_t^T \sigma(t, u) du.
\end{align*}