The idea for both processes are similar. For ease of exposition, we consider the simpler one, that is, $X_t = X_0 + \mu t + \sigma W_t$. Then, for $t>s$,
\begin{align*}
P(X_t \le y \mid X_s) &= P(X_t-X_s \le y-X_s \mid X_s)\\
&=P(\mu(t-s)+\sigma(W_t-W_s) \le y-\mu s -\sigma W_s\mid W_s)\\
&=P(\sigma(W_t-W_s) \le y-\mu t -\sigma W_s\mid W_s)\\
&=\Phi\left(\frac{y-\mu t -\sigma W_s}{\sigma\sqrt{t-s}}\right)\\
&=\Phi\left(\frac{y-\mu (t-s) -X_s}{\sigma\sqrt{t-s}}\right).
\end{align*}
That is,
\begin{align*}
P(X_t \le y \mid X_s=x) &=\Phi\left(\frac{y-\mu (t-s) -x}{\sigma\sqrt{t-s}}\right).
\end{align*}
Here, $\Phi$ is the cumulative distribution function of a standard normal random variable.  The transition density function can be obtained subsequently by taking the derivative with respect to $y$.

For the other one, solve $X_t$ first, and then do the similar computation.