As @byouness pointed out, the answer to question (a) is Itô's Lemma. You know that $\mathrm{d}S_t=\alpha S_t\mathrm{d}t+\sigma S_t\mathrm{d}W_t$, i.e. $(S_t)$ is a geometric Brownian motion. Let $f(x)=x^2$ with $f_x=2x$ and $f_{xx}=2$. Then, $X_t=f(S_t)=S_t^2$ and
\begin{align*}
\mathrm{d}X_t &=\left(\alpha S_t f_x+\frac{1}{2}\sigma^2 S_t^2 f_{xx}\right)\mathrm{d}t+\sigma S_t f_x \mathrm{d}W_t \\
&= \left(2\alpha S_t^2 +\frac{1}{2}\sigma^2 S_t^2 2\right)\mathrm{d}t+\sigma S_t 2S_t\mathrm{d}W_t \\
&= 2\left(\alpha +\frac{1}{2}\sigma^2\right)X_t\mathrm{d}t+2\sigma X_t\mathrm{d}W_t,
\end{align*}
i.e. $X_t=S_t^2$ is again a geometric Brownian motion. In fact, any power of a geometric Brownian motion, $S_t^n$, is again a geoemtric Brownian motion.
Regarding part (b), recall that for any Itô process $\mathrm{d}X_t=\mu(t,X_t)\mathrm{d}t+\sigma(t,X_t)\mathrm{d}W_t$, the quadratic variation of $X_t$ is given by $$[X,X]_t=\int_0^t\sigma(s,X_s)^2\mathrm{d}s,$$ i.e. $\mathrm{d}[X,X]_t=\sigma(t,X_t)^2\mathrm{d}t$. In our case, $X_t$ is a geometric Brownian motion with $\sigma(t,X_t)=2\sigma X_t$. Thus,
\begin{align*}
[X,X]_t=4\sigma^2\int_0^t S_s^4\mathrm{d}s.
\end{align*}
Using Fubini's theorem, you could at least compute the expected quadratic variation of $X_t=S_t^2$.
