Derivation of Ito's Lemma

Question

My question is rather intuitive than formal and circles around the derivation of Ito's Lemma. I have seen in a variety of textbooks that by applying Ito's Lemma, one can derive the exact solution of a geometric Brownian Motion. So, if one has

$dS = a(S,t)dt + b(S,t)dZ$

the first part on the right side of the equation is the (deterministic) drift rate and the second term is the stochastic component. Now I have read that ordinary calculus cannot handle the stochastic integral because for example

$\frac{dZ}{dt} = \phi \frac{1}{\sqrt{dt}}\\ \rightarrow \infty \text{ as } dt \rightarrow 0$

But Ito came along and proposed something which culminated in his famous Ito Lemma which helped calculate the stochastic integral. What I don't understand is the step taken from realizing that ordinary calculus does not work in this context to proposing a function $G = G(S,t) \text{ of } S(t)$, Taylor-expaning it, etc.

Or more precisely: What did Ito realize and propose that helped solve the stochastic integral?

Answer 1

Baxter and Rennie say it better than me, so I will summarize them.

Suppose that $N_t$ is not stochastic and $f(.)$ is a smooth function then the Taylor expansion is $$ df(N_t) = f'(N_t)dN_t + \frac{1}{2}f''(N_t)(dN_t)^2 + \frac{1}{3!} f'''(N_t)(dN_t)^3 + \ldots $$ and the term $(dN_T)^2$ and higher terms are zero. Ito showed that this is not the case in the stochastic case. Suppose $W_t$ follows a Brownian motion and let $$ Z_{n, i} = \frac{W\left(\frac{ti}{n}\right) - W\left(\frac{ti}{n}\right)}{\sqrt{t/n}} $$ now consider $$ \int_0^t (dW_t)^2 \approx t \sum_{i=1}^n \frac{Z^2_{n,i}}{n}. $$ If $n \to \infty$ then $\sum_{i=1}^n \frac{Z^2_{n,i}}{n} \to 1$ and thus $\int_0^t (dW_t)^2 = t \neq 0$. Thus the second order term does not cancel (but higher order terms do) and we have an extra term in our derivative.