Finite difference is a numerical procedure used to approximate derivatives computation by a linear combination of the value of the function at some specific points. This is particularly useful when solving PDEs and SDEs which involve discretization in both time and state dimensions.
The finite difference method allows to approximate the derivatives of a function, say $f$, at a specific point $x$.
Mathematically, the derivatives of $f$, denoted by $f'$ is computed as follows:
$$f^{\prime}(x) = \lim_{h \to 0} \frac{f(x+h) - f(x)}{h}$$
Computing the finite difference of $f$ consists in taking a step $\varepsilon$ small enough such that $f^{\prime}(x)$ can be approximated by the following:
- Forward difference
$$\frac{f(x+\varepsilon) - f(x)}{\varepsilon}$$
- Backward difference
$$\frac{f(x) - f(x-\varepsilon)}{\varepsilon}$$
- Central difference
$$\frac{f(x+\varepsilon) - f(x-\varepsilon)}{2\varepsilon}$$
Higher-order derivatives such as $f^{\prime\prime}$ can be computed by replacing $f$ by $f^{\prime}$ in the above formulas.
