You do not need to know too much Fourier theory. Of course it helps, but it is not necessary. There are many different applications of Fourier methods e.g.
- Carr Madan (1999): you damp the option price as a function of the log-strike price and compute the fourier transform of the entire option price
- Bakshi and Madan (2000), Duffie, Pan and Singleton (2000): You Fourier transform the density and obtain a formula similar to the Black-Scholes solution. A decomposition of the call option price in Delta and exercise probability.
- Lewis (2001): you Fourier transform the option payoff (these have typically polynomial growth, so you need a generalised Fourier transform which extends to the complex domain. This approach requires some complex analysis (residue theorem) and is in fact equivalent to Carr Madan (1999): Choosing a contour to integrate along or an optimal damping factor is the same question.
- CONV and COS method: These are stable numerical algorithms which, for instance, allow very fast computation of European, Bermudan and (as a limit) American options.
In general, keep in mind, that Fourier methods do not apply to strongly path dependent options (asians, look-backs etc.) Furthermore, the main idea is always to replace the integral (expectation) which occurs from risk-neutral pricing by another integral which contains the characteristic function.
Regarding the maths, there are two things I believe are particular helpful.
Gil-Paelz inversion theorem
What is option pricing about? Assuming a model for the distribution of $S_T$ and compute the (discounted) expectation of the payoff. This is an integral of payoff times density. Many processes and models (e.g. SVJ, VG, NIG, CGMY etc) have complicated densities but easy characteristic functions. Note that
\begin{align*}
\varphi_{\ln(S_T)}(u) = \mathbb{E}^\mathbb{Q}\left[e^{iu\ln(S_T)}\right] = \int_\mathbb{R} e^{iux} f_{\ln(S_T)}(x) \mathrm{d}x,
\end{align*}
where $f_{\ln(S_T)}$ is the risk-neutral density of $\ln(S_T)$. Thus, the characterisitc function of $\ln(S_T)$ is the Fourier transform of its density $f_{\ln(S_T)}$. Hence, $\varphi_{\ln(S_T)}$ captures the distribution of $\ln(S_T)$ and we can show that
\begin{align*}
F_{\ln(S_T)}(x) &= \frac{1}{2}+\frac{1}{2\pi} \int_0^\infty \frac{e^{iux}\varphi_{\ln(S_T)}(-u)-e^{-iux}\varphi_{\ln(S_T)}(u)}{iu}\mathrm{d}u.
\end{align*}
This will help a lot in deriving different option pricing formulae. Note that we typically consider $\ln(S_T)$ rather than $S_T$ since the characteristic function already involves somehow the exponential function.
Uncertainty principle. This is a concept from physics and states that if $f$ ''spreads out widely'', then its Fourier transform $\hat{f}$ is rather peaked and has a ''small'' support. This means that if you have an option with a short maturity, the density function will be peaked since there is no time for large movements. After all, a stock won't move a lot in a week or two. Thus, the chracterisitic function, $\hat{f}$, will spread out a lot and due to the oscillating behaviour, it may be challenging to integrate numerically.
Schmelzle wrote a nice survey paper but his work includes some errors in the section about the Lewis (2001) approach. If you're interested in the implementation of these models, have a look at Hirsa.
P.S. When I looked the first time at Fourier methods, my background was real analysis and probability theory. Just like yours. And I could follow it and understand it just fine. Don't you worry :)