Equation (11) in Kammeyer and Kienitz' paper is a very well-known and popular option pricing formula. It goes back to the work from Lewis (2001), see Theorem 3.2 in Lewis' paper.
Original Formula From Lewis (2001)
The formula in Lewis for the value of a European-style derivative is $$V(S_0) = \frac{e^{-rT}}{2\pi} \int_{\color{red}{i}\nu-\infty}^{\color{red}{i}\nu+\infty}\varphi_T(-z)\hat{w}(z)\text{d}z,$$
where
- $\nu$ is a real number. It defines the path along which we integrate in the complex plane: $\{z\in\mathbb{C}:\text{Im}(z)=\nu\}$. I give more information on this below.
- $\varphi_T$ is the (generalised) characteristic function of $\ln(S_T)$, the terminal log stock price on which our option payoff depends
- $w$ is the payoff function (as a function of $\ln(S_T)$). For a vanilla call option, $w(s)=\max\{e^s-K,0\}$. The function $\hat{w}$ is the (generalised) Fourier transform of the function $w$.
Note: both $\varphi_T$ and $\hat{w}$ are evaluated at points in the complex plane (not necessarily on the real line!)
Your Formula
Kammeyer and Kienitz state that the time-$t$ value of a call option is
$$C(t)=\frac{e^{-r(T-t)}}{2\pi} \int_{c-\infty}^{c+\infty} \varphi(-z)\hat{f}(z)\text{d}z.$$
First, two important points
- There is a tiny typo in the formula. The integral bounds should be $\color{red}{i}c-\infty$ and $\color{red}{i}c+\infty$.
- Your option pricing formula is for the time-$t$ option price. Thus, everything is conditional on $\mathcal{F}_t$, the filtration generated up to time $t$, see this answer. Lewis (2001) simply sets $t=0$.
The rest is identical to the original formula from Lewis. $\varphi$ is the characteristic function of $\ln(S_T)$, conditional on $\mathcal{F}_t$ and $f$ is the payoff function and $\hat{f}$ is its (generalized) Fourier transform.
Fourier transform of the payoff function
Let $f(x)=\max\{e^x-e^k,0\}$ be the payoff of a vanilla call option with strike $K=e^k$. This function is not in $L^1$ and has no traditional Fourier transform! It has, however, a generalized Fourier transform. Normally, if $f:\mathbb{R}\to\mathbb{R}$, then we define the Fourier transform (in finance) to be $\hat{f}:\mathbb{R}\to\mathbb{C}, u\mapsto \int_\mathbb{R} e^{iux}f(x)\text{d}x$. For this integral to exist, $f$ needs to decay rapidly or be of compact support. The payoff function does not satisfy this.
The generalized Fourier transform of $f$ is $\hat{f}:\mathcal{S}_f\subset\mathbb{C}\to\mathbb{C}, u\mapsto \int_\mathbb{R} e^{iux}f(x)\text{d}x$. Thus, it is defined for a subset of the complex numbers! As it turns out this $\mathcal{S}_f$ is a horizontal strip in the complex plane. We can compute the transform for the payoff function as follows
\begin{align}
\hat{f}(u) &= \int_{-\infty}^\infty e^{iux} \left(e^x-e^k\right)^+ \text{d}x \\
&= \int_k^\infty \left(e^{x(iu+1)} - Ke^{iux}\right) \text{d}x \\
&= \left[ \frac{e^{x(iu+1)}}{iu+1} - K\frac{e^{iux}}{iu}\right]_{x=k}^{x=\infty} \\
&= −\frac{e^{ik(u−i)}}{u(u-i)}.
\end{align}
This last step is only valid if the term indeed vanishes as $x\to\infty$. This only happens if $\text{Im}(z)>1$. Thus, the strip for the call option payoff is $\mathcal{S}_f=\{z\in\mathbb{C}:\text{Im}(z)>1\}$. Similarly, for a put option, we have $\mathcal{S}_f=\{z\in\mathbb{C}:\text{Im}(z)<0\}$. These are the strips of integration which the payoff functions have valid Fourier transform. Note that both strips exclude the real line (i.e., there is no standard Fourier transform).
Proof of Lewis' Option Pricing Formula
Starting with standard risk-neutral pricing,
\begin{align}
V &= e^{-rT}\mathbb{E}^\mathbb{Q}[w(\ln(S_T)] \\
&=e^{-rT}\mathbb{E}^\mathbb{Q}\left[\frac{1}{2\pi}\int_{i\nu-\infty}^{i\nu+\infty}e^{-iz\ln(S_T)}\hat{w}(z)\text{d}z\right] \\
&=\frac{e^{-rT}}{2\pi}\int_{i\nu-\infty}^{i\nu+\infty}\mathbb{E}^\mathbb{Q}\left[e^{i(-z)\ln(S_T)}\right]\hat{w}(z)\text{d}z \\
&=\frac{e^{-rT}}{2\pi}\int_{i\nu-\infty}^{i\nu+\infty}\varphi_T(-z)\hat{w}(z)\text{d}z \\
\end{align}
Here, we are just using the definition of (inverse) generalized Fourier transforms and Fubini's theorem. A proof using Plancherel's theorem (or Parseval's theorem) is also possible. For Fubini to apply and the integrals to be well-defined, we need to integrate along a path in the complex where all terms are well-defined, hence the $\nu\in\mathcal{S}_V=\mathcal{S}_w\cap\mathcal{S}_f^*$ condition.