Strictly speaking, indices such as the VIX are built to approximate the expected variance (of log-returns) that would effectively realise under a pure diffusion setting (i.e. no jumps)
$$ \frac{dX_t}{X_t} = \mu(t) dt + \sigma(t,.) dW_t^{\mathbb{Q}} $$
Writing out the equations (*) yields the famous static replication formula in terms of strike-weighted OTMF options that you refer to, along with the constant Gamma portfolio interpretation you mention.
Although many people claim that this constitutes a model-free estimate of future variance, this is not completely true since pure diffusion is assumed all the way (but this does not preclude the fact that the diffusion coefficient $\sigma(t,.)$ could exhibit its own source of stochasticity, i.e. that the true diffusion process could be Heston or local volatility or GBM... hence the model-free adjective).
IMHO, you should really see volatility indices such as the VIX as expected realised variances assuming pure diffusion, in a similar way you look at the implied volatility of an option as the figure you should use in a (wrong) GBM setting to retrieve the (right) observed market price.
I hope this clears your confusion.
(*) This requires approximating the sample variance of the log-returns observed over $[0,t]$ as the quadratic variation $\langle \ln X \rangle_t$
[Edit] More details on the derivation + constant Vega feature in this excellent note by Fabrice Rouah.