First of all, quanto options (options denominated in FOR currency but whose value we wish to determine is in DOM currency) are mainly traded over-the-counter, hence their prices are not likely available, as opposed to non-quanto options (options denominated in DOM currency with payoff also in DOM currency) for wich we have some quotes. The idea of the methodology presented by Jackel is to use the later to determine the price of the former.
Regarding your questions:
What does the author mean with "using the forward as if the option was
entirely domestic"?
The author means to derive/calibrate an implied volatility while forgetting about the quanto feature (i.e. you only focus on the skew of the domestic currency, DOM, and not of the skew of the FX rate DOM/FOR).
But which market data do I use? The underlying asset is quoted in FOR
currency. Do I get all the foreign vanilla options on this asset to
calibrate the model? And then how do I derive the "efficient
volatility" from the calibrated model?
For the implied volatility calibration, just use options data on options written in DOM currency (European or American options without quanto feature, e.g. call & put options written on the S&P 500 (in USD) with a payoff in USD and quoted in USD) as they will give you the skew you are interested in (i.e. DOM currency). Using such data, derive your "efficient volatility" using usual models (Heston, CEV, ...).
Once you have your "efficient volatility" $\hat{ \sigma}_{DOM}$ smile, you can extract the at-the-money volatility (or more generally, the volatility corresponding to the moneyness of the option you want to price) and your quanto option price will be computed as:
$$ C_{quanto}(F',K,\hat{ \sigma}_{DOM},r,T)$$
where $ C_{quanto}$ is the Black 76' formula for pricing an option with strike $K$, interest rate $r$ and time-to-maturity $T$. $F'$ is the so-called effective forward capturing the quanto feature derived from equation (1):
$$F' = F \exp (\hat{c}) = F \exp \left(\hat{ \sigma}_{DOM} \cdot \rho \cdot \hat{\sigma}_{FOR} \cdot T\right)$$
The FX volatility $\hat{ \sigma}_{FOR}$ is determined similarly to $\hat{ \sigma}_{DOM}$ using available options from the market written on the DOM/FOR FX rate. Again, the author suggests to extract the at-the-money volatility. Of course, it is not likely that you find an "exact" at-the-money" option in the market as the underlying price moves constantly.
Note: In my answer, I suggested that you calibrate a smile before extracting the ATM volatility. Of course it is your choice to assume which volatility model to use: you can assume a flat volatility taken from the available option which is the closest to the ATM region. I suggested a calibration to generalize the methodology to non-ATM options by building a smile and then extract the volatility at the desired moneyness level.
