1. Let me first reconcile the Black-Scholes pricing formula with the idea of prices being determined by supply-and-demand. Even if it is not explicitly said this way, from an equilibrium perspective, the Black-Scholes formula defines the unique price of risk that is consistent with the absence of arbitrage.
In fact, you explicitly use this price when you invoke the Girsanov theorem to derive the Black-Scholes pricing formula. The martingale process you invoke in that theorem to get the change of measure is the marginal utility of consumption, it is your stochastic discount factor. So, you're implicitly modeling the oequilibrium outcome of supply and demand with the only question being "is it good enough a model for what you want to do?"
The primary interest of Black-Scholes? It's valid for an entire class of equilibrium models.
2. The Black-Scholes-Merton framework does have the very convenient feature of allowing us to work in the implied volatility space. It's very useful in and of itself because it gives you a natural unit to compare many very different contracts. In fact, it's a very smart way to calibrate more complicated pricing model: you minimize the squared difference between observed implied and model implied volatilities.
However, it's not the only use for it. Christoffersen and Jacobs (2004) published a paper in Management Science back in 2004 where they showed that if you taylor your estimation/calibration strategy to your goals, a "cheated" version of Black-Scholes is not easy to beat empirically. Specifically, per Girsanov's theorem, the volatility under the risk-neutral and the physical measure should be the same for BS, but if you do not take the model too seriously, you can force fit the model to the implied volatiltiy surface -- for example, fit a quadratic polynomial on the implied volatility surface and use fitted values as inputs. Or you could try to force fit the model to get good hedging by minimizing a loss on hedging errors.
In practice, what people do seems to be something finding smart ways to put the wrong number in the wrong model to get the job done. BS has the advantage of being simple to use, simple to understand and super efficient numerically speaking. Note, however, that if you are trying to price longer term options, the volatiltiy smile usually isn't all that bad. It's a lot flatter. At that point, BS is actually a pretty good guide.
3. People are presumably wrong all the time.
4. The problem with statements about arbitrage opportunities is that they are actually a joint statement about (1) a pricing equation and (2) the observation that observed prices aren't those implied by the pricing equation. What you observe is that a model fails, but you don't know if it's because you have the wrong model or if it is because the market is wrong.
Now, Giglio and Kelly (2018) have a paper on excess volatility for many types of securities, including equity options. They show that the no-arbitrage restrictions implied by affine (or exponentially affine) Q cash flow dynamics in term structure models are violated across the board. They then, very interestingly, try hard to "rescue" the no arbitrage by seeing if they can find ways to explain away the result as an issue of model misspecification -- they can't. It doesn't mean it cannot be done, but it does mean that if arbitrages do not exist, it's really not obvious why.
Their final conclusion is that there is arbitrage because of a form of investor overreaction and this arbitrage subsist because trying to take advantage of it is too costly (transactions costs, infrequent trading and long holding periods make the trade really pointless).
5. First of all, in the presence of conditional non-normality in returns, realized volatility isn't even a valid estimate of volatility because it is polluted by higher moments. A similar comment applies for the VIX and future implied volatility (see Martin (2017) for details). Second of all, assuming you do a better job of estimating volatility for the underlying, in some models, it is theoretically okay to use volatility under the physical process. That's just Girsanov's theorem. The problem is that it doesn't seem to work too well. Empirically, no matter how you attack the problem, you find that Q-volatility is always higher than P-volatility. It used to be known as a volatility puzzle, but we now have a clear explanation: there probably is a negative variance risk premium.
You can get such a premium in many ways:
1. Stochastic volatility
2. A quadratic pricing kernel
3. In a GARCH model, more than one period ahead;
4. In a GARCH model with conditional nonnormality of returns (e.g., GED shocks or Inverse Gaussian innovations)
You can look up Christoffersen, Elkamhi, Feunou and Jacobs (2010) for details on this, but it's very technical. A good discussion on the variance risk premium can be also found in the Bégin, Dorion and Gauthier (2020) paper. In all those cases, it's more or less easy to show that risk-neutral volatiltiy will be on average higher than the physical volatility.
6. Note that in all the cases I mentionned above, you can have a negative variance risk premium (i.e., the Q-volatility is larger than the P-volatility) and they all impose no-arbitrage. For the issue with practionners, your comment is correct and you figured out the solution yourself: if they believe in their model. Well, the answer is that no one does.