EDITED FROM INITIAL POST
I am sorry that I took so long to edit this. I have been swamped. Let me first motivate quadratic utility before we tear it apart and talk about the implications of tearing it apart.
EDIT TO INCLUDE ANSWER TO 1
I believe implicit in your posting that you are scratching your head because it would seem that quadratic utility has potentially bizarre properties, and for applied people to use it, they must be fixing it in some manner and have a deep and profound reason to use it. In other words, it appears your thought is that people did the science, arrived at quadratic utility and now everyone uses it. That reverses how it happened. Quadratic utility is a backfilling of a problem.
Let’s start with the implicit thoughts that are in many economist’s minds when they see a quadratic utility function. The function you proposed is $$U(x)=x+\frac{\lambda}{2}(x-\mu)^2.$$
The first thing to note is that when an economist sees a $\mu$, they mentally map that to a population mean, just as when they see a $\sigma^2$, it must be a variance. Why?
It is just a known fixed point. The known part is critical because this entire math collapses in the general case when that is not true. This is a gamble similar to craps or a public lottery. It is not similar to cases such as the stock market, commodities, or for that matter, most investments.
Quadratic loss or its negative, quadratic utility, are precision and accuracy utilities. One could imagine throwing darts where the bullseye is $\mu$. If you had two throwing strategies for a given dartboard where one strategy was less accurate, then the more accurate strategy would dominate the less accurate one. You could show that to get people to use a less accurate behavioral rule, a higher payoff would have to be offered. Of course, that is the point of mean-variance finance and is also why the Black Sox Scandal happened.
Now to understand why it is even discussed, you have to realize that it was backed into because it implies that there is a preference for a targeted wealth and a dispreference for wealth on either side of that target.
So you noticed the apparent problem that marginal utility is negative after someone becomes wealthy enough, but it actually has another counter-factual problem with it as well. If we define the Arrow-Pratt measure as $$A(x)=\frac{U''(x)}{U'(x)}$$ then $$A(x)=\frac{\lambda}{1-\lambda{x}},$$ then we get a weird result in that the marginal Arrow-Pratt measure is $$A’(x)=\frac{\lambda^2}{(1-x\lambda)^2}.$$ The implication of which is that the wealthy are increasingly risk-averse, and hedge funds should be filled with guaranteed investment contracts, certificates of deposit and AAA-rated sovereigns.
So how do quants adjust for the absurdities?
They don’t because of the link to ordinary least squares. To adjust for the absurdities would require quadratic loss over a range and then another loss function outside that range. You would be guaranteed a biased estimator if the assumptions of mean-variance finance hold. You will have assured yourself an inaccurate estimator if the assumptions hold. That may be less than fun in litigation if you cannot really explain why you do anything you do to a jury of laypeople.
Now let us work backward to figure out how the topic arose in the first place.
The proto-works of mean-variance finance were intellectually very primitive. The early papers are shockingly lax in rigor if seen from current standards. That ignores the fact that Markowitz feels obligated to explain what a mean and a variance are for his audience. There were two early strains of work going on.
The first was to move forward with von Neumann and Morgenstern utility and to solve uncertainty in a utility framework. The second was the Markowitz framework. Both frameworks use a calculus that assumes that the parameters are known and fixed. A telling warning is that the Markowitz framework collapses under Bayesian axioms in the general case.
Since all admissible estimators are either a Bayesian estimator or the limiting form of a Bayesian estimator, when models do not map together, at least at the limit, then that is a good indication that the Frequentist model is at best problematic, which of course is empirically the case.
The fact that the models are inadmissible is actually frightening. I wrote a long blog post about this at Data Science Central at https://www.datasciencecentral.com/profiles/blogs/tool-induced-arbitrage-opportunities-also-how-to-cut-cakes. That post is on coherence, and it is also true that mean-variance models are not coherent. The post that follows that post shows why they are inadmissible.
Getting back to history, as authors started exploring the models carefully, they arrived at two cases. The first is the case where the error terms (not residuals) are normally distributed. The second is the case where quadratic utility is present. Two other technical issues are not in the economic literature but have to be true for quadratic loss to create mean-variance results.
If the errors are normal, plus a bunch of other requirements such as complete knowledge, identical preferences, infinite liquidity, and so forth, then models similar to the CAPM and Black-Scholes hold. This extends to a variety of less discussed models, not just the primary models.
If quadratic utility holds, the variance is defined, a covariance exists and is defined as a real number, then models like the CAPM always maximize utility independent of the distribution chosen.
The discovery regarding quadratic utility was probably a fortuitous convergence of thinking as an alternative to normality was becoming needed by 1963 when Mandelbrot published “On The Variation Of Certain Speculative Prices.”
To understand why that may be a problem, this fifteen-minute video https://youtu.be/R3fcVUBgIZw discusses the distributions that should be present in returns. Even in the logarithmic case, a covariance cannot exist.
Now, as to why quants, as a group, use them.
First, most quants have been trained in nothing else, so the simple answer is that there is no alternative for a large percentage of them because the training is missing to move to the next idea. Second, the mean-variance models have two Nobels, even though there isn’t a single validation study supporting them and an extensive literature showing anomalies or falsifying them.
Why do the Nobels matter? Because they provide legal safe-haven under the Uniform Prudent Investors Act and common law. Even if they are pure witch doctor material, they protect the user from potentially devastating lawsuits in the face of complete empirical falsification.
The third reason is more profound, though, and far less cynical. Economics is a field built around unbiased estimators. All Pearson-Neyman estimators have a mean buried in them somewhere. Even median statistics are built on the mean in rank space. The point at the mean rank is the sample median. If you dig around unbiased estimators enough, you will keep landing in some transformation of a quadratic loss.
There have been attempts to deal with models without a defined variance but the tool loss is incredible. If you dig to the beginning of my blog, I propose a new stochastic calculus that solves these problems. Again, it is too long to rewrite here, but there is a critical issue of training.
The models by force of the assumptions undergirding the calculus used in these models, the parameters are known. When that assumption is dropped, the bottom falls out of the models. There is a paper in 1958 proving that if the parameters are not known, then models like Black-Scholes or the CAPM cannot have a meaningful Frequentist solution.
Quadratic loss undergirds so much estimation theory. It links estimation to L2 spaces which also make them Hilbert spaces. That opens up a mountain of tools. The very fact that most undergraduates are either taught bi-variate ordinary least squares or ANOVA is a testament of how fundamental this link is in the pedagogy and how valuable it is in standard problem-solving. To take away OLS, GLS, FGLS, 2SLS and 3SLS is going to be like pulling teeth.
EDIT TO INCLUDE 6
You can find a bibliography at http://www.e-m-h.org/bibliography.html.
EDIT TO SOLVE 2
I erred in my original post on $\lambda$. I wrote the sentences sloppily. There are some special cases in the literature, but you can ignore them as they only have historical interest. Ignore the part of the post that gave rise to 3
However, if $\mu>\max({x})$ then you solve the absurdity but create an estimation nightmare. If $\hat{\mu}$ estimates $\mu$ and $\mu$ is not in the support, then any least squares estimator is going to be meaningless. The models do not contemplate the case where $\mu$ is not inside the support.
Indeed, the CAPM not only assumes support on the entire reals, it assumes infinite liquidity. The consequence of both is that intergenerational slavery is legal. A person has to be able to take losses in excess of their present value and there can be no bankruptcy escape or limitation on lending.
Implicitly 4 is covered above while 5 is dropped as it isn’t relevant to the question.
EDIT
What is the linkage between estimation and these models?
That is a fascinating question. The models are forced by the nature of the underlying mathematics to assume that all parameters are known and that everyone has homogenous preferences.
So no economist should ever need to perform estimation because if they need to know the parameters of IBM they need only sincerely consider purchasing the security and it will be imprinted in their mind. Everyone shares complete knowledge about this game. Why randomness happens is a bit mysterious because there are no borrowing constraints.
The answer to this question has always been that markets behave as if the actors know the true values of the parameters. It is Friedman's pool player analogy writ large.
Under that construction, the economist is not competing and so does have things to estimate, although it still begs the question as to where the randomness is coming from. If it is liquidity because people forget they have no borrowing constraints then the scale parameter should be very very narrow.
The models are linked to the estimation method by their construction. If the data is normally distributed, then the least squares associated methods are the only logical methods. Because quadratic utility is a method to derive estimators, least squares minimizing methods are still the only logical tool.
You have to abandon the models to logically go to places such as Theil's regression, quantile regression or other models built on alternative loss functions.
EDIT
I have been trying to think about how to keep this edit concise. If it is too concise let me know.
There are two types of questions presented here. The first is of doubly bounded distributions, the second is regarding the presence of a zero in the system. Unfortunately, I could give a week-long lecture on both.
Let us ignore short sales for the remainder of the post, not because the outcome is vastly different but because it would require an enormous amount of additional work. Also, let us make prices continuous instead of discrete, without a substantive loss of generality.
As to the former question, it is true that all trades are doubly bounded. The boundary on the left is a hard boundary unless intergenerational slavery is permitted. The boundary on the right is stochastic. The planetary budget constraint has a distribution. It is one of the reasons the data cannot follow a truncated Cauchy distribution.
In fact, if the budget constraint is not met, then the trade fails and there is no numerator. So a return is actually a return given the counterparty's budget constraint is met, which would then be multiplied by the probability that the budget constraint would be met. That is a readily solved problem actually. Nonetheless, the effect is small until you get out into the right tail. It is not negligible, but it also is not large either.
If you were managing a million dollars, it is readily ignored. If you were managing a billion dollars, it probably should not be ignored.
Nonetheless, when you multiply a distribution without a variance by one with one, you end up with no variance.
The second one is possible to talk about but without perfect correctness. The zero sort of doesn't matter.
The first reason is that the zero is a removable pole so it is ignored, but if you start thinking about the limit as the price goes to zero then you would think that is the source of infinite variance but it is not.
A simple counter-example would be two securities with identical truncated Cauchy distributions on their returns. So, $\mu_1=\mu_2$ and $\gamma_1=\gamma_2$. The first security is priced at \$1 and the second at $10. The distribution with the higher price isn't more or less Cauchy distributed.
The reason this effect exists is that $\mathbb{R}^2$ is not an ordered set. There is no true zero. Since $p_t\times{p}_{t+1}\subset\mathbb{R}^2$, you have to impose more definition on the problem to solve it.
The traditional solution of $\Pr(Z=\frac{Y}{X})$ turns out to have no useful solution in Cartesian coordinates, though I didn't realize that for a long time. For a variety of reasons, you end up in a dead-end. Although the methodology is completely valid, it generates a string of insurmountable problems for an economist. It is credible that it could work well for people in another domain, but for what turn out to be a laundry list of reasons, economists can't use it. The switch to polar coordinates alleviates the issue.
I have empirically tested the approximation of the Normal to the Cauchy. In theory, using the normal to approximate a truncated Cauchy results in a catastrophic information loss. Using methods designed for the normal with the Cauchy results in a case where a population of sixty million end of day trades has the same statistical power as randomly choosing two prices from a randomly chosen security and using them as their slope as the sole estimator of the center of location.
Calculating this, instead, in terms of the K-L divergence, had something on the order of 8.6 million leading zeroes on the percentage of information retained over the population of trades in the CRSP universe. That isn't a good estimator though as it is really an asymptotic approximation. There is always no more nor less than a sample size of one for any parameter estimator when the Normal approximates the Cauchy or its cousins.
As to the saw-like pattern, that exceeds the scope of the original question, but is complicated by an empirical issue in the real data and cannot be discussed without getting into extreme value theory. It would be its own set of academic papers.
