Interest Rate Convexity - Fundamental Question

I have a very basic question around convexity adjustments in swap valuations. I am comfortable with the mathematical derivation of the convexity adjustment.

My question relates to when and why a convexity adjustment is deemed necessary in some cases and not others.

The general rule seems to be that:

1. For a vanilla IRS, or any other variation in which the floating rate applied to a given period $(T_i, T_{i+1})$ is observed at $T_i$, the "correct" approach is to assume that the expected future spot rates are equal to forward rates, and thus a convexity adjustment is not needed.

2. If on the other hand, the floating rate is set at the end of the period, as in a Libor-in-arrears swap, the "correct" approach is to apply a convexity adjustment to the forward rate to arrive at the expected future swap rate.

Why is the "forward rates will be realized" assumption valid for a simple FRA-based IRS swap, and not for a Libor-in-arrears swap?

Everything I have read seems to simply state this as fact without providing some sort of explanation of why this convention is used?

Vanilla Swap: We observe the LIBOR $L(T_i, T_{i+1})$ at time $T_i$ and payment occurs at $T_{i+1}$. Therefore the correct measure is the classic "forward-risk-neutral" measure with respect to a ZCB expiring at time $T_{i+1}$. In this measure, $L(T_i, T_{i+1})$ is a martingale and therefore $L(t, T_i, T_{i+1})$ = $\mathbb{E}^Q[L(T_i, T_i, T_{i+1})]$ - in other words in this case the forward LIBOR in question is equal to its expected future spot value (in the aforementioned measure) and hence no convexity adjustment is needed.
LIBOR-In-Arrears Swap: We observe the same LIBOR, again at $T_i$, but this time payment also occurs at $T_i$. Therefore the appropriate measure is one which is forward risk neutral with respect to a ZCB expiring at time $T_{i}$. However, in this measure, the LIBOR in question $L(T_i, T_{i+1})$ is not a martingale. Hence the forward LIBOR is not equal to the expected future spot rate, hence the need for the convexity adjustment.
You're almost right, I think. You shouldn't talk about $$L(T_i,T_{i+1})$$ being a martingale, since this is a RV not observed until $$T_i$$. Rather we should talk about $$L(t,T)$$, the regular Libor forward rate, and $$L'(t,T)$$, the fair rate for an arrears FRA. We have that $$L(t,T)$$ is a martingale under the $$ZCB(T_{i+1})$$ measure, as you say. But also, $$L'(t,T)$$ is a martingale under the $$ZCB(T_i)$$ measure. So the situation is more symmetric than you think. What we can say is that the convexity adjustment, $$L'(t,T) - L(t,T)$$ is not a martingale under either measure, and that this spread is a function of volatility. However it is arbitrary which one of these has the volatility exposure. The issue is decided in the end by the decision (in most models) that $$Z(t,T)$$ are given, and have no volatility exposure by definition. For example, if volatility is zero, most models say that Libor follows the regular forward rates, not the arrears ones. But there's no reason why you couldn't write down a model where the arrears FRAs are fundamental. Then the ZCB would have volatility exposure. Hope that doesn't confuse things.