If there is deflation does the Inflation receiver not only pay the fixed leg but also receives a reduced CPI?
I.e. does he lose twice?
sorry, I cannot comment yet as I have less than 50 reputation. So I will to have to put another answer here.
I think you are probably right. This is the same as you are paying fixed on a interest rate swap contract but the floating rate goes negative. In this case, in theory, you will have to pay both legs.
usually, for an inflation swap (either zero coupon swaps or year on year swaps), the quote on the fixed side is a rate and the payment of the floating side is the return rate of the inflation indices in a period, which makes inflation swap case similar to interest rate swap case discussed above.
For inflation linked bonds, there is usually a floor with strike set to zero to protect it from going deflation.