I assume that by "this machinery breaks down" you mean that when it breaks down as theory, but not as a practical tool.

I would say that the exact point where risk neutral pricing approach fails is when the payoff is no more attainable. There exist a precise mathematical characterization for attainable payoffs (see the book of Hans Föllmer, Alexander Schied, "Stochastic Finance: An Introduction in Discrete Time"). And as far as I know, there is no standard approach to valuation and hedging non-attainable payoffs (for example). Note also that there are many reasons for payoff to be non-attainable and it's hard to define them all precisely.

I assume that by "this machinery breaks down" you mean when it breaks down as theory, but not as a practical tool.

I would say that the exact point where risk neutral pricing approach fails is when the payoff is no more attainable. There exist a precise mathematical characterization for attainable payoffs (see the book of Hans Föllmer, Alexander Schied, "Stochastic Finance: An Introduction in Discrete Time"). And as far as I know, there is no standard approach to valuation and hedging non-attainable payoffs (for example). Note also that there are many reasons for payoff to be non-attainable and it's hard to define them all precisely.