Say I want to do arbitrage between Exchange A and Exchange B on USD/AAPL. This requires that I hold equal parts USD and AAPL. I don't want exposure to the movement in AAPL. How do I hedge my AAPL inventory so that I remain market neutral?
Too long for a comment - so I add this here as an answer. Not sure what delta hedging arbitrage is but I think you define delta as a difference in price?
If you ever were to see arbitrage (not talking about a price difference of 500 vs 550), it will be extremely short lived (so no need to worry about inventory). There are market makers in the stock exchanges that make sure that buy and sell orders get filled quickly (at market prices). They ensure liquidity and make money on the spread, the difference between ask and bid. This may not be true in pre and after market hours but even there you will either have voluntary participation of a market maker or ECN or other big players (hedge funds, HFT firms etc) who constantly monitor prices.
If you hold something and have to wait for the price to converge, this is not arbitrage. Arbitrage is defined as the possibility of a risk-free profit after transaction costs (risk free implies instantaneous).
For example, if you have dual listings, this is not arbitrage. Many people call it like that, but it does not satisfy the definition of arbitrage. The most famous example in my opinion being Royal Dutch Shell in the early 1980s, where Royal Dutch was trading at a discount of approximately 30% relative to Shell. Although these dual listed companies (Royal Dutch Shell) function as a single operating business, and as such shares represent claims on the very same underlying cash flows, divergence can be substantial and convergence can take many years. If you do not want exposure and hedge this, you have no way to benefit from price moves (convergence).