Let's say I am hedging an exotic instrument $E$ with $N$ liquid instruments $L_i$, each of which has an associated hedging ratio $R_i$ and a bid-ask spread $\delta_i$ (per dollar of notional). What would you recommend as a cost function to balance the completeness of the hedge and minimize the hedging cost?
I would assign the cost of incompleteness as the 90th percentile of N-period losses expected on the mis-hedged portfolio (where N is perhaps 5 trading days -- enough for a trader to get hit by a bus and someone else to catch up on his book). This is nicely compatible with VaR computations, corrects for the fact that expected cost of a mis-hedge is usually zero, and doesn't involve any tricky utility function theory.
You sometimes see more precise measurements made. For example some papers in the 1990s calculated the exact optimal hedging strategy for European options given a particular bid-offer spread on the underlying and (I seem to recall) utility function assumptions.
If you like utility functions a lot, clearly you can assign one to the variance (or other metric) of P&L arising from mis-hedging, and use the utility function to turn that directly into a cost. You can approximate the "right" function parameters by, say, looking at your firm's recent returns and Sharpe ratio.