I am currently working on a project for which I need the implied volatility surfaces, to estimate the value of plain-vanilla European options with different strikes (cannot be observed directly in the market). I collected American option chain data from a data source and calculated the implied volatilities with the Black-Scholes formulas. As you know, the price of an European option is not always equal to the price of an American option. Normally, in a positive interest rate environment and for non-dividend paying stocks, the price of an American call is equal to an European call. However, my data set do contain a lot of options which has dividend paying security as underlying and we currently live in a negative interest rate environment.
To solve this problem, I picked only out-of-the-money options to calculate the implied volatilities. I made this choice because out-of-the-money options will never be exercised (in this case the American option converts to an "European" option). When I did a random check for three option chains, I got really nice volatility smiles (typical school examples).
As mentioned, I need to make estimate for plain-vanilla options with different strike prices (most of them cannot be directly observed in the market). So I made the following simple regression, to interpolate between observed strike prices:
Which produces a lot of graphs as follows:
The regression matches the observed implied volatilities, I got a really high r2 score and I have no problem to interpolate the implied volatilities. However, sometimes I get a graph as follows (10-15% of the cases) :
As you can see, the regression is not able to interpolate due to the fact that there is a "broken" volatility surface. The observed implied volatility for a put option with strike 17 is equal to +/- 0.42 and the observed implied volatility for a call option with strike 19 is equal to +/- 0.52 (the share price is around 18.75). The time to maturity is 9 months. I already recalculated a couple of times, so, I am 98% sure I did not make a mistake. Furthermore, one extra detail is the following graph, in which you see that the r2 improves over time (last day is a data error):
My exact question is as follows: Do some has also faced a "broken" volatility surface? And what is it caused by? And how did you solved it?