# How to deal with unreliable data sources?

I'm building a little system for myself that helps me make trade decisions, essentially downloading daily OHLC prices for a few hundred stocks once a day, running various sizes of crossing averages on them and sending me a push notification when something interesting pops up.

Right now, I'm using stock data from Yahoo finance, mostly because their API is really easy to use (no need to parse HTML tables and such).

This seems to work quite nicely, except for one thing:

For a few stocks, the price data contains really weird entries. Take ISIN IE00B8FHGS14 for example. On most days, the prices returned by Yahoo's API are reasonable:

Apr 21, 2020    47.54       47.54       46.35       46.64       46.64


But on some days, the prices are wildly out of any reasonable range for that stock:

Feb 17, 2020    4,204.00    4,219.00    4,198.36    4,219.00    4,219.00


While this doesn't throw off my trade signal calculations too much (they're based on averaged prices after all), it makes actually evaluating stocks (i.e. "Does it make sense to trade this stock using crossing averages as the signal?") essentially impossible.

Is there some reason the prices are sometimes way out of range? How do you guys deal with unreliable data? Am I completely on the wrong track crawling Yahoo and should I use something else?

• There have been many complaints here about the quality of Yahoo Finance data (some even saying the API no longer works after May 2017). In my opinion perhaps we should consider using something else... Apr 23, 2020 at 9:31
• I have never seen a data source that never had errors in data. It's something you have to get used to. If you know what errors to expect, you should be able to write code that alerts you when you receive bad data and correct it before adding it to a database. Apr 23, 2020 at 13:03