I am currently working on a large data set (approx 80 million data points over 10 years). I would like another set of data that has one currency in common. Eg, I have EUR/USD and would like USD/CNY or EUR/AUD etc. Doesn't need to be over the full 10 years, 1 year would be more than sufficient.
I found a few places online that sell this data, but the cheapest I could find was approx 60 Euros which is a lot for a student.
Is there anywhere someone can get data like this cheaper?
Dukascopy offers historical tick data. Through their historical data website you can download what you want, but registration is required, and lots of manual clicking.
However if you are comfortable with scripting, you can directly download the tick data yourself. The URL pattern is http://www.dukascopy.com/datafeed/{currency}/{year}/{month}/{day}/{hour}h_ticks.bi5, so for example http://www.dukascopy.com/datafeed/AUDCAD/2017/00/01/23h_ticks.bi5 gets you ticks for AUDCAD from 1 January 2017, 23:00-23:59:59.999 UTC.
Note that the months are zero-based (I don't know why), so Jan-Dec is 00-11 (two digits). Every hour is present as a file, even if the market is closed.
The file format is an LZMA-compressed binary packed file, so you will need to decompress it to CSV or other format according to your need. Each tick is 20 bytes, five four-byte fields:
In python, I use a struct.unpack('>LLLff', bytes) to extract the fields.
The data format is very space efficient for transfer and storage, but not efficient for processing, so I convert it to a time series for my storage.
I based my code on this guide, and there's plenty of GitHub repositories to help.
Take a look as well at TickStory Lite.
IIRC, you should be able to pull tick level FX transactions back to 2003 for free. I used this several months ago, so not sure if anything has changed.
Take a look also at HistData.Com, they have both 1 minute data (which I use) as well as Tick Data. It is free or very inexpensive depending on the method of downloading you choose.
