I’m working in the area of Data Mining and have come up with the following idea for my Masters project.The text may not be the best structured but it’s a working draft to give you a quick idea.
- Can a combination of (Weather Data + Agricultural Data + Social Media (twitter, etc) data + other relevant data) be used to aid an investor to buy futures of product / commodity ‘X’
- I plan to focus on testing the hypothesis on ‘Soy Bean Futures’. The core idea is to test the approach, even if I fail, its fine. My method / approach must be correct.
- potential target audience could be investor, govt agencies or agricultural industries
- Focus on Soy Bean Futures in USA (Worlds largest Soy producer) to narrow down my problem scope
- more specifically on State of Illinois (leads Soy production in US) to zoom in even further
- Understand how the model for pricing of Futures works
- Find Historical trading data on Soy Futures in Illinois [from Quandl?] I still don’t know how I will match Soy Future trading data to where it was produced so that’s an issue, I think
- Weather [temp, humidity, sea pressure, etc] & Agricultural [yields, farm sizes etc] data is easy to get & analyze
- Do some number crunching / data mining to test “IF Weather in Illinois affects Soy yields/ production which is turn affects Soy Futures prices” ; I still need to refine the technique but its a rough idea
- what do you think about the whole idea? totally nuts? not realistic? I need to be Math God to figure this out?
- if you think, that this is even remotely feasible, what are my must do’s, must NOT do’s?
- is my technique fully flawed? what am I missing? under-estimating? How can I improve my technique?