For reference, I am talking on behalf of a small group of math/stats graduate students as well as software engineers (we are 6 total), we know each other for years and decided to make a small (private) fund with our own money to invest in crypto - we all have jobs, and this will be a side gig for everyone.
We are at the point that we already wrote some code as well as setup basic interfaces to plug to a single/multiple exchanges, as well as basic data pipelines, however we are still arguing about whether we should be ingesting book data (either top bid/ask or an arbitrary book depth) , trade data (trades executed at price x, size x) or just stick with bar data (i.e 1minute) .
The S.E say "we just need more power and we can do book data for any number of instruments", and the math/stats guys say "unevenly spaced time series data is a pain to model, we should just stick with bar data which is evenly spaced".
The exchanges we looked at are Coinbase and Binance and they offer very different feeds too.
Binance for example only sends websocket updates every 1 second, so regardless of book_depth at most you get 60 points of data per minute per symbol (this assuming that the particular symbol is very liquid, otherwise you get even less points - which also makes it unevenly), on the other end Coinbase sends updates every time a change happens in either price/size and with i.e book depth = 10 (top 10 bid/ask levels) for BTC/USD we registered 2k+ data points per minute - There's a massive difference on the type of hardware requirements to process this data, both live and backtest phase.
In the end the options are:
PROS - Easier to model; Cheap to acquire historical data (if not free); Needs the least infrastructure;
CONS - No access to the spread; Can't act on any changes that occur within that time-frame (i.e if the price drops 50$ in a minute, can only act on it after it already happened);
PROS - Always know the spread - can both attempt to model it as well as act on changes in price very fast.; Detailed representation of the specific market; Access to different levels of book depth (this is good to model if to place market/limit orders);
CONS - Needs a ton of infrastructure to keep up with multiple feeds/decision pipelines; Expensive to buy historical data; Lot of noise with all the algos already working on the exchanges; Unevenly spaced;
PROS - Halfway between book and bar in terms of infrastructure needed + detailed view of the market; Cheap to acquire historical data (if not free);
CONS - Noisy, and unevenly spaced; No spread;
I am curious to hear your thoughts, in particular from people who had to make the same decision in the past.
Who's right? Which solution could eventually pay off the most?