# What is currently predictable in the stock and bond markets and what is not

Disclaimer: I have some knowledge of statistics, machine learning and probability theory, but next to zero knowledge of finance (I had to look up Wikipedia to refresh my knowledge of the difference between a bond and a stock), so please don't shoot the newbie :)

I was reading this question on Cross Validated, and I noted that some users (included the original poster) noted that the task of predicting the stock market was impossible. To be pedantic, I think they were actually referring to the simpler (?) task of predicting the future values of a specific stock price based on the stock price past values, not predicting the overall trend of the whole stock market, but you get my point. This is something which I've seen often in pop-finance books, and which is usually explained intuitively this way: if there was a model which could be used to predict reliably the future price of one or more stock, everyone would use it, this would affect the future stock exchanges and prices, changing the data generating process (the stochastic process corresponding to the stock price time series), and thus the model wouldn't work anymore. Another argument which is sometimes given is that market crises are never predicted (the subprime mortgage crisis, dot-com bubble, etc.).

However, there are companies, university degrees, research centers, etc. which work in the field of quantitative finance, so, even if the endeavor of predicting the stock and bond markets is understandably hard, there must be some degree of success. What is currently predictable with some margin of accuracy, and what is not? Concerning the level of the answer, I can follow you if you talk about expectation, stochastic processes, martingales, Monte Carlo and Markov Chain Monte Carlo, neural networks, etc.. However I've only heard about stuff such as the Black-Scholes equation, without knowing the actual details. I think I could understand if you explained me the concept, but I cannot say for sure.

• This is really broad, an answer would probably be more essay like and be opinion based. Therefore I close it. Dec 21 '16 at 10:43
• Quantitative Finance is also economics. Economists are famous for never reaching a consensus and long term predictions are famously unreliable. Also, you write 'I would like to have some explanation of the "ideas that work".' Which ideas work is subject to debate. We could have a discussion on the predictability of volatility but I don't believe that is what you want. Dec 21 '16 at 11:46
• I don't find the question too broad. There is some consensus: 1) Volatility seems to be forecastable. 2) Returns are not forecastable. 3) Signs (up or down) should theoretically be predictable, i.e. Christoffersen & Diebold (2006, ssc.upenn.edu/~fdiebold/papers/paper47/…) Dec 21 '16 at 15:15
• I am torn... on the one hand it is not really a good fit for our SE (so I agree with @BobJansen), yet I could imagine that some interesting answers show up... suggested a reopening... we'll see what happens. Dec 21 '16 at 17:26
• Ok. Worst case scenario is that the question will be closed again. A very acceptable risk. Let's hope some interesting answers are given! Dec 21 '16 at 17:28

I'll add my two cents even though there have been a couple of answers. In terms of the actual price of stock XYZ tomorrow, it's not likely to be predicable. The price is an combination of both market move and individual stock move, and all come down to supply and demand. There are market research that gives all kinds of target prices, but are mostly just used to advertise to clients.

But just like you can forecast weather based on both historical information and surrounding atmosphere, you can generate some subjective view in terms of the stock price changes (going up or down) based on your experience and observation of the current market, just like what day traders do. In this case, other people may have gotten similar view and started actions before you do, so price would have changed already. That's called factor-in or price-in.

In some scenarios, certain people can predict the stock price because of information asymmetry, e.g. knowing ahead of time the earning announcement details. Some other people may piggy-back one way or another (could even be their alphas), while most of the uninformed traders just trade randomly and get exploited. It's essentially a zero-sum game. I've heard that most of the liquidity in the US stock market comes from institutional trading, but still, some institutions are better than others.

Down to the micro-structure level, stock-price is predictable if you look at tick-by-tick. The market orders are highly autocorrelated. This is likely because of order execution. This regime is not accessible to average people though.

Not being able to predict stock price doesn't mean you can't make money by investment. The fundamental theory of investment is risk-reward trade-off. You can form/sort your portfolio based on some criteria and hope for growth better than the market portfolio (based on SP500 for example). The relative value here is probably more predictable than individual stock prices.

The quant finance people don't necessarily care about the price levels, as a lot of the theories are based risk-neutral assumptions.

In summary, I don't think that the exact level of stock/bond price is predicable, but it shouldn't affect a lot of the quant finance work.

• This is interesting and I think similar to what @JoshK was saying. A clarification: if I'm not mistaken, a risk-neutral party is one which has no preference between investments with different risks but the same expected value. Why this implies that quant people don't care about price levels? Do you mean they only care about the long term trend, i.e., the mean function of the stochastic processes, but not the short term oscillations, i.e., the covariance function of the stochastic processes? Dec 23 '16 at 10:08
• Also, I would like to elaborate on your analogy with weather forecast. In meteorology, you can either forecast based only on history of measured data, and correspondingly obtain very inaccurate forecasts. Or you can create a model based on measured data AND partial differential equations modeling atmospheric physics, and obtain much more accurate forecasts. Is there something similar in quant finance? I.e., are methods which combine fundamental analysis and technical analysis more accurate , in the short or long term? Or what I just said makes no sense? Dec 23 '16 at 10:31
• @DeltaIV Definitely make sense. The model risk of of weather forecast is low, i.e. not much damage if it's inaccurate. The model risk in finance is high. "not necessarily care about the price levels" has two-fold meanings: one is similar to the relative value investment. By long-short assets, you are essentially trying to pick winners/losers in a bunch of assets. Other other fold is the risk-neutral pricing. Banks make prices and hedge their position. The cool thing about option is that dynamic hedging removes the risk of underlying move, and you can make money from the fees/spreads/etc. Dec 23 '16 at 19:28

The renowned CXO Advisory Group has a section "What Works Best?".

Here some general information is given and many links to their research articles which e.g. summarize lots of current academic research (although most of the linked articles are behind a paywall the links to the original papers are normally provided).

The article closes with "In summary, strategic diversification and momentum and value strategies applied at the asset class level via low-fee funds (especially with momentum and value in combination) may be among the best approaches [...]"

In any case I think this is a valuable place to start. I myself have been a subscriber to their service for many years and it has helped me to keep some perspective.

• Very interesting, thanks! Some of the stuff is written in terms which I can understand, thus it's a great source. Dec 23 '16 at 8:44

I think you have a small misunderstanding in terms of what the folks with all of the various incarnations of quantitative degrees are doing. There are always people trying to punt on the direction of the stock market for sure - and there is always an appetite with investors to try to outperform the market.

What you are missing is that humanity is net long assets as a whole. That means that people have investments that need to be managed. What do you do with those investments? How to manage them? Lend them? Optimize them to suit certain objectives (think of pensions and insurers). So many of the quantitative people in finance work on these kinds of problems.

Many banks will have teams of quantitative people working to improve the process of lending their shares and properly collateralizing them. Just as one example.

I think a lot of times quantitative people are disappointed when they get to a finance job and realize that they are not going to be predicting where a stock is going but are instead trying to solve a problem like figuring out which broker is giving the best value for their execution services.

• Ah, this is brilliant (not that the other answers were not brilliant!): indeed, this is something I didn't think of (I did premise I know little about this stuff). However, I'not sure I understood you, because I know half the terms you used. Basically you said that people have investments (say, money in a bank deposit) that have to be managed ( if left to themselves, they will simply be eroded by inflation ). Thus, many banks hire quants to develop ways to lend this money in a way which minimises risk and increases profit, instead than doing stock prediction. Right? Dec 23 '16 at 8:35
• I have a doubt though: to make these net long assets grow in time, the fund manager (broker? Don't know the terminology) still needs to choose which bond/stock to sell/buy. Right? Thus some kind of prediction is still needed, maybe not at the individual stock level but at least on a aggregate level (groups/classes of stocks or bonds). Does this make sense to you? Dec 23 '16 at 8:42
• I'm just trying to say that a very small part of "quantitative finance" is about predicting where stocks will go tomorrow. Another example, I have a friend who works at a large asset manager. They have about twenty quantitative types in a group devoted to helping people manage real estate portfolios. They look at funding and vacancies, local trends, etc to try to best manage what their clients own. I have a friend in another group at a large bank that works in a quantitative group that helps their stock lending people come up with the best rates to offer people to use their collateral. Dec 23 '16 at 15:18
• ok thanks, I think I got the point. Definitely stock price forecast is not as important as other applications for quantitative finance. Dec 23 '16 at 21:00
• Exactly. It's a lot more of a grind than people think. Dec 25 '16 at 3:12

I think, the popular consensus is that, while in general the stock prices are unpredictable, there are certain market regimes it's possible. That's pretty much the premise of a well-written book by Lo and MacKinlay: http://press.princeton.edu/titles/6558.html which I recommend.

In terms of ML forecasting methods, it often translates to the situation, when the most difficult part is to figure out in what particular situations to use these forecasting methods, and not so much which particular type of a regression technique or a neural net is optimal.