Mr. Soros in his books talked about principles which are not used by today's financial mathematics — namely reflexivity of all actions on the market. Simply it can be given by following: expectations of traders are based on the news and historical prices. They trade based on their expectations and hence influence prices which then influence expectations on the next "turn". I have never met an implementation of these ideas in the scientific framework. Have you? If yes, please give me a reference.
Keynes introduced this idea in the notion of a Keynesian Beauty contest: http://en.wikipedia.org/wiki/Keynesian_beauty_contest
Anyone who uses a rolling window regression where the parameters and/or parameter estimates are re-fitted periodically are implicitly accounting for this reflexivity (i.e. the market's changing behavior as agents respond and adapt to each others actions)
With respect to a scientific-framework, agent-based modeling is something that fits the bill and also game theory: http://en.wikipedia.org/wiki/Agent-based_model
I haven't posted on SE much, so hope you will not mind if I also answer some comments here.
The best paper I have seen articulating what Soros does is by Flavia Cymbalista, a psychologist writing in the tradition of Eugen Gendlin. It's a very brainy paper, and Soros himself has spoken favourably about it.
And her website is here: http://marketfocusing.com/
I have written a bit more myself here: https://www.quora.com/How-did-George-Soros-and-Jim-Rogers-fund-the-Quantum-Fund-earn-33-annualized-returns-for-30+-years
It's a mistake in my view to think that what he discusses is something that can be turned into a mathematical model. In markets one is dealing with conditions of radical/Knightian uncertainty, where in truth assigning a distribution of probabilities is rather fraught since to do so requires understanding the structure of the problem, something that quite often isn't the case. One doesn't know what one doesn't know. Ludwig Lachmann, GLS Shackle, Israel Kirzner, James Buchanan, and Viktor Vanberg as well as Frank Knight have written about this problem. The post-Keynesians too.
So he isn't talking about mean reversion or the fact that markets trend (or that there is momentum). These things are true, and they naturally emerge from his world-view, but that isn't what he was talking about. If that's what he meant he would have said so, and I am pretty sure it isn't.
By the way, it's also not true to say that Soros's approach is bottom-up. It's not top down either. But you could describe it as based on a gestalt that involves a to-ing and froing between different levels. Analysis means breaking something down into pieces to understand it better. (See Stanford dictionary of philosophy). You can't get to a gestalt merely from analysis, because the whole is more than the sum of the parts.
Sornette's work is interesting, but I never met a guy who made money from it.
Non-linear dynamics - either you mean this literally or metaphorically. If literally this raises some problems, because how can anyone possibly have insight using a non-quantitative approach. Soros isn't in essence some kind of calculating savant. If metaphorically, you can call it what you like, but I don't know the metaphor helps you to understand the problem better.
It's my belief that one isn't going to get anywhere as regards deep insight into economic and financial trends with stochastic processes that are ergodic because economic life isn't such. And non-ergodic processes present difficulties of their own.
Indeed, I hadn't seen this before now, but Soros specifically objects to ergodicity: https://rwer.wordpress.com/2012/03/28/the-ergodic-axiom-davidson-versus-stiglitz-and-lucas/
In my view, the essence of reflexivity relates to how humans perceive change. That requires an understanding of psychology, too, and this cannot be separated from what the neuroeconomics tells us about the primacy of affect. (It's a lesson also from the Gestalt school, which tells us that affect does not just shape the hedonic value placed on something but also the perception of that thing itself).
Thus, one will get much further with taking affect seriously than with pretending it doesn't exist.
I write informed by some of the academic literature, but this perspective is based rather more on experience than on academics. Hope it's been interesting to a few.
This paper by Filimonov and Sornette might be interesting or useful to you. I've only read about the first third, but I thought the model was pretty cool. The model for price changes is a self-exciting Poisson process: there is an exogenous factor modeling the "real" price changes, and then there is a feedback mechanism where the overall arrival rate is an exponentially weighted average of recent arrivals. The paper is available here. If you go through it all, would welcome your thoughts.
I havent seen much implementing reflexivity in practice in a comprehensive way. Of course there are small steps in that direction, but no general framework yet.
To comment on answers above: incorporating updates of information is not reflexivity; reflexivity is about taking into account the effect of your own actions while chosing them (jointly with those of other market participants, sure), that is ex ante, not ex post. Also momentum and mean reversion need not have anything to do with reflexivity.
Even the modeling of market impact in algo trading, which should be the application par excellance of reflexivity, is still done in a kind of traditional "blind" way: ex post statistics of impact are taken and then the forecasting model is used to optimize strategies... Reflexivity in algo trading would be best applied to avoiding stop loss crashes, which is currently not yet done afaik. But sure market impact modeling goes a little bit towards reflexivity in a short term sense.
Stochastic control is one area where some reflexivity is indeed present by definition. And sure game theory, but that's not used much in practice.
Something might also be found in the bubble dynamics literature, but I'm not familiar with it. Agent modeling is surely interesting per se, but in its current state is far from even scratching the surface of that topic, and the main research stream doesnt touch reflexivity at all.
More on control theory (w/o stochasticity) which is already pertinent (see also dynamic programming). It deals with the response dynamics of a system to different inputs and how to best plan strategies. E.g. it is used in engineering to study and prevent oscillations. Oscillations are the typical problem arising when no control techniques are used (as seen in economic cycles, bubbles etc...): a target is aimed at, a steering parameter is directed towards that but the actual response of the system is not linear so the steering needs to be corrected, but then the correction itself must be called back with time yet still the dynamics leads away in the other direction, so one has to steer again in the original direction, and so on in an endless cycle... E.g. in algo trading of course your target is the quoted price, but you must adjust it to take into account market impact, which however is not given but depends itself on the your target and strategy generating some circularity. Control theory iirc is applied in pricing of americans for the exercise strategy, and all similar problems, although it's overkill since there's much more to it. Here some examples in finance. (Definitely people at the FED should take a course in control theory, before they start messing that way with the world economy. What will happen with inflation is a classic case of control myopia where they're oversteering without noticing it before it's too late. Here some examples in economics.)
The answers above are good, but I suspect they will be unsatisfactory if you are looking for implementations that are successful in practice. The sort of bottom-up analysis championed by Soros is very difficult to carry out in a rigorous, quantitative manner. This is true very generally, not just in finance. There are certainly models of financial markets that explicitly consider beliefs and actions of individual market participants (see game theory), but these always rely on extreme oversimplifications of actual human cognition and behavior.
On the other hand, many seem to be able to develop a good intuitive sense for what sort of beliefs and biases are driving the behavior of financial market participants and exploit this knowledge. Soros is often counted among their ranks. Although this approach may be sophisticated, it is usually not considered quantitative. Soros often claims that he looks for a sort of "turning point" where market participants realize the flaws in their beliefs. This is in contrast with most quantitative models that implicitly assume that whatever structure is observed in the data will persist long enough for a profit to be extracted.
Sure it is used in non-linear dynamics. What is not used is that this non-linearity is caused by huge operators who cause the deviation from Normal Law delaying the reversion to the mean. This is just common sense and mathematically logics when you go straight to Normal Law premisces: all samples must have about the same weight and being independant. Clearly these premisces are hugely violated in Stock Market.