I am reading Prado's new book, Machine Learning for Asset Managers.
In the page1 of his book, there is this sentence.
To a greater extent than other mathematical disciplines, statistics is a product of its time. If Francis Galton, Karl Pearson, Ronald Fisher, and Jerzy Neyman had had access to computers, they may have created an entirely different field. Classical statistics relies on simplistic assumptions (linearity, independence), in-sample analysis, analytical solutions, and asymptotic properties partly because its founders had access to limited computing power.
I guess the independence in the quote originated from IID (independent, identical distribution) assumption.
But I'm not quite sure where the linearity of classical statistics comes from. The one place I could think of is about the linear regression. But it is pretty easy to extend linear regression to higher-order using the same Gauss-Markov theorem with the basis function approach.