One way of thinking about the problem is with a statistical factor model. Consider the two cases:
- You have more assets than time points
In this case if you accept enough factors, then there is no idiosyncratic risk. But there will be idiosyncratic risk if you restrict the number of factors.
- You have more time points than assets
In this case even if you accept as many factors as there are assets, you will in general have idiosyncratic risk as well. In order for your hypothesis to be true (over the time period of the data), there would need to be no idiosyncratic risk visible in this case.
I don't see a good way of testing the hypothesis, but it doesn't seem like a realistic possibility to me. I think a more reasonable hypothesis is that the distribution of idiosyncratic risk is [smaller, more skewed, ...] in time frame X relative to time frame Y.
Such hypotheses can be tested reasonably by estimating factor models in the two periods. Use the same number of factors in each period and then plot the densities of idiosyncratic risks. And probably try a few different choices of the number of factors.
You could bootstrap to get a sense of how variable the idiosyncratic distributions are.