When back testing an algorithm that relies upon short selling certain stocks, how to limit the short selling so that the back-test results still remain reliable? What kind of controls are generally put on such algorithms during actual trading and how to simulate such controls while backtesting?
You should make your borrow cost sufficient to dissuade unlimited short selling. In practice, each short would require you to borrow shares from your broker. This is usually handled when computing transaction cost. You should account for this in your trading algorithm or in the factor model itself. A simple method would make shorts some N% more expensive than longs. In practice, this could kill your projected returns. You could probably estimate this factor by looking at historical short interest on the stock. If everyone is trying to short it, the cost to borrow is going to be high.