I strongly recommend not assesing risk using the risk neutral measure. Doesn't this already sound like a contradiction (risk and risk-neutral)?
The risk neutral measure is there to derive prices (for derivatives e.g.) that fit to the prices of related contracts and traded assets. With "fit" I mean not allowing for arbitrage. For example if I calculate the forward price of a stock then I implicitely use the risk neutral probability measure and avoid arbitragy of trading the underlying spot.
The drift/expected value is related to interest rates and dividends due to arbitrage considerations - neither due to any risk considerations nor due to any idea where the spot price of the underlying could really be in the future.
If you call the other measure the physical measure then this is the one that should be used to measure risk. The problem is that it is by far not unique. For example if you measure risk by volatility (just as a starting point) then it is known that many volatility estimators exist. You can look at different observation periods, you can use weighted methods or e.g. GARCH.
Very often the expected value/drift of the asset whose risk you want to measure is assumed to be zero. For short periods of time this is very reasonable. Coming up with an expected value different from zero you risk to mix up your "trading idea" with your risk measure - which is in my mind not a good thing to do.