AFAIK typically banks stress test it trading portfolio by assuming stressed value of risk factors or by considering times series corresponding to some historical stress period e.g. 2008-2009 financial crisis period.
So my question is, is there any application of advanced machine learning techniques used by bank in their stress testing application?
U.S. centric answer.
Banks/financial institutions are given standard stress scenarios by regulators for CCAR and DFAST (Dodd-Frank Act Stress Testing).
It's a good bet that many institutions in various jurisdictions will also be given stress scenarios for climate risk in the next few years.
These are the same scenarios for many institutions.
Also many institutions indeed run stress tests based on some historical events, for example, Lehman bankruptcy in 2008, or Russian sovereign default in 1998, or the COVID-19 lockdown in March 2020.
However many institutions also do a "reverse" analysis. They search for plausible stress scenarios that would hurt them the most.
It is not necessary to use AI/ML for this. For example, if the institution has to calculate VaR, and chooses to use Monte Carlo to generate scenarios for VaR, then you get "for free" the MC scenarios in the tail of the distribution that would hurt you. You can analyse these scenarios and think of ways to hedge them.
However the scenarios from MC are not exhaustive. It is possible to have scenarios that don't look like anything like recent history or anything from a Monte Carlo simulation that looks like recent history.
One place I know to be doing interesting work on generating stress scenarios that would hurt a given portfolio / balance sheet (sometimes using AI/ML) is Straterix / Alla Gil. I think some of her recent GARP posts are relevant to your question:
Risk Management in a Sea of Unknown Unknowns: The Complex Quest for Resilience
How to Mine Synthetic Data: Pros and Cons of a Shiny New Tool for Risk Managers
Two applications of machine learing (related) techniques are in valuation using neural nets of complex products, e.g. "Deep xVA solver -- A neural network based counterparty credit risk management framework" or using Adjoint Algorithmic Differentiation, e.g. "Fast Greeks by algorithmic differentiation".
Both of these techniques aim to speed up the computation. I'm not aware of using ML to come up with stress scenarios or stressors.
