So I have the following short-rate model $$dX_t = a_1X_tdt + \sigma_1dW_t$$ $$dY_t = a_2Y_tdt + \sigma_2dB_t$$ $$r_t = X_t + Y_t + f(t)$$ with $X_0 = Y_0 = 0$ where $W$ and $B$ are Brownian motions with correlation $\rho$ and $f(t)$ is some deterministic function chosen to match the current term structure.
I want to calibrate this model by matching the observed caplet prices to a parameter vector $(a_1,a_2,\sigma_1,\sigma_2,\rho)$. My approach is as follows. I first generate a random parameter vector and I compute the corresponding caplet prices for different expiries, strikes and tenor dates. Then I train a neural network (in Matlab, if that matters) that takes this vector of prices as input and outputs the corresponding parameter vector. I expect the function from the price vector to the parameter vector to be smooth so a single-layer neural network with a sufficient number of neurons should be able to approximate this function.
But this method does not seem to work. The fitted values of $a_1$ and $a_2$ match their real values (the ones I gave to the neural network) reasonably well but $\sigma_1,\sigma_2,\rho$ are all over the place. If I fix $\sigma_2$ and $\rho$, then the fitted values of $a_1,a_2$ and $\sigma_1$ are quite good. On the other hand if I fix $a_1$ and $a_2$, then the fitted values of $\rho$ are good whereas the fitted values of $\sigma_1$ and $\sigma_2$ are just terrible. If I fix $a_1,a_2$ and $\rho$, then the fitted values of $\sigma_1$ and $\sigma_2$ are almost perfect. Is there some kind of identifiability issue inherent in this model?
I generate about $10000$ samples. My input vector is 280-dimensional with expiry ranging from 1 week to 10 years, strike ranging from $-2\%$ to $1\%$ and tenor ranging from 3 months to 3 years. I went as high as 100 neurons with no visible improvement. I tried Matlab's different neural network optimization algorithms but again no improvement. It deteriorated in fact. I would appreciate it if someone could give some feedback on this approach.