# Value function does not converge when applying the general value iteration adaptive dynamic programming [closed]

Recently,I am trying to control a marco traffic system with the general value iteration adaptive dynamic programming algorithm.However,the results do not reach my expectation.

Here is the pseudo code:

I constructed three simple neural network to approximate the action,model and critic function like this

action network is used to approximate the control model network is used to approximate the traffic system critic network is used to approximate the value function

However, the value function do not converge when training the critic network and the value function is supposed to be non-increasing

So can anyone who are skilled at approximate dynamic programming figure out the possible reason ? Thanks a lot.

• I'm voting to close this question as off-topic because this question belongs on another site (e.g. StackOverflow or see the sites in this answer) in the Stack Exchange network. – skoestlmeier Jan 25 '19 at 8:41