Recently,I am trying to control a marco traffic system with the general value iteration adaptive dynamic programming algorithm.However,the results do not reach my expectation.

Here is the pseudo code:

enter image description here

I constructed three simple neural network to approximate the action,model and critic function like this enter image description here

action network is used to approximate the control model network is used to approximate the traffic system critic network is used to approximate the value function

However, the value function do not converge when training the critic network and the value function is supposed to be non-increasing enter image description here

So can anyone who are skilled at approximate dynamic programming figure out the possible reason ? Thanks a lot.


closed as off-topic by skoestlmeier, Bob Jansen Jan 25 at 14:53

  • This question does not appear to be about quantitative finance within the scope defined in the help center.
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ I'm voting to close this question as off-topic because this question belongs on another site (e.g. StackOverflow or see the sites in this answer) in the Stack Exchange network. $\endgroup$ – skoestlmeier Jan 25 at 8:41

Browse other questions tagged or ask your own question.