Recently,I am trying to control a marco traffic system with the general value iteration adaptive dynamic programming algorithm.However,the results do not reach my expectation.

Here is the pseudo code:

enter image description here

I constructed three simple neural network to approximate the action,model and critic function like this enter image description here

action network is used to approximate the control model network is used to approximate the traffic system critic network is used to approximate the value function

However, the value function do not converge when training the critic network and the value function is supposed to be non-increasing enter image description here

So can anyone who are skilled at approximate dynamic programming figure out the possible reason ? Thanks a lot.

  • 1
    $\begingroup$ I'm voting to close this question as off-topic because this question belongs on another site (e.g. StackOverflow or see the sites in this answer) in the Stack Exchange network. $\endgroup$ – skoestlmeier Jan 25 '19 at 8:41

Browse other questions tagged or ask your own question.