# Howto Calculate An Error's Partial Derivative in ANN

This is a follow-on question from this post I made, "Multilayer Perceptron (Neural Network) for Time Series Prediction", a few months back.

I'm constructing a feed-forward artificial neural network, using resilient propagation training. At the moment, I'm trying to implement an individual neuron input's weight update algorithm. For the life of me, I can't seem to find a clear and straightforward answer on how to calculate the partial derivative of the error for a given weight. The only thing I can find on the web, is the fact that a neuron's weight update is a function of $\frac{dE}{dW}$ (cf. the original Paper [p. 2 & 3], or this one [p. 4]).

However none of these papers actually outlines how to calculate this.

I understand the concept of a partial derivative in a mathematical sense. And I assume that the current neuron input's weight change calculation is the operation at hand, while all other neuron input values are held constant.

So for each of these neurons below, I calculate each inputs' individual error by taking a total error ( -0.3963277746392987 ), that's been multiplied by that neuron input's weight (each :calculated-error is the sum of the individual inputs' error).

For both neurons, what would be the weight change for each input?

Here is my data structure:

:input-layer
({:calculated-error -1.0991814559154283,
:calculated-value 0.9908633780805893,
:inputs
({:error -0.07709937922001887,
:calculated 0.4377023624017325,
:key :avolume,
:value 2.25,
:weight 0.19453438328965889,
:bias 0}
{:error -0.19625185888745333,
:calculated 1.4855269156904067,
:key :bvolume,
:value 3.0,
:weight 0.4951756385634689,
:bias 0}
{:error -0.3072203938672436,
:calculated 1.0261589301119642,
:value 1.32379,
:weight 0.7751674586693994,
:bias 0}
{:error -0.36920086975057054,
:calculated 1.2332848282147972,
:key :bid,
:value 1.3239,
:weight 0.9315543683169403,
:bias 0}
{:error -0.14940895419014188,
:calculated 0.5036129016361643,
:key :time,
:value 1.335902400676,
:weight 0.37698330460468044,
:bias 0}),
:id "583c10bfdbd326ba525bda5d13a0a894b947ffc"},
...)

:output-layer
({:calculated-error -1.1139741279964241,
:calculated-value 0.9275622253607013,
:inputs
({:error -0.2016795955938916,
:calculated 0.48962608882549025,
:input-id "583c10bfdbd326ba525bda5d13a0a894b947ffb",
:weight 0.5088707087900713,
:bias 0}
{:error -0.15359996014735702,
:calculated 0.3095962076691644,
:input-id "583c10bfdbd326ba525bda5d13a0a894b947ffa",
:weight 0.38755790024342773,
:bias 0}
{:error -0.11659507401745359

:calculated 0.23938733624830652,
:input-id "583c10bfdbd326ba525bda5d13a0a894b947ff9",
:weight 0.2941885012312543,
:bias 0}
{:error -0.2784739949663631,
:calculated 0.6681581686752845,
:input-id "583c10bfdbd326ba525bda5d13a0a894b947ff8",
:weight 0.7026355778870271,
:bias 0}
{:error -0.36362550327135884,
:calculated 0.8430641676611533,
:input-id "583c10bfdbd326ba525bda5d13a0a894b947ff7",
:weight 0.9174868039523537,
:bias 0}),
:id "583c10bfdbd326ba525bda5d13a0a894b947ff6"})

-

It's a bit unclear to me what you're trying to do, and maybe a better place to ask your question is Stats.SE but I would encourage you to go and have a look at this online class on machine learning which provides an implementation of the backpropagation algorithm.

You can either register or hit preview and go to the NN:Learning chapter, but I would recommend you to register and do the programming exercise on NN. Maybe this is not exactly what you are trying to do, but I think you might find a basic example of how to compute the partial derivative of the error term.

-
Thanks for this. Andrew Ng is actually the guy whose videos I first started following, when I was getting into AI. But this course seems to emphasize using NNs for classification problems (discreet values). I want to compute a continuous value in a time series. But I will dig further into this series. Thanks again. –  Nutritioustim Oct 5 '12 at 3:25
There is another class specifically for NN if you look on the site. –  SRKX Oct 5 '12 at 8:50