09: Neural Networks - Learning

Previous Next Index

Neural network cost function

Types of classification problems with NNs

Cost function for neural networks
Woah there - lets take a second to try and understand this!First half
Second half


Summary of what's about to go down

The following section is, I think, the most complicated thing in the course, so I'm going to take a second to explain the general idea of what we're going to do;

Back propagation algorithm
                             

What is back propagation?
Analyzing the mathematics

Why do we do this?
Putting it all together to get the partial derivatives!
Back propagation intuition
Forward propagation with pictures!




Back propagation

Implementation notes - unrolling parameters (matrices)


Example


Gradient checking
Random initialization
Putting it all together
for i = 1:m {
     Forward propagation on (xiyi) --> get activation (a) terms
     Back propagation on (xiyi) --> get delta (δ) terms
     Compute Δ := Δl + δl+1(al)T
}

With this done compute the partial derivative terms