bp-mlp-5

---

Interpretation

$$\begin{array}{rlrl}{\mathbf{\delta }}^L& ={\nabla }_a\mathcal{L}\odot {\sigma }^{\prime }({\mathbf{z}}^L)& & \text{(BP1)}\\ {\mathbf{\delta }}^l& =(({\mathbf{W}}^{l+1}{)}^T{\mathbf{\delta }}^{l+1})\odot {\sigma }^{\prime }({\mathbf{z}}^l)& & \text{(BP2)}\\ \frac{\partial \mathcal{L}}{\partial b_j^l}& ={\delta }_j^l& & \text{(BP3)}\\ \frac{\partial \mathcal{L}}{\partial w_{jk}^l}& =a_k^{l-1}{\delta }_j^l& & \text{(BP4)}\end{array}$$