Almeida–Pineda recurrent backpropagation explained

Almeida–Pineda recurrent backpropagation is an extension to the backpropagation algorithm that is applicable to recurrent neural networks. It is a type of supervised learning. It was described somewhat cryptically in Richard Feynman's senior thesis, and rediscovered independently in the context of artificial neural networks by both Fernando Pineda and Luis B. Almeida.^{[1] [2] [3]}

A recurrent neural network for this algorithm consists of some input units, some output units and eventually some hidden units.

For a given set of (input, target) states, the network is trained to settle into a stable activation state with the output units in the target state, based on a given input state clamped on the input units.

Notes and References

1. Feynman. Richard P.. Forces in Molecules. Physical Review. American Physical Society. August 1939. 56. 4. 340–3. 10.1103/PhysRev.56.340. 1939PhRv...56..340F.
2. Pineda. Fernando. Generalization of Back-Propagation to Recurrent Neural Networks. Physical Review Letters. 19. 59. 9 November 1987. 2229–32. 10.1103/PhysRevLett.59.2229. 10035458. 1987PhRvL..59.2229P.
3. Almeida. Luis B.. A learning rule for asynchronous perceptrons with feedback in a combinatorial environment. June 1987. IEEE First International Conference on Neural Networks. San Diego, CA, USA. 608–18.