On Sun, Oct 24, 2010 at 8:53 PM, Troy Lee <troy.lee2008@gmail.com> wrote:
The theoretical formalism described in this paper seems to be well suited to the description of many different variations of back-propagation.From a historical point of view, back-propagation had been used in the field of optimal control long before its application to connectionist systems has been porposed.The central problem that back-propagation solves is the evaluation of the influence of a parameter on a function whose computation involves several elementary steps.This paper presents a mathematical framework for studying back-propagation based on the Lagrangian formalism. In this framework, inspired by optimal control theory, back-propagation is formulated as an optimization problem with non-linear constraints.The Lagrange function is the sum of an output objective function, which is usually a squared sum of the difference between the actual output and the desired output, and a constraint term which describes the network dynamics.This approach suggests many natural extensions to the basic algorithm.Other easily described variations involve either additional terms in the error function, additional constraints on the set of solutions, or transformations of the parameter space.
No comments:
Post a Comment