Wolfe duality explained

In mathematical optimization, Wolfe duality, named after Philip Wolfe, is type of dual problem in which the objective function and constraints are all differentiable functions. Using this concept a lower bound for a minimization problem can be found because of the weak duality principle.^[1]

Mathematical formulation

For a minimization problem with inequality constraints,

\begin{align} &\underset{x}{\operatorname{minimize}}& & f(x) \\ &\operatorname{subject to} & &g _{i(x)} \leq 0 , i = 1 , ... ,m \end{align}

the Lagrangian dual problem is

\begin{align} &\underset{u}{\operatorname{maximize}}& & inf _{x} \left(f(x) + \sum ^{m} _{j =1} u _{j} g _{j(x)\right)} \\ &\operatorname{subject to} & &u _{i} \geq 0 , i = 1 , ... ,m \end{align}

where the objective function is the Lagrange dual function. Provided that the functions

f and g _{1,} \ldots, g _{m} are convex and continuously differentiable, the infimum occurs where the gradient is equal to zero. The problem \begin{align} &\underset{x, u}{\operatorname{maximize}}& & f(x) + \sum ^{m} _{j =1} u _{j} g _{j(x)} \\ &\operatorname{subject to} & & \nabla f(x) + \sum ^{m} _{j =1} u _{j} \nabla g _{j(x)} = 0 \\ &&&u _{i} \geq 0 , i = 1 , ... ,m \end{align}

is called the Wolfe dual problem.^[2] This problem employs the KKT conditions as a constraint. Also, the equality constraint

\nabla f(x) + \sum ^{m} _{j =1} u _{j} \nabla g _{j(x)} is nonlinear in general, so the Wolfe dual problem may be a nonconvex optimization problem. In any case, weak duality holds.^[3]

See also

• Lagrangian duality
• Fenchel duality

Notes and References

1. Philip Wolfe. A duality theorem for non-linear programming. Quarterly of Applied Mathematics. 19. 1961. 3 . 239–244. 10.1090/qam/135625 . free.
2. Web site: Chapter 3. Duality in convex optimization. October 30, 2011. May 20, 2012.
3. Geoffrion . Arthur M. . Duality in Nonlinear Programming: A Simplified Applications-Oriented Development . 2028848 . SIAM Review . 13 . 1971 . 1–37 . 1 . 10.1137/1013001.