Notions of constrained optimization

To be at an optimum point we must have the equi-value line (the contour) of tangent to the constraint border . In other words, when we have constraints, we must have (see illustration in Figure 13.2) (the gradient of and the gradient of must aligned):

Where E is the set of active constraints, that is, the constraints which have We define Lagrangian function as:

(13.20) |

The Equation 13.19 is then equivalent to:

In unconstrained optimization, we found an optimum when . In constrained optimization, we find an optimum point ( ), called a KKT point (Karush-Kuhn-Tucker point) when:

The second equation of 13.22 is called the

To get an other insight into the meaning of Lagrange Multipliers , consider what happens if the right-hand sides of the constraints are perturbated, so that

Let , denote how the solution and multipliers change as changes. The Lagrangian for this problem is:

(13.24) |

From 13.23, , so using the chain rule, we have

(13.25) |

Using Equation 13.21, we see that the terms and are null in the previous equation. It follows that:

(13.26) |

Thus the Lagrange multiplier of any constraint measure the rate of change in the objective function, consequent upon changes in that constraint function. This information can be valuable in that it indicates how sensitive the objective function is to changes in the different constraints.