Engineering Analysis/Minimization

Khun-Tucker Theorem
The Khun-Tucker Theorem is a method for minimizing a function f(x) under the constraint g(x). We can define the theorem as follows:


 * $$L(x) = f(x) + \langle \Lambda, g(x)\rangle$$

Where &Lambda; is the lagrangian vector, and <, > denotes the scalar product operation. We will discuss scalar products more later. If we differentiate this equation with respect to x first, and then with respect to &Lambda;, we get the following two equations:


 * $$\frac{\partial L(x)}{\partial x} = x + A\Lambda$$


 * $$\frac{\partial L(x)}{\partial \Lambda} = Ax - b$$

We have the final result:


 * $$x = A^T[AA^T]^{-1}b$$