Lecture #20: Optimization of Multivariable Functions

Optimizing Multivariable Functions

Last lecture, we discussed how to optimize single variable functions.

Local Extrema of Functions \(f(x, y)\)

How can we find the local extrema of a function \(f(x, y)\) like the graph below?

Local Extrema of Functions \(f(x, y)\)

Let’s try using the partial derivatives for \(f(x, y)\), \(\dfrac{\partial f}{\partial f}{x}\) and \(\dfrac{\partial f}{\partial f}{y}\), the way we used first derivatives for \(f(x)\), \(\dfrac{df}{dx}\).

Critical Points of Functions \(f(x, y)\)

A critical point of a function \(f(x, y)\) is a point \((x, y)\) where:

\[ \dfrac{\partial f}{\partial f}{x} (x,y) = \dfrac{\partial f}{\partial f}{y} (x, y) = 0 \]

Example: Critical Points of Functions \(f(x, y)\)

Example: Find the critical points of the function \(f(x, y) = x^2 - y^2 + 4x + 2y\).

The Gradient, \(\nabla f\)

The gradient of a function \(f(x, y)\) is the coordinate pair \(\left( \partial_x f, \partial_y f \right)\). We write this as \(\nabla f\).

Example: The Gradient, \(\triangledown f\)

Example: Find the gradient of the function \(f(x, y) = x^2 - y^2 + 4x + 2y\).

Local Minimum

Local Maximum

Saddle Point

The Second Derivative Test

Suppose \((x_0, y_0)\) is a point where \(\nabla f (x_0, y_0) = (0, 0)\). Let

\[ D = f_{xx} (x_0, y_0) f_{yy} (x_0, y_0) - \left( f_{xy} (x_0, y_0) \right)^2 \]

1. If \(D > 0\) and \(f_{xx} (x_0, y_0) > 0\), then \(f\) has a local minimum at \((x_0, y_0)\).
2. If \(D > 0\) and \(f_{xx} (x_0, y_0) < 0\), then \(f\) has a local maximum at \((x_0, y_0)\).
3. If \(D < 0\), then \(f\) has a saddle point at \((x_0, y_0)\).
4. If \(D = 0\), anything can happen: \(f\) can have a local maximum, or a local minimum, or a saddle point, or none of these at \((x_0, y_0)\).

Example: The Second Derivative Test

Example: Determine the behavior of the function \(f(x, y) = x^2 - y^2 + 4x + 2y\) at its critical point.