Lec 9: Max-min problems; least squares | MIT 18.02 Multivariable Calculus, Fall 2007

This paragraph introduces the concept of partial derivatives and their application in optimization problems involving functions of multiple variables. It explains how partial derivatives, such as partial f with respect to x (f_x) and partial f with respect to y (f_y), are used to measure the rate of change of a function when one variable is changed while keeping the other constant. The approximation formula for the change in a function when both variables are varied is discussed, highlighting its importance in understanding the behavior of multivariable functions. The tangent plane approximation is introduced as a method to justify the approximation formula, providing a geometric interpretation of partial derivatives.

The paragraph delves deeper into the tangent plane approximation, explaining how the slopes of the tangent lines to the graph of a function in the x and y directions determine the tangent plane. The equations of these tangent lines are derived, and it is shown how they combine to form the equation of the tangent plane. The paragraph also discusses how the graph of a function closely approximates its tangent plane, especially for small changes in x and y, and how this approximation can be used to estimate changes in the function's value. The concept is further clarified with the idea that if both partial derivatives are zero, the tangent plane is horizontal, indicating a potential maximum or minimum.

This section focuses on the concept of critical points in multivariable functions, where both partial derivatives are zero. It explains the necessity of this condition for a point to be considered a local minimum or maximum. The importance of having both partial derivatives being zero simultaneously is discussed, as it ensures that the function does not change to the first order when either variable is perturbed. The paragraph also introduces the idea that not all points where partial derivatives are zero are minima or maxima, and that further analysis is required to classify them.

The paragraph presents an example function and guides through the process of finding its critical points by setting the partial derivatives to zero. It emphasizes the importance of maintaining two separate equations when solving for the variables. The example demonstrates how to simplify the equations and solve for the critical point, which in this case is found to be (-1, 0). The discussion also touches on the challenge of determining whether a critical point is a maximum, minimum, or saddle point, with a promise to address this in the next session.

This paragraph continues the discussion on the example function by attempting to express it as a sum of squares, which simplifies the process of identifying the nature of the critical point. The speaker completes the square for the given function and shows that the critical point corresponds to the minimum value of the function. The method of completing the square is illustrated, and it is noted that while this approach was successful for the example, more general methods will be discussed in future sessions.

The paragraph introduces the application of minimization in the context of least squares fitting, a common technique in experimental sciences. It discusses how to find the 'best fit' line for a set of data points, emphasizing that the goal is to minimize the total square error between the observed data and the predicted values from the line. The paragraph also touches on the limitations of using such models for unpredictable phenomena like the stock market.

This section provides a detailed explanation of how to derive the best fit line using the least squares method. It discusses setting up the error function and finding its critical points by taking partial derivatives with respect to the coefficients of the line. The process of simplifying the equations to form a linear system that can be solved for the coefficients is outlined. The paragraph also explains why this method results in a minimum error, although the detailed justification is reserved for a future session.

The final paragraph extends the concept of least squares fitting to more complex relationships beyond linear, such as exponential and quadratic fits. It uses Moore's law as an example to illustrate how an exponential fit can be determined by linearizing the problem through a logarithmic transformation. The paragraph also briefly mentions how to approach fitting a quadratic function to data, which involves setting up and solving a three-by-three linear system for the coefficients.