Lecture 1: The Column Space of A Contains All Vectors Ax

Professor Gilbert Strang begins the course by expressing his excitement about teaching a subject that involves learning from data, which heavily relies on linear algebra. He mentions that the course is a continuation of a previous course and introduces a public website, math.mit.edu/learningfromdata, which will be used as a resource. The professor also talks about a forthcoming book that will complement the course materials. The lecture starts with a review of linear algebra concepts, specifically matrix-vector multiplication, and emphasizes the importance of understanding the underlying concepts correctly.

The discussion shifts to the concept of the column space of a matrix, which is the set of all possible outputs from multiplying the matrix by different vectors. The professor illustrates this with a 3x3 matrix and explains that the column space could be the entire R3 space or a subset like a plane or a line. Using the example of a matrix whose columns are not independent, the audience is guided to see that the column space is a plane. The third column is identified as a sum of the first two, indicating the rank of the matrix is two, which is the number of independent columns.

The concept of rank is further explored with the help of a matrix example where the column space is a line, indicating a rank of one. The professor explains that such matrices, where the columns are multiples of each other, are fundamental in linear algebra and data science and are known as rank one matrices. A special way to write these matrices is introduced, which involves multiplying a column vector by a row vector, resulting in the original matrix.

The process of finding a basis for the column space is detailed, emphasizing the importance of selecting independent columns that span the space. The professor demonstrates this by examining each column of a given matrix and determining which ones are independent. The third column, being a combination of the first two, is excluded from the basis. The resulting matrix, C, has two independent columns and is shown to have a rank of two, which is the number of vectors in the basis.

A new matrix, R, is introduced to show how the original matrix A's columns can be obtained from the basis matrix C. This leads to a matrix factorization where A equals C times R. The professor highlights the importance of this factorization in numerical linear algebra. The discussion then leads to a key theorem in linear algebra: the column rank equals the row rank. The professor challenges the audience to find a combination of rows from matrix A that equals the third row, emphasizing the necessity of understanding this fundamental concept.

The professor continues to elaborate on the matrix factorization A = CR and its implications. He explains that the row space of a matrix is equivalent to the column space of its transpose, and that the dimensions of these spaces (the row rank and column rank) are equal. This is a fundamental fact in linear algebra. The importance of understanding the row space and how it relates to the column space is emphasized, and the professor outlines the steps to prove that certain rows form a basis for the row space of matrix A.

The lecture touches on practical applications, particularly how to deal with large matrices that cannot fit into fast memory. The concept of random sampling of a matrix is introduced, where random vectors are multiplied by the matrix to obtain a representative sample of the column space. The professor also discusses the relationship between the product of matrices and the column space of the original matrix, confirming that the product ABCx remains within the column space of A.

The professor provides details about the course structure, mentioning linear algebra problems and online homework assignments that can be completed using MATLAB, Python, or Julia. He acknowledges the contributions of Professor Raj Rao and Professor Edelman to the course and discusses the growing popularity of Julia. The professor also mentions a tutorial on Julia that will be given by Professor Johnson and encourages students to attend, especially if they are unfamiliar with the language.

The lecture concludes with a deeper exploration of matrix multiplication, specifically the multiplication of matrix A by matrix B. The professor moves beyond the traditional row-dot-column approach to a new perspective that involves multiplying columns of A by rows of B. This method is shown to be equivalent to the traditional method but offers a different insight into the process. The lecture ends with a question about the number of multiplications required for matrix multiplication, highlighting the computational cost.