Degrees Of Freedom Explained | What is Degrees of freedom | Degrees of freedom in statistics

In this introductory paragraph, the speaker, Aman, a data scientist, welcomes the audience to a video on the concept of degrees of freedom in the context of data science. He aims to simplify the topic, which is commonly misunderstood and frequently asked about in interviews. Aman plans to explain the generic definition of degrees of freedom using a straightforward example and then relate it to regression analysis. He also promises to demonstrate the concept with a Python example. The paragraph sets the stage for a deeper dive into degrees of freedom, establishing it as a pivotal topic for those in the field.

This paragraph delves into the concept of degrees of freedom through a classroom example involving three students, A, B, and C, with respective scores of 10, 5, and 15. Aman illustrates how the average score is calculated and how the differences from the mean must sum to zero, a fundamental principle in statistics. He uses this example to explain that the third score is not freely variable because it must balance the other two to maintain the sum at zero, thus introducing the idea of degrees of freedom as the number of values that can be varied without breaking the underlying rules of the system. The paragraph concludes with a clear definition of degrees of freedom in this context, which is two, as only the first two scores can be freely changed.

Aman extends the discussion of degrees of freedom to the realm of linear regression. He uses the equation y = mx + c to explore how many lines can be drawn without constraints, which is infinite, and then introduces constraints to demonstrate how degrees of freedom decrease as more information is given. For instance, fixing the value of c reduces the degrees of freedom to one, as lines must pass through a specific point. If both m and c are fixed, only one line can be drawn, reducing the degrees of freedom to zero. The speaker then transitions to a more complex scenario with one target variable and one independent variable, discussing the minimum number of data points required to estimate the relationship between them. He explains that with three data points, a meaningful relationship can be derived, as this provides one degree of freedom for the model to learn the pattern. The paragraph concludes with the general formula for degrees of freedom in regression, which is n - k - 1, where n is the number of data points, and k is the number of independent variables.

Building upon the previous discussion, Aman introduces a scenario with two independent features and a target variable, aiming to establish the relationship y = beta0 + beta1*x1 + beta2*x2. He asks how many minimum data points are needed to define this relationship meaningfully, using a three-dimensional plot as a visual aid. The speaker explains that in a multi-dimensional space, the minimum degrees of freedom required increases because a plane, rather than a line, is needed to represent the relationship. The formula n - k - 1 still applies, but with an adjusted value for k to account for the additional independent variable. Aman emphasizes that the number of data points (n) must be at least four to achieve a single degree of freedom when dealing with two independent variables. The paragraph reinforces the concept that more degrees of freedom allow for greater model learning capability.

In the final paragraph, Aman provides a practical demonstration of degrees of freedom using a Python example with a dataset consisting of x_train, x_test, y_train, y_test, and three independent columns. He explains that with 100 rows of training data and three independent features, the degrees of freedom would be calculated as n - k - 1, which should equal 96. However, he notes that the example in Python, using a stats model OLS (Ordinary Least Squares), shows 97 degrees of freedom because the model does not include an intercept by default. If an intercept were included, the degrees of freedom would align with the formula. Aman concludes by reiterating the importance of understanding degrees of freedom as a measure of the number of freely varying parameters in a model and encourages viewers to ask questions and suggest topics for future videos.