subject: Least Squares Regression [print this page] Introduction to Least Squares Regression Introduction to Least Squares Regression
Suppose we wish to predict the values of the variable Y from the value of the variable X.
Principle of least square states that the curve of best fit for the given pair of observation is that curve that makes the sum of the squares of the differences between observed value and the estimated value called residuals a minimum.
The straight lines are filled by the principle of least squares to the pair of observations (x , y) plotted on the scatter diagram are called regression lines. The points on the scatter cluster themselves along these lines.
Check this awesome Function Calculator i recently used, click here to see
Let us take the regression equation as y=a+bx
Thus, if xi is an observed value of X, then the predicted value of Y for the given value of X will be a+bxi.We are thus restricting ourselves to the use of linear regression function.The value of a and b can be determined by the method of differential calculus for obtaining the maximum and minimum of functions.
Once the values of a and b is determined , the regression equation takes the simple form
' y - bary = (Cov(X ,Y))/sigma^2 (x - bar x)'
where 'sigma^2 '
denotes the variance of X.
This equation is the equation of the least square line of regression of Y on X.
The constant '(Cov(X, Y))/sigma^2 '
is called the regression coefficient of Y on X and is denoted by bxy.
Similarly we can obtain the least square line of regression of X on Y and obtain
' x - bar x = (Cov(X ,Y))/sigma^2 (y - bar y)'
where 'sigma^2 '
denotes the variance of Y.
This equation is the equation of the least square line of regression of X on Y.
The constant '(Cov(X, Y))/sigma^2 '
is called the regression coefficient of X on Y and is denoted by byx.
The lines of regression are called least square lines of regression because they have been obtained by minimising the sums of squares.
