Regression Analysis

Definition

Regression analysis is the process of determining how the value of a dependent variable changes when any one of independent variable changes. The values of other independent variables do not change in this process.

Application

Regression analysis is carried out to estimate the conditional expectation of dependent variables. In simple language, it is used for prediction and forecasting in a business context. This exercise also helps determine which of the independent variables is related to the dependent variables.

Procedures

Regression analysis can be done in a number of ways most common parametric methods are linear regression and ordinary least squares regression. In parametric techniques the regression function is defined as a finite number which is determined from the data. There are also non parametric techniques that do not involve a numeric value but the regression function lies in a set of functions.

Implications and limitations

The effectiveness of result generated by various regression analysis methods depends upon the form of the data generating process. Some assumptions have to be made in this analysis as a true form of the data generating process in unknown. When large amount data are available these assumptions could be testified. Even if the assumptions are a bit debased or violated, the regression model is still effective to make predictions but for the best results assumption must be accurate. Regression analysis may also produce misleading results with minor effects on observational data.

Regression Diagnostics

Effectiveness of the model and statistical significance of estimated parameters need to be confirmed, once analysis is complete and model has been created. This process is known as regression diagnostics and various techniques include: R-Squad; analyses of residual pattern; and hypothesis testing; F-test; and T-test. Interpretations of these tests rely on model assumptions.

Regression Analysis Software

Many software packages have been developed to perform least squares regression and inference. Spreadsheets and some calculators can be used for simple linear or multiple regression, while software packages can also perform various types of non parametric regression but these are less standardized.