Regression

Definition

Regression is a statistical method used to model the relationship between a dependent variable (Y) and one or more independent variables (X₁, X₂, …, Xₙ).

Purpose

To understand how changes in the independent variables are associated with changes in the dependent variable, and to make predictions.

General Form

$$Y={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+\cdots +{\beta }_n{X}_n+\epsilon$$

• $Y$: dependent (response) variable
• $X_i$: independent (predictor) variables
• ${\beta }_i$: coefficients showing the effect of each $X_i$
• $\epsilon$: random error term

Key Idea

Each coefficient ${\beta }_i$ represents the expected change in Y when $X_i$ increases by one unit, holding all other variables constant.

Example

Predicting house price ($Y$) from size ($X_1$) and number of bedrooms ($X_2$):

$$\text{Price}={\beta }_0+{\beta }_1\text{Size}+{\beta }_2\text{Bedrooms}+\epsilon$$