p-Values

Definition

The p-value tells us “what is the probability that I’m completely wrong and this variable actually does nothing.”

A p-value is the probability of observing a test statistic as extreme or more extreme than the one obtained, assuming the null hypothesis is true.

It helps us measure the strength of evidence against the null hypothesis ($H_0$).

Purpose

To decide whether the observed relationship or effect in a regression (or any statistical test) is statistically significant, rather than due to random chance.

Interpretation

• A small p-value (typically $<0.05$) → strong evidence against $H_0$.
• A large p-value (typically $>0.05$) → weak evidence against $H_0$ (fail to reject).

In regression:

• The null hypothesis (or the hypothesis that this variable does not affect our results) for a coefficient is:

$$H_0:{\beta }_i=0$$

• A low p-value suggests that predictor $X_i$ significantly contributes to explaining variation in $Y$.

Common Thresholds

p-value	Interpretation	Decision
$p<0.05$	Strong evidence against $H_0$	Reject $H_0$
$p\ge 0.05$	Weak evidence against $H_0$	Fail to reject $H_0$

Example

Suppose a regression output gives:

Variable	Coefficient ($\beta$)	p-value
$X_1$ (Hours Studied)	3.2	0.01
$X_2$ (Tutoring)	1.1	0.45

Interpretation:

• $X_1$ is statistically significant → more study hours increase grades.
• $X_2$ is not significant → tutoring shows no measurable effect after controlling for other factors.

Key Idea

The p-value does not measure the size or importance of an effect — only how likely it is that the observed result occurred by random chance under $H_0$.