Correlation and Regression
UGC NET Commerce: scatter diagram, Karl Pearson's r, probable error, Spearman's rank correlation with ties, regression lines and coefficients, worked examples.
- 17 min
On this page · 14 sections
Prerequisites
- Central Tendency, Dispersion and Skewness — means and standard deviations are the raw material of every formula here.
- Methods and Steps of Research — correlational research, and why correlation is not causation.
- Business Statistics and Research Methods: chapter overview
The Hook: Advertising and Sales
1. Correlation: Meaning and Types
Two variables are correlated when a change in one is accompanied by a change in the other.
| Basis | Types |
|---|---|
| Direction | Positive (both move the same way: price and supply) · Negative (opposite ways: price and demand) |
| Form | Linear (constant ratio of change; points near a straight line) · Non-linear / curvilinear |
| Number of variables | Simple (two variables) · Partial (two variables, others held constant) · Multiple (one variable with two or more jointly) |
Methods of studying correlation
| Method | Nature | Gives |
|---|---|---|
| Scatter diagram (dot diagram) | Graphic | Direction and rough degree; non-linearity visible; no exact value |
| Karl Pearson’s coefficient (r) | Algebraic, product-moment | Exact degree and direction of linear relation |
| Spearman’s rank correlation (R) | Algebraic, on ranks | Correlation for qualitative or ranked data |
| Concurrent deviation method | Algebraic, on directions of change only | A quick, rough coefficient |
| Correlation graph | Graphic (two curves over time) | Visual co-movement |
2. Karl Pearson’s Coefficient of Correlation
$$r = \frac{\text{Cov}(X, Y)}{\sigma_X \, \sigma_Y} = \frac{\sum (X - \bar{X})(Y - \bar{Y})}{\sqrt{\sum (X - \bar{X})^2 \; \sum (Y - \bar{Y})^2}}$$A direct-values version, useful when means are not whole numbers:
$$r = \frac{n\sum XY - \sum X \sum Y}{\sqrt{n\sum X^2 - (\sum X)^2} \; \sqrt{n\sum Y^2 - (\sum Y)^2}}$$Interpreting r
| Value of r | Meaning |
|---|---|
| +1 | Perfect positive linear correlation |
| 0.75 to < 1 | High positive (a common textbook convention) |
| 0 | No linear correlation (a curved relation may still exist) |
| −1 | Perfect negative linear correlation |
Properties of r
- −1 ≤ r ≤ +1.
- r is a pure number, free of units.
- r is independent of change of origin and scale: if \( U = (X - a)/h \) and \( V = (Y - b)/k \), then \( r_{UV} = r_{XY} \) when h and k have the same sign (if their signs differ, r changes sign only).
- r is the geometric mean of the two regression coefficients: \( r = \pm\sqrt{b_{YX} \times b_{XY}} \), taking the common sign of the coefficients.
- If X and Y are independent, r = 0; but r = 0 does not imply independence (only absence of a linear relation).
Coefficient of determination
\( r^2 \) is the coefficient of determination: the proportion of the variance in Y explained by its linear relation with X. With r = 0.8, \( r^2 = 0.64 \), so 64% of the variation in Y is explained and 36% (\( 1 - r^2 \), the coefficient of non-determination) is not. This is why r = 0.6 is not “twice as strong” as r = 0.3: it explains 36% against 9%.
Probable error
The probable error of r tests whether a sample r is meaningful:
$$PE_r = 0.6745 \times \frac{1 - r^2}{\sqrt{n}}$$Rule of thumb: if \( r < PE \), the correlation is not significant; if \( r > 6\,PE \), it is significant. The population r is expected to lie within \( r \pm PE \) with probability 0.5 (the reason for the constant 0.6745: the middle 50% of a normal curve lies within ±0.6745σ). The modern equivalent is the t-test for r on the hypothesis testing page.
3. Spearman’s Rank Correlation
When the data are ranks (judges ranking contestants, consumers ranking brands) or are qualitative (beauty, honesty), use Charles Spearman’s coefficient (1904):
$$R = 1 - \frac{6\sum D^2}{n(n^2 - 1)}$$where D is the difference between the two ranks of each item. R also lies between −1 and +1, and \( \sum D = 0 \) always (a quick check on your ranking).
Tied ranks. Give tied items the average of the ranks they would occupy, and add a correction factor \( \frac{m(m^2 - 1)}{12} \) to \( \sum D^2 \) for each group of m tied items:
$$R = 1 - \frac{6\left[\sum D^2 + \sum \frac{m(m^2 - 1)}{12}\right]}{n(n^2 - 1)}$$Merits: simple, works for qualitative data, not distorted by extreme values. Limits: loses information when actual values are converted to ranks; tedious for large n. Pearson’s r computed on the ranks themselves equals Spearman’s R when there are no ties.
Concurrent deviation method
Note only the direction of change (+ or −) of each series from the previous value. With c the number of concurrent (same-sign) deviations and n the number of pairs of deviations (one fewer than the observations):
$$r_c = \pm\sqrt{\pm\frac{2c - n}{n}}$$The signs inside and outside follow the sign of \( 2c - n \). It is quick but ignores the size of changes.
4. Regression Analysis
The term regression comes from Francis Galton, whose 1886 paper on “regression towards mediocrity in hereditary stature” found that the children of very tall parents tended to be less tall than their parents, “stepping back” towards the average.
Regression estimates the average relationship between a dependent variable (explained, predicted) and an independent variable (explanatory, predictor), and uses it for prediction.
Two regression lines
There are two lines, because minimising errors in Y is not the same as minimising errors in X.
| Line | Used to estimate | Equation (deviation form) | Coefficient |
|---|---|---|---|
| Y on X | Y for a given X | \( Y - \bar{Y} = b_{YX}(X - \bar{X}) \) | \( b_{YX} = r\dfrac{\sigma_Y}{\sigma_X} = \dfrac{\sum xy}{\sum x^2} \) |
| X on Y | X for a given Y | \( X - \bar{X} = b_{XY}(Y - \bar{Y}) \) | \( b_{XY} = r\dfrac{\sigma_X}{\sigma_Y} = \dfrac{\sum xy}{\sum y^2} \) |
Each line is fitted by the method of least squares: it minimises the sum of squared vertical deviations (Y on X) or horizontal deviations (X on Y). For \( Y = a + bX \), the normal equations are:
$$\sum Y = na + b\sum X, \qquad \sum XY = a\sum X + b\sum X^2$$Properties of regression coefficients and lines
- \( r = \pm\sqrt{b_{YX} \times b_{XY}} \), so \( b_{YX} \times b_{XY} = r^2 \le 1 \). If one coefficient exceeds 1, the other must be less than 1.
- Both coefficients and r carry the same sign.
- The arithmetic mean of the two coefficients is at least r (in absolute value): \( \frac{b_{YX} + b_{XY}}{2} \ge r \) for positive r.
- Regression coefficients are independent of change of origin but not of scale (contrast r, which is independent of both).
- The two lines intersect at the means \( (\bar{X}, \bar{Y}) \).
- If \( r = \pm 1 \), the two lines coincide. If \( r = 0 \), they are perpendicular (parallel to the axes: \( Y = \bar{Y} \) and \( X = \bar{X} \)). The smaller the angle between them, the higher the correlation.
- The standard error of estimate of Y on X is \( S_{YX} = \sigma_Y\sqrt{1 - r^2} \): the typical scatter of actual Y values around the regression line. It is zero when \( r = \pm 1 \).
Finding means and r from two given lines
A favourite numerical. Given two lines without being told which is which:
- Means: solve the two equations simultaneously (the lines meet at the means).
- Which is which: assume one is Y on X, find both coefficients, and check that their product is ≤ 1. If it exceeds 1, swap the assumption.
- r: the square root of the product, with the common sign.
5. Correlation vs Regression
| Basis | Correlation | Regression |
|---|---|---|
| Measures | Degree and direction of association | Nature (form) of the average relationship |
| Variables | Treated symmetrically; \( r_{XY} = r_{YX} \) | Dependent and independent distinguished; \( b_{YX} \ne b_{XY} \) in general |
| Causation | Says nothing about cause | Presumes a direction (X explains Y) for prediction |
| Prediction | Not possible | Main purpose |
| Origin and scale | r independent of both | Coefficients independent of origin, not scale |
| Range | −1 to +1 | Coefficients can exceed 1 (but not both) |
| Spurious results | Nonsense correlation possible | Nonsense regression possible too |
Memory Tricks
Don’t Confuse
| This | vs | That |
|---|---|---|
| r: symmetric, −1 to +1 | vs | b: asymmetric, can exceed 1 (but not both) |
| r²: coefficient of determination (explained share) | vs | 1 − r²: coefficient of non-determination |
| Pearson’s r: actual values, linear | vs | Spearman’s R: ranks, works for qualitative data |
| Probable error: 0.6745 × (1 − r²)/√n | vs | Standard error of r: (1 − r²)/√n |
| Y on X: minimises vertical deviations; estimates Y | vs | X on Y: minimises horizontal deviations; estimates X |
| r = 0: lines perpendicular | vs | r = ±1: lines coincide |
| Simple correlation | vs | Partial (others held constant) and multiple (one with several jointly) |
| Galton: coined regression | vs | Spearman: rank correlation (1904); Karl Pearson: product-moment r |
Common Mistakes
Practice
All questions below are original practice questions written in the UGC NET format. They are not previous-year questions.
Level 1: Foundation
- If \( b_{YX} = 0.8 \) and \( b_{XY} = 0.45 \), what is r?
- What is the coefficient of determination when r = −0.7?
- Who introduced the term “regression”?
Answers
- \( r = \sqrt{0.8 \times 0.45} = \sqrt{0.36} = \) 0.6 (positive, as both coefficients are positive).
- \( r^2 = \) 0.49: 49% of the variation is explained.
- Francis Galton.
Level 2: Exam-style MCQs
Q1. (Practice) The covariance between X and Y is 12, and their standard deviations are 3 and 5. The coefficient of correlation is:
- (a) 0.48
- (b) 0.64
- (c) 0.80
- (d) 1.25
Answer and explanation
(c) 0.80. \( r = 12/(3 \times 5) = 0.8 \).
Q2. (Practice) For ten pairs of ranks without ties, \( \sum D^2 = 33 \). Spearman’s rank correlation is:
- (a) 0.67
- (b) 0.80
- (c) 0.20
- (d) 0.33
Answer and explanation
(b) 0.80. \( R = 1 - \dfrac{6 \times 33}{10 \times 99} = 1 - \dfrac{198}{990} = 1 - 0.2 = 0.8 \).
Q3. (Practice) Which pair cannot be the two regression coefficients of the same data?
- (a) 0.9 and 0.4
- (b) −1.2 and −0.5
- (c) 1.5 and 0.8
- (d) 0.3 and 2.0
Answer and explanation
(c) 1.5 and 0.8. Their product is 1.2, which would make \( r^2 > 1 \). The others give products 0.36, 0.6 and 0.6.
Q4. (Practice) \( \sigma_Y = 10 \) and r = 0.6. The standard error of estimate of Y on X is:
- (a) 4
- (b) 6
- (c) 8
- (d) 6.4
Answer and explanation
(c) 8. \( S_{YX} = 10\sqrt{1 - 0.36} = 10 \times 0.8 = 8 \).
Q5. (Practice) Consider the following statements:
- The two regression lines intersect at the point of means.
- If r = 0, the two regression lines coincide.
- Regression coefficients are independent of change of origin.
Which of the statements given above is/are correct?
- (a) 1 and 2 only
- (b) 1 and 3 only
- (c) 2 and 3 only
- (d) 1, 2 and 3
Answer and explanation
(b) 1 and 3 only. If r = 0 the lines are perpendicular; they coincide when r = ±1.
Level 3: Assertion–reason, match and sequence
For assertion–reason questions, use these options:
- (a) Both A and R are true, and R is the correct explanation of A
- (b) Both A and R are true, but R is not the correct explanation of A
- (c) A is true, but R is false
- (d) A is false, but R is true
Q1. (Practice) Assertion (A): If one regression coefficient is greater than one, the other must be less than one. Reason (R): The product of the two regression coefficients equals \( r^2 \), which cannot exceed one.
Answer and explanation
(a). R is exactly the reason: \( b_{YX} b_{XY} = r^2 \le 1 \).
Q2. (Practice) Assertion (A): A correlation coefficient of zero shows that the two variables are unrelated. Reason (R): Karl Pearson’s r measures only the linear relationship between two variables.
Answer and explanation
(d). A is false: r = 0 rules out only a linear relationship. R is true, and it is the reason A is false.
Q3. (Practice) Match List I with List II.
| List I (Measure) | List II (Formula or feature) |
|---|---|
| A. Coefficient of determination | I. \( 0.6745 (1 - r^2)/\sqrt{n} \) |
| B. Probable error of r | II. \( r^2 \) |
| C. Standard error of estimate | III. \( 1 - 6\sum D^2 / n(n^2 - 1) \) |
| D. Rank correlation | IV. \( \sigma_Y\sqrt{1 - r^2} \) |
- (1) A-II, B-I, C-IV, D-III
- (2) A-I, B-II, C-IV, D-III
- (3) A-II, B-IV, C-I, D-III
- (4) A-IV, B-I, C-II, D-III
Answer and explanation
(1) A-II, B-I, C-IV, D-III.
Q4. (Practice) Match the contributor with the contribution.
| List I | List II |
|---|---|
| A. Francis Galton | I. Rank correlation coefficient |
| B. Charles Spearman | II. Coined “regression” |
| C. Karl Pearson | III. Product-moment correlation coefficient |
- (1) A-II, B-I, C-III
- (2) A-I, B-II, C-III
- (3) A-III, B-I, C-II
- (4) A-II, B-III, C-I
Answer and explanation
(1) A-II, B-I, C-III.
Q5. (Practice) Arrange the steps for finding r from two given regression lines in the correct order:
A. Take the square root of the product of the coefficients, with their common sign
B. Assume which line is Y on X and write both coefficients
C. Check that the product of the coefficients does not exceed 1, swapping the assumption if it does
D. Solve the two equations simultaneously for the means
(1) D, B, C, A
(2) B, D, A, C
(3) D, C, B, A
(4) B, C, D, A
Answer and explanation
(1) D, B, C, A. The means can be found first (they do not depend on which line is which); the coefficients need an assumption, a check, and then the square root.
Quick Revision Table
| Item | Formula or fact |
|---|---|
| Pearson’s r | \( \text{Cov}(X, Y)/(\sigma_X\sigma_Y) \); −1 to +1; free of origin and scale |
| r² / 1 − r² | Determination / non-determination |
| Probable error | \( 0.6745(1 - r^2)/\sqrt{n} \); significant if r > 6 PE |
| Spearman (1904) | \( 1 - 6\sum D^2/n(n^2 - 1) \); ties add \( m(m^2 - 1)/12 \) per group |
| Concurrent deviation | \( \pm\sqrt{\pm(2c - n)/n} \) |
| Galton (1886) | “Regression towards mediocrity” |
| \( b_{YX} \) / \( b_{XY} \) | \( r\sigma_Y/\sigma_X \) / \( r\sigma_X/\sigma_Y \) |
| Link | \( r = \pm\sqrt{b_{YX}b_{XY}} \); product ≤ 1; same signs |
| Origin and scale | b free of origin only |
| Lines | Meet at \( (\bar{X}, \bar{Y}) \); coincide if r = ±1; perpendicular if r = 0 |
| Least squares | \( \sum Y = na + b\sum X \); \( \sum XY = a\sum X + b\sum X^2 \) |
| SE of estimate | \( \sigma_Y\sqrt{1 - r^2} \) |
Key Takeaways
What’s Next
- Probability and Probability Distributions — from describing data to modelling uncertainty
- Hypothesis Testing and Report Writing — testing whether r and R are significant
- Data Interpretation — the Paper 1 unit where two-variable tables reappear
- Quick Revision: Business Statistics and Research Methods