Skip to content

Correlation and Regression

UGC NET Commerce: scatter diagram, Karl Pearson's r, probable error, Spearman's rank correlation with ties, regression lines and coefficients, worked examples.

  • 17 min read
  • Updated
On this page · 14 sections
  1. Prerequisites
  2. The Hook: Advertising and Sales
  3. 1. Correlation: Meaning and Types
  4. 2. Karl Pearson’s Coefficient of Correlation
  5. 3. Spearman’s Rank Correlation
  6. 4. Regression Analysis
  7. 5. Correlation vs Regression
  8. Memory Tricks
  9. Don’t Confuse
  10. Common Mistakes
  11. Practice
  12. Quick Revision Table
  13. Key Takeaways
  14. What’s Next

Prerequisites

The Hook: Advertising and Sales


1. Correlation: Meaning and Types

Two variables are correlated when a change in one is accompanied by a change in the other.

BasisTypes
DirectionPositive (both move the same way: price and supply) · Negative (opposite ways: price and demand)
FormLinear (constant ratio of change; points near a straight line) · Non-linear / curvilinear
Number of variablesSimple (two variables) · Partial (two variables, others held constant) · Multiple (one variable with two or more jointly)

Methods of studying correlation

MethodNatureGives
Scatter diagram (dot diagram)GraphicDirection and rough degree; non-linearity visible; no exact value
Karl Pearson’s coefficient (r)Algebraic, product-momentExact degree and direction of linear relation
Spearman’s rank correlation (R)Algebraic, on ranksCorrelation for qualitative or ranked data
Concurrent deviation methodAlgebraic, on directions of change onlyA quick, rough coefficient
Correlation graphGraphic (two curves over time)Visual co-movement

2. Karl Pearson’s Coefficient of Correlation

$$r = \frac{\text{Cov}(X, Y)}{\sigma_X \, \sigma_Y} = \frac{\sum (X - \bar{X})(Y - \bar{Y})}{\sqrt{\sum (X - \bar{X})^2 \; \sum (Y - \bar{Y})^2}}$$

A direct-values version, useful when means are not whole numbers:

$$r = \frac{n\sum XY - \sum X \sum Y}{\sqrt{n\sum X^2 - (\sum X)^2} \; \sqrt{n\sum Y^2 - (\sum Y)^2}}$$

Interpreting r

Value of rMeaning
+1Perfect positive linear correlation
0.75 to < 1High positive (a common textbook convention)
0No linear correlation (a curved relation may still exist)
−1Perfect negative linear correlation

Properties of r

  1. −1 ≤ r ≤ +1.
  2. r is a pure number, free of units.
  3. r is independent of change of origin and scale: if \( U = (X - a)/h \) and \( V = (Y - b)/k \), then \( r_{UV} = r_{XY} \) when h and k have the same sign (if their signs differ, r changes sign only).
  4. r is the geometric mean of the two regression coefficients: \( r = \pm\sqrt{b_{YX} \times b_{XY}} \), taking the common sign of the coefficients.
  5. If X and Y are independent, r = 0; but r = 0 does not imply independence (only absence of a linear relation).

Coefficient of determination

\( r^2 \) is the coefficient of determination: the proportion of the variance in Y explained by its linear relation with X. With r = 0.8, \( r^2 = 0.64 \), so 64% of the variation in Y is explained and 36% (\( 1 - r^2 \), the coefficient of non-determination) is not. This is why r = 0.6 is not “twice as strong” as r = 0.3: it explains 36% against 9%.

Probable error

The probable error of r tests whether a sample r is meaningful:

$$PE_r = 0.6745 \times \frac{1 - r^2}{\sqrt{n}}$$

Rule of thumb: if \( r < PE \), the correlation is not significant; if \( r > 6\,PE \), it is significant. The population r is expected to lie within \( r \pm PE \) with probability 0.5 (the reason for the constant 0.6745: the middle 50% of a normal curve lies within ±0.6745σ). The modern equivalent is the t-test for r on the hypothesis testing page.


3. Spearman’s Rank Correlation

When the data are ranks (judges ranking contestants, consumers ranking brands) or are qualitative (beauty, honesty), use Charles Spearman’s coefficient (1904):

$$R = 1 - \frac{6\sum D^2}{n(n^2 - 1)}$$

where D is the difference between the two ranks of each item. R also lies between −1 and +1, and \( \sum D = 0 \) always (a quick check on your ranking).

Tied ranks. Give tied items the average of the ranks they would occupy, and add a correction factor \( \frac{m(m^2 - 1)}{12} \) to \( \sum D^2 \) for each group of m tied items:

$$R = 1 - \frac{6\left[\sum D^2 + \sum \frac{m(m^2 - 1)}{12}\right]}{n(n^2 - 1)}$$

Merits: simple, works for qualitative data, not distorted by extreme values. Limits: loses information when actual values are converted to ranks; tedious for large n. Pearson’s r computed on the ranks themselves equals Spearman’s R when there are no ties.

Concurrent deviation method

Note only the direction of change (+ or −) of each series from the previous value. With c the number of concurrent (same-sign) deviations and n the number of pairs of deviations (one fewer than the observations):

$$r_c = \pm\sqrt{\pm\frac{2c - n}{n}}$$

The signs inside and outside follow the sign of \( 2c - n \). It is quick but ignores the size of changes.


4. Regression Analysis

The term regression comes from Francis Galton, whose 1886 paper on “regression towards mediocrity in hereditary stature” found that the children of very tall parents tended to be less tall than their parents, “stepping back” towards the average.

Regression estimates the average relationship between a dependent variable (explained, predicted) and an independent variable (explanatory, predictor), and uses it for prediction.

Two regression lines

There are two lines, because minimising errors in Y is not the same as minimising errors in X.

LineUsed to estimateEquation (deviation form)Coefficient
Y on XY for a given X\( Y - \bar{Y} = b_{YX}(X - \bar{X}) \)\( b_{YX} = r\dfrac{\sigma_Y}{\sigma_X} = \dfrac{\sum xy}{\sum x^2} \)
X on YX for a given Y\( X - \bar{X} = b_{XY}(Y - \bar{Y}) \)\( b_{XY} = r\dfrac{\sigma_X}{\sigma_Y} = \dfrac{\sum xy}{\sum y^2} \)

Each line is fitted by the method of least squares: it minimises the sum of squared vertical deviations (Y on X) or horizontal deviations (X on Y). For \( Y = a + bX \), the normal equations are:

$$\sum Y = na + b\sum X, \qquad \sum XY = a\sum X + b\sum X^2$$

Properties of regression coefficients and lines

  1. \( r = \pm\sqrt{b_{YX} \times b_{XY}} \), so \( b_{YX} \times b_{XY} = r^2 \le 1 \). If one coefficient exceeds 1, the other must be less than 1.
  2. Both coefficients and r carry the same sign.
  3. The arithmetic mean of the two coefficients is at least r (in absolute value): \( \frac{b_{YX} + b_{XY}}{2} \ge r \) for positive r.
  4. Regression coefficients are independent of change of origin but not of scale (contrast r, which is independent of both).
  5. The two lines intersect at the means \( (\bar{X}, \bar{Y}) \).
  6. If \( r = \pm 1 \), the two lines coincide. If \( r = 0 \), they are perpendicular (parallel to the axes: \( Y = \bar{Y} \) and \( X = \bar{X} \)). The smaller the angle between them, the higher the correlation.
  7. The standard error of estimate of Y on X is \( S_{YX} = \sigma_Y\sqrt{1 - r^2} \): the typical scatter of actual Y values around the regression line. It is zero when \( r = \pm 1 \).

Finding means and r from two given lines

A favourite numerical. Given two lines without being told which is which:

  1. Means: solve the two equations simultaneously (the lines meet at the means).
  2. Which is which: assume one is Y on X, find both coefficients, and check that their product is ≤ 1. If it exceeds 1, swap the assumption.
  3. r: the square root of the product, with the common sign.

5. Correlation vs Regression

BasisCorrelationRegression
MeasuresDegree and direction of associationNature (form) of the average relationship
VariablesTreated symmetrically; \( r_{XY} = r_{YX} \)Dependent and independent distinguished; \( b_{YX} \ne b_{XY} \) in general
CausationSays nothing about causePresumes a direction (X explains Y) for prediction
PredictionNot possibleMain purpose
Origin and scaler independent of bothCoefficients independent of origin, not scale
Range−1 to +1Coefficients can exceed 1 (but not both)
Spurious resultsNonsense correlation possibleNonsense regression possible too

Memory Tricks

Don’t Confuse

ThisvsThat
r: symmetric, −1 to +1vsb: asymmetric, can exceed 1 (but not both)
r²: coefficient of determination (explained share)vs1 − r²: coefficient of non-determination
Pearson’s r: actual values, linearvsSpearman’s R: ranks, works for qualitative data
Probable error: 0.6745 × (1 − r²)/√nvsStandard error of r: (1 − r²)/√n
Y on X: minimises vertical deviations; estimates YvsX on Y: minimises horizontal deviations; estimates X
r = 0: lines perpendicularvsr = ±1: lines coincide
Simple correlationvsPartial (others held constant) and multiple (one with several jointly)
Galton: coined regressionvsSpearman: rank correlation (1904); Karl Pearson: product-moment r

Common Mistakes


Practice

All questions below are original practice questions written in the UGC NET format. They are not previous-year questions.

Level 1: Foundation

  1. If \( b_{YX} = 0.8 \) and \( b_{XY} = 0.45 \), what is r?
  2. What is the coefficient of determination when r = −0.7?
  3. Who introduced the term “regression”?
Answers
  1. \( r = \sqrt{0.8 \times 0.45} = \sqrt{0.36} = \) 0.6 (positive, as both coefficients are positive).
  2. \( r^2 = \) 0.49: 49% of the variation is explained.
  3. Francis Galton.

Level 2: Exam-style MCQs

Q1. (Practice) The covariance between X and Y is 12, and their standard deviations are 3 and 5. The coefficient of correlation is:

  • (a) 0.48
  • (b) 0.64
  • (c) 0.80
  • (d) 1.25
Answer and explanation

(c) 0.80. \( r = 12/(3 \times 5) = 0.8 \).

Q2. (Practice) For ten pairs of ranks without ties, \( \sum D^2 = 33 \). Spearman’s rank correlation is:

  • (a) 0.67
  • (b) 0.80
  • (c) 0.20
  • (d) 0.33
Answer and explanation

(b) 0.80. \( R = 1 - \dfrac{6 \times 33}{10 \times 99} = 1 - \dfrac{198}{990} = 1 - 0.2 = 0.8 \).

Q3. (Practice) Which pair cannot be the two regression coefficients of the same data?

  • (a) 0.9 and 0.4
  • (b) −1.2 and −0.5
  • (c) 1.5 and 0.8
  • (d) 0.3 and 2.0
Answer and explanation

(c) 1.5 and 0.8. Their product is 1.2, which would make \( r^2 > 1 \). The others give products 0.36, 0.6 and 0.6.

Q4. (Practice) \( \sigma_Y = 10 \) and r = 0.6. The standard error of estimate of Y on X is:

  • (a) 4
  • (b) 6
  • (c) 8
  • (d) 6.4
Answer and explanation

(c) 8. \( S_{YX} = 10\sqrt{1 - 0.36} = 10 \times 0.8 = 8 \).

Q5. (Practice) Consider the following statements:

  1. The two regression lines intersect at the point of means.
  2. If r = 0, the two regression lines coincide.
  3. Regression coefficients are independent of change of origin.

Which of the statements given above is/are correct?

  • (a) 1 and 2 only
  • (b) 1 and 3 only
  • (c) 2 and 3 only
  • (d) 1, 2 and 3
Answer and explanation

(b) 1 and 3 only. If r = 0 the lines are perpendicular; they coincide when r = ±1.

Level 3: Assertion–reason, match and sequence

For assertion–reason questions, use these options:

  • (a) Both A and R are true, and R is the correct explanation of A
  • (b) Both A and R are true, but R is not the correct explanation of A
  • (c) A is true, but R is false
  • (d) A is false, but R is true

Q1. (Practice) Assertion (A): If one regression coefficient is greater than one, the other must be less than one. Reason (R): The product of the two regression coefficients equals \( r^2 \), which cannot exceed one.

Answer and explanation

(a). R is exactly the reason: \( b_{YX} b_{XY} = r^2 \le 1 \).

Q2. (Practice) Assertion (A): A correlation coefficient of zero shows that the two variables are unrelated. Reason (R): Karl Pearson’s r measures only the linear relationship between two variables.

Answer and explanation

(d). A is false: r = 0 rules out only a linear relationship. R is true, and it is the reason A is false.

Q3. (Practice) Match List I with List II.

List I (Measure)List II (Formula or feature)
A. Coefficient of determinationI. \( 0.6745 (1 - r^2)/\sqrt{n} \)
B. Probable error of rII. \( r^2 \)
C. Standard error of estimateIII. \( 1 - 6\sum D^2 / n(n^2 - 1) \)
D. Rank correlationIV. \( \sigma_Y\sqrt{1 - r^2} \)
  • (1) A-II, B-I, C-IV, D-III
  • (2) A-I, B-II, C-IV, D-III
  • (3) A-II, B-IV, C-I, D-III
  • (4) A-IV, B-I, C-II, D-III
Answer and explanation

(1) A-II, B-I, C-IV, D-III.

Q4. (Practice) Match the contributor with the contribution.

List IList II
A. Francis GaltonI. Rank correlation coefficient
B. Charles SpearmanII. Coined “regression”
C. Karl PearsonIII. Product-moment correlation coefficient
  • (1) A-II, B-I, C-III
  • (2) A-I, B-II, C-III
  • (3) A-III, B-I, C-II
  • (4) A-II, B-III, C-I
Answer and explanation

(1) A-II, B-I, C-III.

Q5. (Practice) Arrange the steps for finding r from two given regression lines in the correct order:

  • A. Take the square root of the product of the coefficients, with their common sign

  • B. Assume which line is Y on X and write both coefficients

  • C. Check that the product of the coefficients does not exceed 1, swapping the assumption if it does

  • D. Solve the two equations simultaneously for the means

  • (1) D, B, C, A

  • (2) B, D, A, C

  • (3) D, C, B, A

  • (4) B, C, D, A

Answer and explanation

(1) D, B, C, A. The means can be found first (they do not depend on which line is which); the coefficients need an assumption, a check, and then the square root.


Quick Revision Table

ItemFormula or fact
Pearson’s r\( \text{Cov}(X, Y)/(\sigma_X\sigma_Y) \); −1 to +1; free of origin and scale
r² / 1 − r²Determination / non-determination
Probable error\( 0.6745(1 - r^2)/\sqrt{n} \); significant if r > 6 PE
Spearman (1904)\( 1 - 6\sum D^2/n(n^2 - 1) \); ties add \( m(m^2 - 1)/12 \) per group
Concurrent deviation\( \pm\sqrt{\pm(2c - n)/n} \)
Galton (1886)“Regression towards mediocrity”
\( b_{YX} \) / \( b_{XY} \)\( r\sigma_Y/\sigma_X \) / \( r\sigma_X/\sigma_Y \)
Link\( r = \pm\sqrt{b_{YX}b_{XY}} \); product ≤ 1; same signs
Origin and scaleb free of origin only
LinesMeet at \( (\bar{X}, \bar{Y}) \); coincide if r = ±1; perpendicular if r = 0
Least squares\( \sum Y = na + b\sum X \); \( \sum XY = a\sum X + b\sum X^2 \)
SE of estimate\( \sigma_Y\sqrt{1 - r^2} \)

Key Takeaways

What’s Next