Comparison of two proportions
The
material covered in the
probability and
confidence interval pages should be understood or at least be familiar to
you before proceeding.
A
Randomised Controlled Trial (RCT) can be considered the most transparent and
effective research model to compare the outcomes of two or more interventions.
The design is randomised because suitable research subjects recruited into the
trial are randomly allocated to interventions (usually according to a series of
computer generated random numbers). Consequently, subjects in each group should
not be different other than the interventions allocated. Therefore, the
difference between groups will reflect the difference between interventions.
There are other outcome measures and model variations, but this page will only
consider differences in proportions (binary outcome) between two groups. Four
commonly used algorithms will be covered along with risk difference and Numbers
Needed to Treat.
The
following nomenclature is commonly used when comparing binary outcomes in two
groups.
The
2 by 2 table:
The two
columns represent the two groups and the two rows represent the outcome (e.g.,
positive or negative).
Group 1 (first column) is usually
assigned as the intervention group and group 2 (second column) the control
group. Row 1 contains the outcome of interest and row 2, the absence of that
outcome. Cells are labelled either as A, B, C, D, or p1, p2,
n1, n2 (e.g., p1 = positive outcome for group
1) and represent the number of events. Differences between groups are usually
calculated as group 1 - group 2, and ratios as group 1 / group 2.
|
|
Group 1 |
Group 2 |
|
Outcome positive |
A (p1) |
B (p2) |
|
Outcome negative |
C (n1) |
D (n2) |
Proportion, incidence, prevalence and risk: All these terms refer to the same
mathematical concept; the proportion of outcome positives in a group. The
notation is a number between 0 and 1, so that 25% is presented as 0.25.
Proportion is the generic mathematical term, and for group 1 is p1 /
(p1 + n1), and for group 2 is p2 / (p2
+ n2). Prevalence is used if the proportion referred to that
exists in a population. Incidence is used in epidemiology if the disease or
condition is new in a previously disease free population. Risk is a concept of
probability and the term is used if the proportion represents the probability of
having outcome positive. Risk is the term usually used in statistical
calculation.
Odds: The ratio of numbers with outcome
positive and outcome negative. The odds in group 1 are p1 / n1,
and group 2 p2 / n2.
Risk
difference and its 95% confidence interval together is the standard statistical
model for comparing two proportions (risks) in a randomised controlled trial.
Another expression of risk difference is the numbers needed to treat, which is
the inverse of the risk difference, rounded to the nearest whole number, NNT =
round(1 / RD). This represents the additional number of people needed to be
given the intervention to affect an additional case with a different outcome.
This indicator is intuitively easier to understand by clinicians, so is often
presented. However, it provides no additional information.
Formulae for Risk Difference and Numbers Needed to Treat
Risk
of group 1, R1 = p1 / (p1+ n1)
Risk of group 2, R2 = p2 / (p2+ n2)
Risk Difference, RD = R1 - R2
Standard Error of RD, SE = sqrt(R1(1 - R1) / (p1
+ n1) + R2(1 - R2) / (p2 +
n2))
95% Confidence Interval = RD
±
1.96 x SE
Numbers Needed to Treat, NNT = 1 / RD
Example of calculation for Risk Difference and Numbers Needed to Treat
A
randomised controlled trial was conducted to see if giving someone money makes
them happy. We randomly allocated all research subjects into two groups. In
Group 1, each subject was given a bundle of banknotes and asked if he/she was
happy or unhappy. In group 2, a bundle of cut up newspapers was given to each
subject. Fifteen people were given banknotes; 12 were happy and 3 unhappy.
Thirteen people were given newspapers; 4 were happy and 9 unhappy. What is
the difference in risk of being happy in the two groups, and how many people
are required to give money to in order to produce an extra happy person.
Answers
The data can be represented by the following table
|
|
Given money |
Given paper |
|
Happy |
12 |
4 |
|
Not happy |
3 |
9 |
The risk of being happy after receiving money is 12/15 = 0.80 (80%)
The risk of being happy after receiving newspaper is 4/13 = 0.31 (31%)
The risk difference (RD) is 0.80 - 0.31 = 0.49 (49%)
The Standard Error of the difference (SE) = sqrt(0.80(0.20)/15 +
0.31(0.69)/13) = sqrt(0.0271) = 0.16
The 95% Confidence Interval is 0.49
±
1.96 x 0.16 = 0.17 to 0.81
Numbers needed to treat (NNT) = 1 / RD = 1 / 0.49 = 2
The 95% CI for NNT = 1/0.17 to 1/0.81 = 5.9 to 1.2.
The interpretation is that the difference in risk between the two groups
is 17% to 81%, and two additional people (1 to 6) need to be given money to make
one additional person happy.
A
program to calculate risk difference and NNT is available in the
Programs and Exercises section.
Relative Risk is sometimes also called Risk Ratio and is the ratio of the risks
in two groups. Relative Risks were initially developed in epidemiological
models to examine the effects of exposure to an influence on an outcome, but
results of controlled trials are sometimes also presented as Relative Risks.
A
particular advantage of Relative Risk is when a number of influences and
outcomes that have different prevalence are compared. An example of this could
be a study on the effect of introducing a new screening test on both the
detection and death rate of a cancer. If the new test detected 80% of cases
compared with 40% by the old test, the risk difference is 40%, and the Relative
Risk is 2. The death rate when using the new test is 5% instead of 10% using
the old test. Here, the risk difference is 5%, but the Relative Risk is 0.5 or
1/2. In other words, using Relative Risks provides a logical conclusion that
the new test is twice as good in detecting cancer and halved the death rate.
Using risk difference, the difference is 40% in detection, and 5% in death rate,
and the two cannot be compared.
Relative Risk is a ratio and as such, has an exponential distribution. The
variance of Relative Risk has shown to increase with the ratio. Therefore, the
logarithm of the ratio is used in statistical calculations and results are
converted back to the natural scale at the end of calculations. As a result,
the mean value does not lie in the middle of the 95% confidence interval.
Formulae for Relative Risks
Risk
of group 1 (R1) = p1 / (p1+ n1)
Risk of group 2 (R2) = p2 / (p2+ n2)
Log Relative Risk (LRR) = log(R1 / R2)
Standard Error (SE) of Log Relative Risk = sqrt(1/p1 - 1/(p1
+ n1) + 1/p2 - 1/(p2 + n2))
95% Confidence Interval of LRR = LRR
±
1.96 x SE
95% Confidence Interval of RR = exp(LRR
±
1.96 x SE)
Example of Relative Risks Calculation
A
study was conducted to see if rich people are happier than poor people. A
random sample was surveyed which included 15 rich people and 13 poor people. Of
these, 12 rich people were happy and 3 unhappy while 4 poor people were happy
and 9 unhappy. From this, the 95% confidence interval for the Relative Risk of
happiness between rich and poor people can be calculated.
Answers
The data can be represented by the following table
|
|
Rich |
Poor |
|
Happy |
12 |
4 |
|
Not happy |
3 |
9 |
The risk of being happy amongst rich people is 12/15 = 0.80 (80%)
The risk of being happy amongst poor people is 4/13 = 0.31 (31%).
The Relative Risk (RR) = 0.80 / 0.31 = 2.6
Log Relative Risk (LRR) = log(2.6) = 0.96
The Standard Error (SE) of LRR = sqrt(1/12 - 1/15 + 1/4 - 1/13) =
sqrt(0.1897) = 0.44
The 95% Confidence Interval of LRR is 0.96
±
1.96 x 0.44 = 0.10 to 1.82
The 95% Confidence Interval of RR is exp(0.10) to exp(1.82) = 1.1 to 6.1
Rich people are 2.6 times more likely to be happy than poor people with a
95% Confidence Interval of 1.1 to 6.1
A
program to calculate Relative Risks is available in the
Programs and Exercises section
Odds
are different to risk in that it is the ratio between numbers with an
index attribute to numbers without. In a two group comparison, the odds
for group 1 is O1 = p1/n1, and for group 2 is O2
= p2/n2. The Odds Ratio (OR) = O1 / O2.
Unlike risks, odds do not presume a causal sequence, so they represent an
association or correlation between two binary variables. This advantage allows
it to be used in a multivariate model such as logistic regression or in
retrospective control studies where case selections are based not on causal
factors but on outcomes.
Odds
Ratio is a ratio and as such, has an exponential distribution. The variance of
the odds ratio increases as the ratio increase. Therefore, the logarithm
of the ratio is used in statistical calculations and the results are converted
back to the natural scale at the end of calculations. As a result, the mean
value does not lie in the middle of the 95% confidence interval.
Formulae for Odds Ratio
Odds of group 1 (O1) = p1 / n1
Odds of group 2 (O2) = p2 / n2
Log Odds Ratio (LOR) = Log(O1 / O2)
Standard Error (SE) of LOR = sqrt(1/p1 + 1/n1 + 1/p2
+ 1/n2)
95% Confidence Interval of LOR = LOR ± 1.96 x SE
95% Confidence Interval of OR = exp(LOR ± 1.96 x SE)
Example of calculation for Odds Ratio
A
study was conducted to see whether winning a prize in the lottery makes people
smile. People coming out of lottery offices were observed. Whether they smiled
or not was recorded. The people were then asked if they had just won the
lottery. We found 16 people smiled; 12 of whom just won a prize and 4 did not.
We also found 12 people not smiling; 3 of them just won a prize and 9 did not.
What is the odds ratio that associates smiling with winning a prize and what is
the 95% confidence interval?
Answers
The data can be represented by the following table
|
|
Smiling |
Not smiling |
|
Won prize |
12 |
3 |
|
No prize |
4 |
9 |
The odds of having just won a prize amongst smilers are 12/4 = 3
The odds of having just won a prize by the non-smilers are 3/9 = 0.33
The Odds Ratio (OR) = 3 / 0.33 = 9.0
The Log Odds Ratio (LOR) = log(9) = 2.2
The Standard Error of LOR (SE) = sqrt(1/12 + 1/4 + 1/3 + 1/9) =
sqrt(0.7778) = 0.88
The 95% Confidence Interval of LOR is 2.2 ± 1.96 x 0.88 = 0.47 to 3.93
The 95% Confidence Interval of OR is exp(0.47) to exp(3.93) = 1.6 to 50.7
The odds of having just won a prize amongst smilers are 9 time that of
non smilers with a 95% confidence interval of 1.6 to 50.7
A
program to calculate Odds Ratio is available in the
Programs and Exercises section
The
conversion of risk differences, risk ratios and odds ratios into normally distributed
measurements is based on the theory of large samples, as the binomial
distribution is approximated by the normal distribution when the number of
measurements is large. When numbers are small, the assumptions of the normal
distribution becomes increasingly unrealistic and if any of the cells (p1,p2,n1
or n2) has no case in it then risk and odds ratios cannot be calculated at all.
When this happens, an alternative calculation based on the binomial
distribution is used. Fisher's exact test (two-sided) tests the probability
that the two proportions are the same. The smaller the probability (i.e.,
p-value), the less likely they are the same.
The
formula for Fisher's exact probability will not be described here. For those
interested in developing the calculations, an algorithm can be found in the text
book, Practical Statistics for Medical Research, (1994) F.Altman, Chapman Hall,
London, ISBN 0 412 276205 p.233
A
program to calculate Fisher's exact probability is available in the
Programs and Exercises section.
Program to calculate differences between two proportions
Exercises for differences between two proportions
We
have conducted a multi-centre study to see whether regular exercise makes
people slimmer. The data obtained is shown in the table below.
Ex. 1.
Calculate the
95% confidence intervals for the Risk Differences, Relative Risks, and the
Odds Ratios for these sets of data and produce Forest plots to represent the
results.
Ex. 2.
Calculate the probability that the two groups are not different.
|
|
Exercise
(Grp1) |
Couch potato
(Grp2) |
|
|
Slim |
Obese |
Slim |
Obese |
|
Centre A |
10 |
8 |
9 |
12 |
|
Centre B |
12 |
9 |
5 |
7 |
|
Centre C |
6 |
3 |
8 |
15 |
|
Centre D |
5 |
5 |
11 |
12 |
|
Centre E |
15 |
8 |
10 |
9 |
|
Centre F |
13 |
9 |
3 |
8 |
|
Centre G |
12 |
8 |
9 |
16 |
|
Total |
73 |
50 |
55 |
79 |
Answers
Ex. 1. The results of the study on exercise and slimming are as follows. RD = Risk
Difference, NNT = Numbers needed to treat, RR = Relative Risks, OR = Odds Ratio.
|
Exercise
(Grp1) |
Couch potato(Grp2) |
Results |
|
Slim |
Obese |
Slim |
Obese |
Risk1 |
Risk2 |
RD
(95% CI) |
NNT |
RR
(95% CI) |
OR
(95% CI) |
| Centre A |
10 |
8 |
9 |
12 |
0.56 |
0.43 |
0.13
(-0.18 to 0.44) |
8 |
1.30
(0.68 to 2.47) |
1.67
(0.47 to 5.93) |
| Centre B |
12 |
9 |
5 |
7 |
0.57 |
0.42 |
0.15
(-0.18 to 0.49) |
7 |
1.37
(0.64 to 2.95) |
1.87
(0.44 to 7.85) |
| Centre C |
6 |
3 |
8 |
15 |
0.67 |
0.35 |
0.32
(-0.01 to 0.65) |
4 |
1.92
(0.93 to 3.96) |
3.75
(0.73 to 19.14) |
| Centre D |
5 |
5 |
11 |
12 |
0.50 |
0.48 |
0.02
(-0.32 to 0.36) |
47 |
1.05
(0.49 to 2.22) |
1.09
(0.25 to 4.82) |
| Centre E |
15 |
8 |
10 |
9 |
0.65 |
0.53 |
0.13
(-0.18 to 0.43) |
8 |
1.24
(0.74 to 2.09) |
1.69
(0.49 to 5.85) |
| Centre F |
13 |
9 |
3 |
8 |
0.59 |
0.27 |
0.32
(0.00 to 0.64) |
4 |
2.17
(0.78 to 6.04) |
3.85
(0.80 to 18.62) |
| Centre G |
12 |
8 |
9 |
16 |
0.60 |
0.36 |
0.24
(-0.04 to 0.52) |
5 |
1.67
(0.88 to 3.14) |
2.67
(0.79 to 8.95) |
| Total |
73 |
50 |
55 |
79 |
0.59 |
0.41 |
0.18
(0.06 to 0.30) |
6 |
1.45
(1.13 to 1.86) |
2.10
(1.27 to 3.45) |
The Forest plots show that those who exercise are generally slimmer. However,
the sample size in the individual centres was not large enough to conclude that
this was a significant observation. When the data from all centres are
combined, the sample size is increased and the difference between those who
exercise and those who do not becomes significant.
The
Forest plots also show that although the patterns are very similar using the 3
methods, they are numerically different as they represents different sorts of
comparisons. Note that Relative Risks and Odds Ratio are ratios so they are
only normally distributed in the logarithmic scale.
Ex. 2. Fishers Exact Probability are 0.48,
0.13, 1.00, 0.53, 0.14, 0.14 respectively, and 0.004 for the combined results.
Practical Statistics for medical Research. (1994) Altman D. Chapman Hall,
London. ISBN 0 412 276205 (First Ed. 1991)
Statistics with confidence. Second Edition. (2002) Altman DG, Machin D, Bryant
TN and Gardiner MJ. BMJ Books ISBN 0 7279 1375 1
Version 1.1 Last change 10th July 2006