StatisticsInferential Statistics

Opinion Poll Bias: The 1936 Literary Digest Sample

Explain why millions of responses did not make the 1936 opinion poll representative, and distinguish coverage, nonresponse, and quota sampling.

Question

In advance of the 1936 Presidential Election, a magazine titled Literary Digest released the results of an opinion poll predicting that the republican candidate Alf Landon would win by a large margin. The magazine sent post cards to approximately 10,000,000 prospective voters. These prospective voters were selected from the subscription list of the magazine, from automobile registration lists, from phone lists, and from club membership lists. Approximately 2,300,000 people returned the postcards.

a. Think about the state of the United States in 1936. Explain why a sample chosen from magazine subscription lists, automobile registration lists, phone books, and club membership lists was not representative of the population of the United States at that time.

b. What effect does the low response rate have on the reliability of the sample?

c. Are these problems examples of sampling error or nonsampling error?

d. During the same year, George Gallup conducted his own poll of 30,000 prospective voters. His researchers used a method they called "quota sampling" to obtain survey answers from specific subsets of the population. Quota sampling is an example of which sampling method described in this module?

a. Why the opinion poll's sampling frame was unrepresentative

Magazine subscribers, automobile owners, telephone owners, and club members were not a representative cross-section of all prospective voters during the Great Depression. Those lists favored people with the resources to buy subscriptions, cars, phones, or memberships and missed many poorer voters. If voting preferences differed between covered and uncovered groups, the poll could systematically misestimate support. Its millions of respondents did not remove that coverage bias.

b. The low return rate and reliability

Using the approximate counts supplied in the exercise:

return rate=2,300,00010,000,000×100%≈23%.\text{return rate}=\frac{2{,}300{,}000}{10{,}000{,}000}\times100\%\approx23\%.

About 77% did not return the card. That creates a substantial opportunity for nonresponse bias if respondents and nonrespondents differ in voting preference. A low response rate alone does not prove how large the bias is or even its direction. The problem provides no preferences for the nonrespondents, so neither their vote shares nor an exact bias can be calculated. These approximate historical counts are a simple returned-to-mailed ratio, not a modern standardized response-rate calculation.

c. Sampling error or nonsampling error?

The defective coverage and potentially selective nonresponse are nonsampling errors. Sampling error is random variation arising because a sample, rather than the entire population, is observed. Taking more responses from a systematically unrepresentative frame can reduce random fluctuation while preserving bias.

d. How should quota sampling be classified?

Within the module's list of sampling methods, the intended comparison is stratified sampling: both seek responses from specified subgroups. That similarity has an essential limit. Ordinary quota sampling fills subgroup targets without requiring random selection within each subgroup. Proper stratified random sampling selects randomly within strata with a probability-based design. Therefore quota sampling is generally a nonprobability method, not automatically stratified random sampling. The prompt does not describe Gallup's within-quota selection in enough detail to establish random selection.

Original exercise attribution

Adapted in layout from Barbara Illowsky and Susan Dean, Introductory Statistics (OpenStax College, 2013), CC BY 4.0. Download for free at http://cnx.org/content/col11562/latest/. The exercise data and all subquestions are retained; the worked explanation and plotted dots are provided here.

Evidence boundary

This answer uses Homework 76 in the 2013 OpenStax PDF, including its named magazine, approximate counts, and all four parts. The 23% value is an approximate postcard-return ratio. Respondents’ and nonrespondents’ preferences are not supplied, so exact bias is unknown. Quota sampling resembles subgroup stratification but does not guarantee random selection.

Sources

Các nguồn này hỗ trợ những khái niệm và phương pháp được giải thích ở trên.