Loading…
Identify sources of bias in sampling and surveys including voluntary response and convenience sampling.
Learn step-by-step with practice exercises built right in.
Undercoverage
Nonresponse Bias
Voluntary Response Bias
Wording Effects
A survey asking 'Don't you agree that more funding should go to schools?' is an example of what type of bias? Explain what response bias is and how to fix it.
This is an example of response bias (specifically leading question bias or wording bias).
What response bias means: Response bias occurs when question wording, question order, or surveyor behavior influences responses, causing answers to differ from true opinions.
Why this question is biased: The phrasing 'Don't you agree...' is leading — it suggests the desired answer is 'yes.' Most respondents will comply rather than disagree with a leading question, inflating support for school funding.
How to fix it:
Avoid these 3 frequent errors
Review key concepts with our flashcard system
Explore more AP Statistics topics
Interviewer Effects
Question Order Effects
Timing and Setting
| Bias | Reduction |
|---|---|
| Undercoverage | Use comprehensive sampling frame; multistage or stratified to reach hard-to-reach groups |
| Nonresponse | Multiple contact attempts, incentives, follow-up with non-respondents |
| Voluntary response | Use random sampling instead |
| Wording | Pilot-test questions; use neutral language; avoid leading questions |
| Interviewer | Train interviewers; use telephone/email; randomize interviewer assignment |
| Social desirability | Ensure anonymity; use indirect questions (third-party phrasing) |
Is bias fixed (doesn't decrease with larger )? → Reduce via improved design Is variability decreasing with ? → Increase sample size
When critiquing a survey, identify: potential source of bias (which type), explain mechanism, and propose mitigation strategy. Larger samples fix variability, not bias.
Reword as neutral and balanced alternatives:
Key principle: Survey questions must be neutral and not suggest a particular answer. Both sides of an issue should be presented equally.
Other response bias types: Acquiescence bias (always agreeing), social desirability bias (answering what sounds good, not true), and question order effects.
An online poll asks: 'Have you voted in the last election?' Those who don't respond are removed from the survey. What biases might this introduce?
Biases introduced:
1. Nonresponse bias:
People who don't respond differ from respondents. Who doesn't respond?
Result: Sample overrepresents voters and politically engaged people; underrepresents non-voters.
2. Voluntary response bias (self-selection):
Online polls are voluntary. Who volunteers?
Result: Response represents passionate people, not average population.
3. Undercoverage (selection bias):
Only people with internet access can take online survey. Missing:
These groups have different voting patterns than internet users.
4. If the question assumes voting:
'Have you voted...?' assumes respondents are voters. People who think they shouldn't answer (non-voters, non-citizens) might skip the survey, creating further nonresponse bias.
Consequences:
Estimated voter turnout would be inflated — the poll would overestimate how many people actually voted.
How to reduce bias:
Compare three sources of bias in surveys: sampling error, undercoverage, and non-response. Give an example showing how all three could occur in one survey and explain which are most problematic for decision-making.
Sampling Error:
What it is: Natural variability in sample statistics due to random sampling. Even with a perfectly unbiased sample, different samples give different results.
Example: Two SRS of 500 voters each might give 52% and 48% support for a candidate just by chance.
Is it a bias?: NO — it's unavoidable randomness, not systematic error. Reduced by larger sample sizes.
Undercoverage:
What it is: Some population groups are systematically excluded or harder to reach.
Example: Phone survey doesn't reach people without phones (young, mobile-only) or language minorities.
Result: Sample doesn't reflect population structure; biased estimates.
Is it a bias?: YES — systematic, not random.
Non-response:
What it is: People who don't respond differ systematically from those who do.
Example: Survey of job satisfaction has 30% response rate. Non-responders might be very satisfied (don't bother responding) or very dissatisfied (don't want to engage).
Result: Biased estimate depending on who non-responders are.
Is it a bias?: YES — systematic, not random.
Example: All three in one survey
A company conducts phone survey about workplace satisfaction:
Result: Satisfaction estimate is biased AND has random variability.
Which biases are most problematic?
Ranking by severity:
Why?: Sampling error is predictable and decreases with n. Biases are systematic distortions we can't easily quantify or correct without knowing the truth.
Lesson for decision-making: Always ask: "Who is missing?" (undercoverage) and "Who didn't respond?" (non-response). These are far more problematic than sampling error.