AP Statistics · Exam guide
AP Statistics Sampling and Experiments: Bias, Random Samples, and Good Design
Questions about collecting data ask what a study can and cannot show. The answer depends on two things: how the subjects were chosen, which decides who the results apply to, and whether treatments were randomly assigned, which decides whether the results show cause and effect. This guide covers both.
Sampling: how subjects are chosen
| Method | How it works | A weakness to watch for |
|---|---|---|
| Simple random sample | Every group of the chosen size has an equal chance. | Needs a complete list of the population. |
| Stratified random sample | Split the population into groups, then take a random sample from each. | The groups must be chosen so they are similar inside. |
| Cluster sample | Randomly choose whole groups and survey everyone in them. | Less accurate if the groups are not alike. |
| Systematic sample | Choose every kth person after a random start. | A hidden pattern in the list can bias it. |
| Convenience or voluntary response | Whoever is easy to reach, or who chooses to respond. | Almost always biased. |
Kinds of bias
- Undercoverage: some part of the population has no chance to be chosen.
- Nonresponse: chosen people do not answer, and they differ from those who do.
- Response bias: the wording or setting pushes people toward an answer.
- Voluntary response: people with strong opinions are more likely to reply.
Observational studies and experiments
In an observational study, the researchers measure but do not assign the treatment. It can show association but not cause and effect, because of confounding. In an experiment, researchers randomly assign the treatments, which balances other variables across groups and allows a conclusion about cause and effect.
What a good experiment includes
| Feature | Why it matters |
|---|---|
| Random assignment | Makes the groups comparable and balances unknown variables. |
| A control or comparison group | Gives something to compare the treatment against. |
| Replication | Enough subjects that chance variation is small. |
| Blinding | Subjects or evaluators do not know who got which treatment, which avoids expectation effects. |
| Blocking | Group similar subjects first, then randomize within each block, to remove the effect of a known variable. |
A weak and a strong answer
A school wants to know whether a new reading program improves scores. A teacher gives the program to a class that volunteered and compares its scores with another class. What can the school conclude?
It claims cause and effect from a study without random assignment.
It names the problem, explains the confounding in context, and suggests a fix.
Common mistakes
- Concluding cause and effect from an observational study.
- Describing random assignment as random sampling. They answer different questions.
- Saying a study is biased without saying how and in which direction.
- Forgetting a control group.
Frequently asked questions
What is the difference between random sampling and random assignment?
Random sampling decides who is in the study, and allows results to apply to a population. Random assignment decides who gets which treatment, and allows conclusions about cause and effect.
Can an experiment use volunteers?
Yes, but the results then apply only to people like the volunteers. Random assignment among them still supports cause and effect.
What is a confounding variable?
A variable that is related to both the groups and the outcome, so its effect cannot be separated from the treatment.
What is blocking?
Grouping similar subjects before randomizing, so differences among the groups do not hide the treatment effect.
Where can I find the official scoring rules?
Exam formats change a little from year to year, so check the current course and exam description on the College Board site before test day.
Keep practicing
Every lesson in the AP Statistics course ends with practice problems and an Exam focus box that shows how the topic is tested, and every unit has a 10-problem practice set with worked answers. These free lessons also build the skills the free-response section tests:
See the AP Statistics course Free lessons and lesson list
More AP Statistics guides (8)
Start here
Free-response and writing
- AP Statistics Inference for Proportions Explained: Intervals and Tests with a Worked Example
- AP Statistics Inference for Means Explained: The One-Sample t Test and Checking Conditions
- AP Statistics Chi-Square Test Explained: Homogeneity and Independence with a Worked Example
- AP Statistics Regression Questions Explained: Interpreting Slope, r-Squared and Residuals
- AP Statistics Describing Data: Shape, Center, Spread, and Outliers in Context
- AP Statistics Probability and Random Variables: Expected Value, Binomial, and Normal
Multiple choice
AP is a registered trademark of the College Board, which is not affiliated with and does not endorse this site. This guide is our own explanation of the question types and scoring. For the official scoring guidelines, see the College Board.