Lesson 1.1 · Unit 1 · Science practice skills 2.A to 2.C

Experiments and non-experimental designs

Psychology is the scientific study of behavior and mental processes, and what makes it a science is not the topic but the method. Every claim in this course, from how neurons fire to why people obey, rests on a study somebody designed, and the AP Exam keeps asking you to look at a described study and say what kind it was, what was measured, and what the results can and cannot show. Both free-response questions hand you summarized research to read, and about a quarter of the multiple-choice questions test research methods directly. So this course starts with the toolkit. The next three lessons cover sampling, ethics, and statistics; after that every lesson introduces a topic through the studies behind it, and you will use these ideas on every page.

Required skills · 2.A, 2.B, 2.C

Science practice skills.

2.A. Determine the type of research design(s) used in a given study.
2.B. Evaluate the appropriate use of research design elements in experimental methodology.
2.C. Evaluate the appropriate use of research design elements in non-experimental methodologies.

What the exam asks you to do.

2.A: Determine whether a study is using experimental or non-experimental methodologies.
2.A: Experimental methodology involves the use of independent variable(s) and random assignment to groups.
2.A: Non-experimental methodologies include case study, correlation, meta-analysis, and naturalistic observation.
2.B: State the hypothesis of a research scenario, including whether the hypothesis is falsifiable.
2.B: Identify operational definitions of variables in research scenarios to determine how well the definitions allow for the study to be replicated.
2.B: Identify the independent variable(s), dependent variable(s), and/or confounding variable(s) in a research scenario.
2.B: Identify the experimental and control groups, including the process and purpose for assigning people to groups, whether a placebo was used with the control group, and the effect of the placebo on the results.
2.B: Determine whether the study used single-blind or double-blind procedures to control for confounding variables like experimenter bias or the social desirability bias.
2.C: Identify the variables of interest in a research scenario.
2.C: Evaluate whether the results of a correlational study have a directionality problem or third variable problem that demonstrates that correlation does not equal causation.
2.C: Determine whether the research used a survey technique and whether the wording of the survey could lead respondents to demonstrate self-report bias or social desirability bias.

From a question to a testable hypothesis

A research project begins with a question, such as whether sleep affects memory. A theory is an organized set of principles that explains observations and predicts new ones. From a theory a researcher derives a hypothesis, a specific, testable prediction: "Students who sleep eight hours after studying will recall more words the next day than students who stay awake." Notice what a good hypothesis does. It names the variables and says how they should be related, and it can be shown wrong.

That last property is called falsifiability. A hypothesis is falsifiable if some possible result would count against it. "Sleeping students will recall more than awake students" is falsifiable because the awake students might do as well or better. "Some people are destined to be lucky" is not, because any outcome can be explained after the fact. The exam asks you to state a hypothesis from a scenario and to say whether it is falsifiable, so practice turning vague claims into predictions that could fail.

A variable is anything that can take different values or levels across people or situations: hours of sleep, number of words recalled, anxiety score. To test a hypothesis researchers must give each variable an operational definition, a precise statement of how the variable is measured or manipulated in this particular study. "Memory" is a construct; "the number of words from a 20-word list recalled correctly after 24 hours" is an operational definition. A good operational definition is specific enough that another researcher could repeat the procedure exactly. That is replication, repeating a study (often with different participants) to see whether the finding holds, and it is the reason clear operational definitions matter.

Experiments: the one design that tests cause and effect

An experiment is the research method that can test whether one variable causes a change in another. The exam describes experimental methodology as the use of an independent variable and random assignment to groups. The pieces are:

The independent variable (IV) is the variable the researcher manipulates on purpose. It is the proposed cause.
The dependent variable (DV) is the variable that is measured to see whether it changed. It is the proposed effect, and it depends on the IV.
The experimental group receives the treatment (a level of the IV). The control group does not, or receives a standard or neutral condition, and provides a baseline for comparison.
Random assignment means every participant has an equal chance of landing in any condition, decided by chance (a coin flip, a random number list). Its purpose is to spread pre-existing differences such as mood, intelligence, or motivation evenly across groups, so that the groups are equivalent before the treatment.
A confounding variable is any factor other than the IV that differs between the groups and could explain a difference in the DV. Random assignment and keeping everything else the same (testing everyone in the same room at the same time of day, with the same instructions) are the defenses against confounds.

Random assignment is easy to confuse with random sampling. Random assignment decides which condition a participant is placed in, and it is what allows a cause-and-effect conclusion. Random sampling decides who is chosen from the population to be in the study at all, and it affects how far the results can be generalized. The next lesson returns to sampling.

Controlling expectations: placebos and blinding

People's beliefs change their behavior. In a drug trial, some participants feel better simply because they expect to, a result called the placebo effect. To separate the effect of the drug from the effect of expectation, researchers give the control group a placebo, an inactive substance or sham treatment that looks the same as the real one. If the drug group improves more than the placebo group, the extra improvement can be attributed to the drug.

Researchers have expectations too, and these can leak into the results through the way they treat participants or score the data, which is called experimenter bias. Participants may also try to look good or give the answers they think are wanted, called social desirability bias. Two procedures guard against these problems:

In a single-blind procedure the participants do not know which condition they are in.
In a double-blind procedure neither the participants nor the researchers who interact with them or score the results know who is in which group.

If a scenario says "the nurse giving the pills did not know which were real," that is a clue to double-blind. If only the patients were kept uninformed, it is single-blind.

Study spotlight · Dunn, Aknin, and Norton (2008)

A team of researchers wanted to know whether spending money on other people makes us happier than spending it on ourselves. In one experiment, university students were given an envelope in the morning containing either $5 or $20 and were randomly assigned to spend the money by 5 p.m. either on themselves or on someone else. That evening the participants rated their happiness. The students told to spend on others reported greater happiness than those told to spend on themselves, and the amount of money ($5 or $20) did not matter.

Reading the research: the IV had two parts (who the money was spent on, and how much money). The DV was a self-reported happiness rating, which is the operational definition of happiness here. Random assignment made the groups equivalent at the start, so the researchers could conclude that spending on others caused the higher happiness. The sample was small and made up of university students, which limits generalizing, and a self-report measure can be affected by social desirability.

Non-experimental methods

When an experiment is impractical or unethical (you cannot randomly assign children to be abused or to lose a brain region), researchers use non-experimental designs. These methods can describe and predict behavior, and some of them are the source of famous hypotheses, but because they lack a manipulated IV and random assignment they cannot show that one variable causes another.

A naturalistic observation records behavior as it happens in its natural setting without interfering. Jane Goodall's long study of wild chimpanzees in Tanzania, in which she observed tool use, is the standard example. The strength is realistic behavior; the weaknesses are that observers can be biased and that the situation cannot be controlled.
A case study is an in-depth investigation of one person or a small group, often someone with an unusual condition. Case studies generate hypotheses and document rare events, but one person may not be typical, so the results may not generalize.
A survey collects self-reported answers from many people. Surveys are fast and reach large samples, but they are affected by self-report bias (people may not remember accurately or may not be honest), by social desirability bias, and by wording effects: leading or loaded questions steer answers. Compare "Do you support helping the poor?" with "Do you support giving tax money to people who don't work?"
A correlational study measures two or more variables as they naturally occur and determines whether and how strongly they are related. Correlations are reported with a coefficient that the statistics lesson explains.
A meta-analysis is a statistical combination of the results of many separate studies on the same question. It gives a more stable estimate than any single study and helps show whether a finding replicates.

Correlation does not equal causation

Because a correlational study does not manipulate anything, a relationship found in it can be explained in more than one way. Suppose a correlational study finds that students who eat breakfast have higher grades. There are three possible explanations:

Directionality problem: the causal arrow could run the other way. Perhaps students who are organized and doing well in school are the ones who eat breakfast.
Third variable problem: a hidden variable could cause both. Perhaps family income or a stable home routine produces both regular breakfasts and better grades.
Causation in the direction suggested: breakfast helps learning. This is possible but cannot be concluded from the correlation alone.

Only an experiment (randomly assigning students to eat or skip breakfast) can sort these out. When you read "linked to," "associated with," or "related to," think correlation; when you read "caused" or "led to" after random assignment, think experiment.

Choosing and recognizing a design

Method	Manipulates an IV?	Random assignment?	Can show cause and effect?	Typical weakness
Experiment	Yes	Yes	Yes	Artificial settings; some questions are unethical to test
Correlational study	No	No	No	Directionality and third-variable problems
Naturalistic observation	No	No	No	Observer bias; no control of conditions
Case study	No	No	No	One case may not generalize
Survey	No	No	No	Self-report bias, wording, social desirability
Meta-analysis	No (combines studies)	No	Only as strong as the studies included	Depends on the quality of the studies pooled

Worked example · Classifying a scenario

A researcher recruits 60 volunteers and flips a coin for each to decide whether they will spend 10 minutes writing about something they are grateful for or 10 minutes writing about their day. Afterward, everyone completes a 10-item mood questionnaire scored from 10 to 50. The researcher predicts that the gratitude group will score higher. Identify the design, IV, DV, and the role of the coin flip.

Step 1, look for random assignment and a manipulated variable. The researcher decides what each person writes about, and the coin flip assigns people to conditions. So this is an experiment.

Step 2, the IV is the type of writing (gratitude versus daily events). The gratitude group is the experimental group and the writing-about-the-day group is the control group.

Step 3, the DV is the mood score, and the operational definition of mood is the score on the 10-item questionnaire.

Step 4, the coin flip is random assignment, which makes the groups roughly equal in mood, personality, and other characteristics before the writing, so a difference in the scores can be attributed to the writing task.

Step 5, the hypothesis is falsifiable: the gratitude group might not score higher.

Worked example · Evaluating a correlational claim

A survey of 500 high school students finds that those who spend more hours on social media each day report lower self-esteem. A news headline says, "Social media is lowering teens' self-esteem." Evaluate the headline.

Step 1, name the design: a survey that measured two variables as they occur, so it is correlational. The researchers did not assign anyone to use more or less social media.

Step 2, directionality: students who already feel poorly about themselves might turn to social media more often, so low self-esteem could cause heavier use.

Step 3, third variable: loneliness or a stressful home life could both lower self-esteem and increase online time.

Step 4, add the survey limitation: both variables are self-reported, so they can be affected by inaccurate memory of hours and by social desirability.

Step 5, conclusion: the data support "heavier use is associated with lower self-esteem," not "social media lowers self-esteem." To test cause, a researcher would randomly assign students to reduce their use and measure self-esteem afterward.

Worked example · Spotting a confound

In a study of whether a new teaching method improves test scores, one class (taught by the researcher's favorite teacher) uses the new method and a second class uses the usual method. The new-method class scores 8 points higher. What went wrong?

Step 1, the IV is the teaching method, but the classes were not randomly assigned, and the teacher is a second difference between the groups.

Step 2, the teacher differs between conditions. It is a confounding variable, so the 8-point difference could come from the teacher, from differences between the students in the two classes, or from the method.

Step 3, a better design randomly assigns students to classes, or has the same teacher teach both methods, and uses the same test and conditions for both.

Exam focus

Experiments (manipulated IV + random assignment) are the only method that can show cause and effect; the DV is what is measured.
Name the design from the scenario: coin flip or random assignment means experiment; "measured two variables" means correlational; one person means case study; recording behavior in natural settings means naturalistic observation; self-reports from many people mean a survey; combining many studies means meta-analysis.
A placebo tests expectation; single-blind hides the condition from participants, double-blind hides it from participants and researchers.
For any correlation, give the directionality problem and the third-variable problem.
An operational definition is a measurement or procedure specific enough to replicate. A hypothesis must be falsifiable.

Practice

A researcher says, "People who are meant to be creative will always find a way to show it." Is this hypothesis falsifiable? Rewrite it as a falsifiable hypothesis.
Show answer
No, because any behavior (including no creative behavior) could be explained away as not "meant to be." A falsifiable version: "Students who receive 30 minutes of brainstorming training will list more unusual uses for a brick in 3 minutes than students who receive no training." This names the variables, operational definitions, and a result that could fail.
In a drug trial for headaches, patients receive either the drug or a sugar pill. The nurses who hand out the pills also do not know which is which. Name the placebo, the control group, and the blinding procedure.
Show answer
The sugar pill is the placebo and the patients who take it are the control group. Both the patients and the nurses are unaware of the assignment, so it is a double-blind procedure.
A study measures the number of hours teens sleep and their GPA in 300 students and finds r = +0.30. Give one directionality explanation and one third-variable explanation.
Show answer
Directionality: students who do well in school feel less stressed, so they sleep more (GPA affects sleep). Third variable: a stable, supportive home might both give teens time and a quiet place to sleep and help them study (home environment affects both).
A researcher watches how often children share toys on a playground without interfering and records each sharing act. What method is this and what is one weakness?
Show answer
Naturalistic observation. A weakness: the observer cannot control conditions or may be biased about what counts as sharing; the observed children may also behave differently if they know they are watched.
Why does random assignment, but not random sampling, allow the conclusion that the IV caused a change in the DV?
Show answer
Random assignment makes the groups equivalent at the start (spreading pre-existing differences by chance), so a later difference in the DV is likely due to the IV. Random sampling affects who is in the study and so how far the results generalize to the population, but it does not make the groups equal or control confounds.
A survey question reads, "Do you agree that wasteful government spending should be cut?" Identify the problem and its effect.
Show answer
The wording is loaded: it labels spending as "wasteful," so it steers respondents toward agreeing. This is a wording effect that biases the results (and it can combine with social desirability bias).