Teacher resources and professional development across the curriculum

Teacher professional development and classroom resources across the curriculum

Session 6, Part C:
Comparative Experimental Studies

In This Part: Measuring Short-Term Recall | An Experiment | Experimental Design
Analysis of the Experiment

You would not want to make conclusions about memory by examining only one person. Therefore, you should use more than one subject in this experiment.

Let's assume that you will use 16 subjects in your experiment. You will need to make some decisions about how to measure short-term recall in your 16 subjects.

Recall the original question: "Is it easier to remember strings of characters that spell familiar words than to remember arbitrary strings of characters?"

As stated, the question is perhaps not as specific as it should be. For example, we have not clarified the population of people we are studying. Age may have an impact on a person's ability to memorize. Are we interested in adults only?

If you believe that age makes a difference in a person's recall ability, then perhaps you need to refine the question.

Below are five different ways that you might assign the 16 subjects to groups and collect your measurements. These are referred to as designs. You should consider only one design at a time; do not move on to Design 2 until you have answered all four questions for Design 1, and so on. Your focus should be on the methods of data collection for each design.

Problem C4

For each design, answer the following questions:

 a. What are the strengths of the design? b. What are the weaknesses of the design? c. Do you have concerns about bias that are not addressed by the design? d. Do you have suggestions for improving the design?

Design 1

 • Divide the 16 subjects into two groups of eight. Ask for volunteers to form the first group. • When the two groups have been formed, measure each of the eight subjects in Group 1 using List A; measure each of the subjects in Group 2 using List B. • Compare the results (the eight measurements) for Group 1 with the results for Group 2.

Design 2

 • Measure each of the 16 subjects using List A. Then measure each of the 16 subjects using List B. • Compare the results (all 16 measurements) for List A with the results for List B.

Design 3

 • Divide the 16 subjects into two groups, and assign eight subjects to each group. • When the two groups have been formed, measure each of the eight subjects in Group 1 using List A; measure each of the subjects in Group 2 using List B. • Compare the results (the eight measurements) for Group 1 with the results for Group 2.

Design 4

 • Divide the 16 subjects into two groups of eight. Assign eight subjects to each group. • When the two groups have been formed, measure each of the eight subjects in Group 1 using List A. Then measure each of the subjects in Group 1 using List B. • Measure each of the subjects in Group 2 using List B. Then measure each of the subjects in Group 2 using List A. • Compare the results (all 16 measurements) for Group 1 with the results (all 16 measurements) for Group 2.

Design 5

 • Divide the 16 subjects into two groups of eight. Randomly assign eight subjects to each group. • When the two groups have been formed, measure each of the eight subjects in Group 1 using List A. Then measure each of the subjects in Group 1 using List B. • Measure each of the subjects in Group 2 using List B. Then measure each of the subjects in Group 2 using List A. • Compare the results (all 16 measurements) for Group 1 with the results (all 16 measurements) for Group 2.

 Problem C5 Of the five designs in Problem C4, which do you think does the best job of eliminating potential sources of bias? Does this design eliminate all potential sources of bias, or are there further possible improvements?

 Video Segment In this video segment, participants compare Designs 4 and 5. Watch this segment after completing Problem C5. Why do participants decide that Design 5 is better than Design 4? If you're using a VCR, you can find this segment on the session video approximately 12 minutes and 38 seconds after the Annenberg Media logo.

 You may say that of the five designs, the best uses random assignment to divide the subjects into two groups. Half of the subjects see List A first and half see List B first, and each subject ultimately sees both lists. When measurements are paired in this way (i.e., each person reads both lists), we are able to compare two different measurements for each subject. Without pairing, we must compare measurements for different people, and the differences in the people themselves may affect the difference in the measurements. Pairing will prove especially useful when we analyze our data. Note 6

Problem C6

 a. Describe a method you might use to randomly assign the 16 subjects to two groups of eight subjects each. b. Why would it be preferable to have each subject read both lists, instead of having eight subjects read List A and eight subjects read List B?

 You need to make sure that the selection process is random and that the person(s) conducting the experiment does not have any influence on dividing the subjects into groups. Regarding the second question, what would happen if the eight subjects randomly chosen for List B were unusually smart?   Close Tip You need to make sure that the selection process is random and that the person(s) conducting the experiment does not have any influence on dividing the subjects into groups. Regarding the second question, what would happen if the eight subjects randomly chosen for List B were unusually smart?

 Video Segment In this video segment, researchers at the Brigham and Women's Hospital discuss the design of a study on the affects of aspirin. Watch this segment after completing Problem C6. How was the study on aspirin designed? What characteristics of the study's design are most important in eliminating bias? If you're using a VCR, you can find this segment on the session video approximately 20 minutes and 47 seconds after the Annenberg Media logo.

 Session 6: Index | Notes | Solutions | Video