• Collect data appropriate to a specified purpose, and recognize limitations in existing data.
• Explain the benefits of the statistical approach to design of experiments and use it.
• Analyze data using appropriate graphs and numerical tools (primarily ANOVA).
• Derive appropriate, actionable conclusions from data analysis.
• Present results and conclusions in both technical and non-technical terms, in writing and orally.

• Describe the role of statistical thinking and methods for problem solving.
• Discuss the value of understanding, quantifying, and reducing variation.
• Develop an understanding of how statistics is integral to research in cognitive psychology.
• Determine the appropriate questions statistical consultants should ask.
• Develop the ability to read scientific literature.

Materials:

On Line Perfection Game:

Perfection is a popular game in which a person is expected to place specifically shaped pegs into the appropriate holes within a short time period. The Perfection Game is an example of a reaction time test.

The on-line Perfection Game provides students the opportunity to design many versions of the original game in order to test which variables have the largest effect on reaction time. You can leave all the variables blank when you are simply trying out the game, however, if you want to find your score is the database of results, input any specific course ID and student ID.

The following link allows you to play to the Perfection Game.

In the following lab you will have the opportunity to design and analyze an experiment to determine which factors are the most influential in successfully playing the Perfection Game.

Perfection Lab: Printable Student Version
Some older versions of Adobe Acrobat may have some trouble displaying the pdf files. You can download the most recent version of Adobe Acrobat Reader at www.adobe.com.

Instructors Notes:

Prerequisites: This lab can easily be done as a 2-sample t-test or a paired t-test using one explanatory variable. If students are planning to conduct a multi-variable experiment, they should read and answer questions from the ANOVA: Factorial Designs tutorial before Day 2. More complex, within subjects designs are discussed in ANOVA: Advanced Designs.

Paper Review: I have found that some students are initially frustrated or intimidated when they are asked to read a research paper outside of their major. In my experience, providing more time to read the paper has not been helpful. It may, however, be appropriate to:

• clarify that the paper review questions are worth only a small amount of the overall grade
• allow students to complete the assignment in groups
• allow students to turn in a revised version of their paper review questions after they are discussed in class.

Class Discussion: I suggest spending at least part of three separate class periods to discuss the Stroop (1935) paper and develop a class project or several group projects.  It is helpful to initially identify that your goal as a professor is not to be able to answer all of their questions about current research in cognitive psychology, but to teach them a process in which they can find their own answers.

You may want to consider additional discussion topics:

• You can instruct the students to go to the library and read a chapter on attention, memory, or reaction time from any introductory psychology book. Ask them to identify some of the factors that could influence reaction time.
• You may want to discuss the Woodworth Wells Color Sheet before the students read the paper by Stroop.
• If you want students to find their own research papers on reaction time tests, http://www.apa.org/psycinfo/ is a good psychology resource.

Day 1: Spend at least 20 minutes discussing the paper review questions. Students may not feel qualified to design a psychology study, but they often can suggest appropriate modifications to a given study. Start students thinking about their own primary research questions and ask them:

• How would you design a test to answer these questions?
• Which factors and what levels could be used?
• Which questions relate to main effects and which relate to interaction effects? 
• How can you design an experiment that is general enough to be of scientific interest, but specific enough that it can be conducted within a lab in a few weeks time?

Day 3: After students have submitted their experimental ideas, the class can vote on one experiment to conduct as an entire class. This allows you to test more factors and levels with replicates in less time. However, each group of students could just as easily design and analyze their own experiments.

It is beneficial to talk about scale-attenuation effects (i.e. floor or ceiling effects) before students finalize their design.  If the subjects rarely finish before the timer expires (i.e. the dependant variable is almost always at the highest time level), no differences will be seen between the conditions (even if one truly exists). While students may not consider it as fun as the other options, "no time limit" provides the most straightforward response variable to analyze.

One related topic not discussed in the listed research papers is how stress can impact reaction time. The students can calculate the average time per piece and determine if a time limitation of "short", "medium", "long", or "no time limit" impacts there average time per piece.

Final Paper Discussion: During the in-class review day, spend some time discussing how the process details (which are rarely discussed in textbooks and are often only given cursory comments in research papers) can significantly impact the analyses and conclusions of the data. My students are typically surprised by: