Wednesday, July 11, 2012

The Linda Hypothetical and Testing

Most readers are familiar with the Kahneman and Tversky  classic Linda hypothetical which involves this fact pattern and the follow-up question: "Linda is thirty-one years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in antinuclear demonstrations.”

Which alternative is more probable?
 (1) Linda is a bank teller.
 (2) Linda is a bank teller and is active in the feminist movement.

Most people choose (2) over one although it is illogical to do so since it includes (1) and involves a joint and, thus, lower probability.

I've never been a believer in the Linda hypothetical. First, I am not sure it tells us as much about logic as it does interpretation. Having listed feminist as a possibility in choice (2), "not a feminist" may be inferred to be included in (1). If so,both are examples of joint probability. I am also not sure if many people know what "more probable" means. Suppose one reads that as "best" answer meaning the one that captures Linda in a more precise fashion. Finally, how about telling subjects this is a test on logic?  In short, is the test a valid test of reasoning?

For law professors, especially those using multiple choice machine graded exams the same questions are relevant.  There are many reasons for choosing a wrong answer and sometimes the answers reveal more about the teaching and testing than the students. On the typical essay exam or multiple choice with explanation test  the teacher can assess the quality of the question by examining why people missed it. Flaws in the questions are revealed. On a machine graded exam the process of "testing the test" needs to happen before the test is used. I wonder how many machine graders either copy the questions from another source and assume the questions are pretested or actually do test the questions by having a sample of students answer the questions and then debrief those giving the wrong answer. I am betting not many. Ironically, when the of issue machine graded testing comes up many of the defenses are also illogical.