Assessing How Students Learn
March 2009
Bill Cerbin
Educator Bill Cerbin argues that if the goal of higher education is to improve students' future performance, then the common practice of assessing what students have learned is not enough. What is also required is "assessment that reveals how students learn."
In higher education the dominant mode of assessment is to measure what students have learned in a course or program. By measuring what students learn educators can monitor student progress, determine learning gaps and gains, and document achievement.
But measuring what students learn is of limited use if our goal is to improve their future performance. It is akin to taking a person's temperature. You may learn the individual has a fever but the measurement produces no insight into the cause. Suppose we find that students score in the 60th percentile on a standardized test or that half the students in a course have significant writing problems. What should we do to improve future performance? Unfortunately, the assessment data provide little direction. The result is a kind of guesswork by which we consider alternative teaching practices or programs without understanding how or why they would work better than standard approaches.
To reduce the guesswork we need assessment that reveals how students learn—how they interpret and make sense of the subject, where they stumble, what they do when they do not understand the material, how they respond to different instructional practices, and so on. Understanding the basis of student performance can help us identify appropriate teaching practices or approaches.
A compelling example of this form of assessment is the Berkeley calculus project which took place more than 25 years ago. At the time there was a large disparity between the performance of African American students and other students in introductory calculus at UC Berkeley. About 40 percent of African American students received grades of D or F in calculus compared to about 5-6 percent of Caucasian and Asian students. Concerned about the disparity, mathematics educator Uri Treisman decided to explore the problem by focusing on how students learn. He wanted to understand
. . . how students actually learn calculus. Do they use the textbook? With whom and why do they discuss homework assignments? What do they do when they get stuck on a problem?—the really basic questions about how students learn mathematics. (Uri Treisman's Dolciani Lecture)
Treisman observed 40 students (20 African American and 20 Chinese American) as they went about studying and learning calculus. He was able to identify key differences in the ways that successful and unsuccessful students tried to learn mathematics. For example, Chinese students formed study groups outside of class and devoted their time to the most difficult material rather than simply reviewing the mathematics they already knew. They compared solutions, tested one another, and talked through difficult concepts. The African American students also invested a lot of time studying calculus, but did it alone. Only two ever studied with classmates.
Based on a detailed understanding of these patterns, Treisman established a program to alter the way students learned calculus in the course. It included, for example, "honors sections" of the course in which small groups of students worked on particularly challenging mathematics problems. The program addressed each obstacle that had been uncovered by observing the students. After the changes were fully implemented the percentage of D and F grades for African American students dropped to 4 percent, a stunning improvement. (See a contemporary version of the project at Emerging Scholars Program.)
A large scale study like the Berkeley project is not a practical option for most teachers. However, assessing how students learn can be integrated with classroom teaching. Teachers can scale down to examine how students learn during a single exercise, assignment, or class period, or focus on how they learn a specific concept, skill, or ability. (See the Carnegie sponsored project, Strengthening Pre-collegiate Education in Community Colleges.)
Consider several methods accessible to most classroom teachers.
Observations of Student Learning. As the venerable American philosopher Yogi Berra put it, "You can observe a lot just by watching." What better way to explore how students learn than to observe them engaged in learning during a class period? Teachers can do this during class discussions, group work, active learning exercises, online chat or discussion forums. Better yet, instructors can do periodic observations of student learning in one another's classes and then meet to discuss their findings.
Think Aloud. The think aloud is a procedure during which students say out loud what they are thinking while working on a task. Think aloud pair problem solving involves student pairs, in which one student acts as problem solver, the other as listener. The instructor circulates among the pairs to observe students thinking aloud as they work on an assigned task.
Lesson Study. In lesson study several instructors jointly plan, teach, observe and analyze student learning in the context of a single lesson. As one member of the group teaches the lesson, the others observe students and collect evidence of their learning. Lesson study allows instructors to observe the interaction between instructional activities and student learning during an entire class period. (See examples of lesson studies by instructors at University of Wisconsin campuses at College Lesson Study Project.)
Strategies that probe the learning process offer close up views of students grappling with new material, engaging in complex thinking and responding to instruction in the classroom. For example, when asked to explain social behavior college students tend to rely on a single dominant factor such as a person's upbringing or a personality trait. Psychology instructors at the University of Wisconsin-La Crosse used lesson study to explore ways to move students beyond these everyday theories of behavior. They designed a lesson in which students produced more varied and comprehensive explanations consistent with discipline-based models of behavior. But exposing students to the "correct theory" and engaging them in more complex theorizing did not change their minds. As one student said, "There may be all these other factors but I still believe the way you act depends on what kind of person you are." The episode prompted the instructor to develop sets of mini-cases in which students used psychological principles to explain behavior in "real life like" situations throughout the course.
College teachers are aware of gaps in student learning as a result of routinely grading their students' work. Encouraging teachers to assess student learning as it takes place in the classroom can help them answer questions about how and why the gaps exist. Assessing how students learn can lead to the kind of information we need to make decisions about how to improve teaching and learning.