Course evaluations ineffective, misused, report finds

Student course evaluations, which can shape a professor’s yearly earnings and track for tenure at Boston University, may not adequately reflect an instructor’s teaching quality, according to an academic paper published Sept. 26.

The paper, written by two scholars at the University of California Berkeley is titled “An Evaluation of Course Evaluations” and examines course evaluations from a statistical perspective while discussing how evaluations are used and can be improved, said study co-author Philip Stark.

“It is a call to arms saying, why don’t we pay more attention to teaching as instructors and really do a serious scholarly job on documenting teaching, rather than taking an easy way out and relying on these numbers and averages that are only tenuously connected to teaching effectiveness,” he said.

Relying on statistics to gauge the value of a course and instructor, fails to accurately represent what occurs in the classroom, according to the report.

“Averages of SET [student evaluations of teaching] ratings have an air of objectivity simply by virtue of being numerical,” the report stated. “We don’t measure teaching effectiveness. We measure what students say and pretend it’s the same thing. We calculate statistics, report numbers and call it a day.”

Asking students better questions about experiences in class can provide a better sense of a professor’s effectiveness, Stark said.

“The questions we should be asking are not omnibus questions, like, ‘Overall how effective was the instructor and how much did you learn?’” he said. “Those are things that we only know much later in retrospect.”

At BU, student course evaluations play an important role in faculty development, said BU’s Associate Provost for Undergraduate Affairs Elizabeth Loizeaux.

“Especially when students take them seriously and offer thoughtful, well-considered written feedback, such feedback helps the faculty member improve the course and hone his or her teaching skills,” she said in an email.

Student evaluations can also shape the salary of a professor and their direction of their tenure at the university, Loizeaux said.

“Student course evaluations play an important role in faculty salary decisions and in decisions about tenure and promotion when taken with other measures of teaching effectiveness, including classroom observations [and] the evidence of syllabi,” she said.

Several BU professors said student course evaluations provide them with useful information, though they could be revised to account for the complexity of a classroom experience.

Jeffrey Furman, academic director of undergraduate programs at the School of Management and professor of strategy and innovation, said course evaluations only provide a snapshot of a student’s satisfaction with a course and instructor.

“It’s hard to imagine a system that is perfect, and quite likely, it’s worthwhile to combine student ratings with additional measures of impact and teacher quality,” he said in an email.

Richard Primack, a professor of biology in the College of Arts of Sciences, said the student input he receives from course evaluations tend to be helpful and informative.

“The strength of evaluations is that [it] allows student input into the process,” he said in an email. “The weaknesses are that not all students fill out the forms, unhappy students are free to write harsh comments and most importantly, the system works against demanding professors who do not give out easy grades.”

Several students said despite course evaluation weaknesses, they do serve a useful purpose.

Kayla Cantrall, a senior in the College of Fine Arts, said students should have more time to complete course evaluations to provide more thoughtful and helpful responses to faculty.

“It might be more useful to have an online survey because it gives you a chance to think about the course,” she said. “There isn’t pressure with professors standing right outside the door.”

Stephanie Brown, a senior in the College of Communication, said course evaluations should account for how a class is run rather than how a student feels about the course subject matter.

“In evaluations, I usually pay attention to how relevant the course work is versus whether it is busy work or something that is not necessary or childish,” she said. “I would fill out a statistics evaluation differently than a public relations one because one is my expertise, and one is not.”