Teaching Evaluations: Biased Beyond Measure

Monday, April 11, 2016
4:00 pm - 5:30 pm Reception to Follow
8th Floor Barrows Hall, Social Science Matrix (map)
Philip Stark
Professor of Statistics and Associate Dean at the University of California, Berkeley

Student evaluations of teaching (SET) are widely used in academic personnel decisions as a measure of teaching effectiveness. Joint work with Anne Boring (SciencesPo) and Kellie Ottoboni (UC Berkeley) shows: that SET are biased against female instructors by an amount that is large and statistically significant. This bias affects how students rate even putatively objective aspects of teaching, such as how promptly assignments are graded. The bias varies by discipline and by student gender, among other things and it is not possible to adjust for the bias, because it depends on so many factors. Research showed that SET are more sensitive to students' gender bias and grade expectations than they are to teaching effectiveness, Furthermore, gender biases can be large enough to cause more effective instructors to get lower SET than less effective instructors. These findings are based on two datasets, 23,001 SET of 379 instructors by 4,423 students in six mandatory first-year courses in a five-year natural experiment at a French university, and 43 SET for four sections of an online course in a randomized, controlled, blind experiment at a US university.


Philip B. Stark is the Associate Dean for the Division of Mathematical and Physical Sciences at UC Berkeley. Stark's research centers on inference (inverse) problems, especially confidence procedures tailored for specific goals. Applications include the Big Bang, causal inference, the U.S. census, climate modeling, earthquake prediction, election auditing, food web models, the geomagnetic field, geriatric hearing loss, information retrieval, Internet content filters, nonparametrics (confidence sets for function and probability density estimates with constraints), risk assessment, the seismic structure of Sun and Earth, spectroscopy, spectrum estimation, and uncertainty quantification for computational models of complex systems. In 2015, he received the Leamer-Rosenthal Prize for Transparency in Social Science award. Stark was former Department Chair of Statistics and Director of the Statistical Computing Facility at UC Berkeley.

Watch: https://youtu.be/yhxUxBk-6GE