May 2, 2016 – A new CSHE analysis proposes eliminating the SAT in Berkeley admissions. Based on UC admissions data dating back over two decades, the analysis finds that SAT scores have become almost entirely redundant and add little to the large body of applicant data now employed in Berkeley’s holistic admissions process. After taking that information into account, SAT scores predict less than 2 percent of the variance in students’ first-year grades at Berkeley.
At the same time, the analysis shows that SAT scores have a far more adverse impact on admission of black and Latino applicants than other selection criteria. High school GPA, for example, has significantly less adverse impact but is a much better predictor of student success at UC. “The cost of the SAT, in terms of its adverse effect on admission of students of color, is far out of proportion with its marginal benefit as an indicator of how students will perform at Berkeley,” the analysis concludes.
The author, Saul Geiser, is a Research Associate at the Center for Studies in Higher Education and former director of admissions research for the UC system.
SAT scores are considered at two main decision points in UC admissions. First is eligibility: Determining whether applicants meet the minimum requirements for admission to the UC system. Second is selection: At high-demand campuses such as Berkeley, with many more eligible applicants than places available, test scores are used to select from among them.
Under the proposal, while retaining the SAT for eligibility, Berkeley would eliminate the test as a factor in local selection decisions. UC delegates authority to the faculty at each campus to establish local selection procedures that reflect “campus values and academic priorities,” so that the proposed change falls within the authority of the campus and would not require approval by the UC system. The proposal differs from “test optional” admissions adopted at some other US colleges insofar as all applicants would still be required to take the SAT or ACT in order to establish minimum eligibility for the UC system.
The change would also apply to ACT scores. Most California high school graduates take the SAT, but UC also accepts the ACT, which reflects most of the same problems as the SAT.
According to the analysis, the SAT’s weak predictive power introduces a substantial element of error in selection decisions at campuses such as Berkeley, where most applicants have relatively high scores and score differences are small. Using SAT scores to compare and rank applicants results in two types of error. First are “false positives,” that is, instances where an applicant admitted on the basis of higher test scores performs worse than an applicant denied admission would have performed. Second are “false negatives,” that is, students denied admission who would have performed better than some of those admitted. Both kinds of error are inevitable when the predictive power of tests is low and score differences are small.
Despite this, some argue that SAT scores provide a cost-saving and administratively convenient metric for managing admissions at campuses, like Berkeley, that receive tens of thousands of applications. Test scores do have some marginal predictive value “on average,” that is, over large groups of students. Whatever errors they produce in individual cases may be justified by their usefulness as an administrative tool, on that argument.
The problem with that argument, according to the CSHE analysis, is that the impact of SAT scores is non-random. Compared to other admissions criteria such as high school grades, SAT scores are more strongly affected by socioeconomic background factors such as family income, parents’ education, and race/ethnicity.
And the correlation has been growing. Based on a sample of 1.1 million California residents who applied to UC over the past two decades, Geiser finds that family income, education, and race together account for over 35% of the variation in applicants’ SAT scores today, up from 25% in 1994.
Of those factors, moreover, race now accounts for the largest share of the variance in test scores among UC applicants. Rather than declining in salience, race has become more important than either family income or parents’ education in accounting for test-score differences among California high school graduates who apply to UC. As a result, the SAT has become a more significant barrier to admission of disadvantaged students in general and students of color, in particular.
Yet the strong adverse effect SAT scores on admission of students of color is unwarranted by the weak validity of the test as an indicator of student success at Berkeley, according to the analysis. Not only is the SAT a weak predictor of graduation rates for UC students overall, but it is an especially poor predictor for black and Latino students, a phenomenon known to psychometricians as “differential prediction.” Similarly, studies conducted by other Berkeley researchers demonstrate that the SAT also exhibits “differential item functioning,” or DIF, when employed with black and Latino examinees. DIF occurs “when equally able test takers differ in their probabilities of answering a test item correctly as a function of group membership,” according to National Standards for Educational and Psychological Testing.
The Standards, published jointly by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, require that colleges and universities be sensitive to the differential validity of their exams for racial and ethnic minorities. While they do not carry the force of law, the Standards provide guidance to colleges and universities on best practices to ensure “fairness in testing.” Where tests do exhibit differential validity for different subgroups, test users have a responsibility to mitigate such effects as, for example, by taking account of race in evaluating applicants’ test scores.
In California, on the other hand, Proposition 209 bars consideration of race in university admissions. And by eliminating sensitivity to race as a contextual factor in admissions, it has also barred sensitivity to how other admissions criteria, such as SAT scores, are themselves affected by race.
As Geiser concludes: “Berkeley is thus faced with a choice of some consequence. One option is to continue to employ an admissions criterion with known collateral effects on students of color, even while admissions officials are barred by law from acting on that knowledge. Continuing to use the SAT under the constraints of Proposition 209 means accepting some level of residual bias against black and Latino applicants beyond what can be justified by test validity.”
“The alternative is to discontinue use of the SAT in admissions altogether,” states Geiser. “If Berkeley cannot consider race as a contextual factor in admissions, neither should it consider the SAT.”
For access to the article, please visit: http://cshe.berkeley.edu/rops-by-year
Information on author: Saul Geiser is a Research Associate at the Center for Studies in Higher Education at Berkeley and former director of research for admissions and outreach for the UC system.
CONTACT: Saul Geiser sgeiser@berkeley.edu