In the anti-affirmative action era, the process of “Holistic Review” has been used in college admissions as a comprehensive approach to evaluating students beyond the traditional high school GPA and standardized test score. In the UC system, holistic review is defined as the process in which applicants are assessed in terms of the full range of their academic and personal achievements, viewed in the context of the opportunities and challenges each has encountered (UC Office, 2008). To add, in my former admissions office, holistic review was a process that examined a student’s social position (i.e. socioeconomic status, race/ethnicity, gender) and intersectionality of identity dimensions (i.e. spiritual, academic, social, community) to build a case for admissions into the university. The use of holistic review has been long used at private institutions, but many public flagships have incorporated this process after race-based admissions lawsuits such as Regents of the University of California v. Bakke, Gratz v. Bollinger (Michigan), and Hopwood v. Texas. Beyond race, holistic review examines non-cognitive (non-cog) variables, such as a student’s self-concept, realistic self-appraisal, and handling racism in the evaluation process (Sedlacek, 2004). Through the use of non-cog variables, there is more information to evaluate students based on their lived experience and opportunities afforded within that contextual environment in addition to the heavily relied upon high school GPA and standardized test score. Although, holistic review and use of non-cognitive variables are meant to increase the diversity of students at selective colleges and universities, there is new research that suggests cognitive biases in admissions offices inhibit the goals of improving access and equity.
This post reflects on the AERA Early Career Award lecture given by Michael Bastedo (Michigan) on his research, “Cognitive Repairs in the Admissions Office: New Strategies for Improving Equity and Excellence at Selective Colleges.” Using fieldwork data from two public flagship institutions, Bastedo examined the cognitive biases in admissions offices and the impact on access and equity. From his talk, I address three research findings: holistic review not as robust as espoused, issues of anchoring and confirmation biases, and high cognitive load leading to cognitive closure. As he reiterated, these issues were found in the processes of admissions offices, not with the values of the individuals in them.
First, Bastedo stated that holistic review as it emerged in the data was more about reading all the documents provided in a students file, than truly using an array of documents, contextual information, and data points to build a compelling case for a student. The lack of a comprehensive review was rooted in only having 6-8 minutes of evaluation time per student file. Similarly, within holistic review, those who benefited from the process were students with high cultural capital to understand how to craft a consistent story across an application.
Another finding was the cognitive biases at play during the review process. Bastedo found issues of adjustment, anchoring, confirmation, and correspondence biases in the evaluation process. Much of these biases were rooted in the training and evaluation process guidelines. For brevity, I focus only on anchoring bias which is the tendency to rely too heavily on the first piece of information offered when making decisions. Bastedo found that holistic review processes that began with the evaluation of academics (GPA/SAT) lead evaluators to have an anchoring bias. If you read a student who was a high academic achiever, you were more likely to give additional points in the remain areas of the application based on the anchored evidence. Conversely, if you read a student with low academic data, a reader increased the likelihood of providing low scores in the remaining sections of the application. Similarly, it is common practice in admissions offices to have “clear admits” and “clear denies,” where only the academic factors are assessed, this practice, Bastedo states was “training people in confirmation bias.” Evaluators were reinforced that the greatest factors in a “good file” were only the academic aspects, which is inconsistent with the holistic review process.
Lastly, Bastedo found that the pressure of having to read files within a 6-8 minute window increased evaluator’s cognitive load and need for cognitive closure. Cognitive load relates to the amount of information that must be processed in a complex activity, such as admissions evaluation. Cognitive closure focuses on the individual’s need for firm answers and aversion towards ambiguity. The more complex the task, such as reading an unfamiliar transcript, an essay over the standard word count, or multiple letters of recommendation can cause the evaluator to seek cognitive closure and dismiss a “good” applicants file based on working-memory processes rather than the content of the file itself. Based on these cognitive theories, evaluators may unfairly reward students who have created an application where everything is consistent and nothing “sticks out,” which was found amongst students with high SES.
Dr. Bastedo provides a new lens to look at admissions practices, through cognitive biases and repairs. If admissions offices do not take these issues into account in the holistic review process, then the goal of access and equity may be undermined. Bastedo suggests that more training is necessary in defining what holistic review is for that individual admissions office and how they can reduce the cognitive load in the evaluation process. For anchoring bias, Bastedo recommends changing the way the data is shown. If non-cog variables are of importance, offices should begin the evaluation with those, or at least mitigate the bias for anchoring based on academics. The findings on the impact of cognitive load and need for closure in reading admissions files begs the question to reevaluate the process in general. Is 6-8 minutes enough time for an individual to read a student’s file? In my experience, I read files at a 16 minute pace and that was never enough. If increased reading time is not possible, Bastedo suggests that much of these issues can be improved with better evaluation processes, high-quality information, breaking up the work into smaller chunks, and understanding the role of cognition and cognitive biases in the holistic review process. As a former admissions officer, this type of research provides an opportunity for admissions offices to reflect on their practices and make adjustments that can improve their goals to enhance access and equity through holistic review.
For more information on Dr. Michael Bastedo, check out his website.