The admissions process is only a partial glimpse into who you are. No one can truly summarize themselves in such a succinct format—test scores don’t tell the whole story, nor do your transcript, work history, or extracurricular activities. Great MBA candidates don’t always dazzle on paper, and interviews are limited opportunities to highlight hidden strengths or transcended setbacks. The Yale School of Management admissions committee is on a continual quest to make the application review process fuller, less biased, and more equitable. Assistant Dean for Admissions Bruce DelMonico explains the ways Yale SOM looks at complex—i.e., human—candidates.
Admissions officers often refer to their review of an application as “holistic.” What does that mean?
That’s a great question. By holistic review, admissions officers mean that they seek to look beyond a few basic data points—typically undergraduate grades and standardized test scores—to get a full sense of an applicant using all the information in the application. This approach acknowledges that things like grades and test scores offer important but partial insights into an applicant’s candidacy. It also recognizes that those metrics might contain biases that can disadvantage certain profiles and populations of applicants. This last point is ironic since standardized tests were initially used in the first half of the 20th century to eradicate a previous form of overt discrimination in the college admissions process (though they also reflected their developers’ own prejudices), but they have since come to be seen as perpetuating a different, more subtle form of bias.
What is Yale’s position on holistic review?
We very much subscribe to the idea of holistic review—we do not believe that applicants should be defined by a limited set of reductive data points. But I think we approach this process in a unique way. The downside of reducing the emphasis on grades and test scores—which were meant to create a consistent yardstick across applicants—is that admissions officers then tend to rely more heavily on even more subjective measures such as essays and interviews. By simply shifting emphasis from grades and scores to essays and interviews, holistic review runs the risk of replacing one form of bias with a different one.
At Yale, we’ve worked hard—and I’ve made it a focus for the better part of my 17 years at the school—not just to shift our focus among an existing set of application elements but to think of new ways to look at applicants that will help us reduce bias and also improve outcomes in the evaluation process. The most notable of these tools is our behavioral assessment, which we use to supplement grades and scores in helping us evaluate applicants’ academic potential.
What is the behavioral assessment, and how does it work?
The behavioral assessment is a research-based online admissions tool that measures a set of interpersonal and intrapersonal competencies associated with business school success. We’ve been working with the Educational Testing Service (ETS) for close to a decade to adapt the behavioral assessment for the high-stakes admissions context. The assessment consists of 120 pairs of statements that describe various behaviors; applicants choose which of the pair best describes them. By the end of these 120 statements, which take 20-25 minutes to complete, we have a better sense of you as a person and of your qualities. An important thing about the instrument is that there are no right or wrong answers—we’re not looking for a particular thing in the test—and thus, it’s not possible to game or fake it. It requires no specialized knowledge or advanced preparation.
Why did you adopt the behavioral assessment?
We developed the assessment because we found that although standardized tests had strong predictive power, there were some students who did well on the tests and then underperformed in the program and others who did modestly and overperformed relative to their test scores. We were—and still are—looking for means of finding secondary and tertiary traits that help us identify these outliers and thus better evaluate applicants. The goal is to reduce our reliance on the tests to broaden the range of candidates we admit, which has the benefit of opening up the school to a more diverse cohort of students.
A couple of key aspects of the behavioral assessment are: first, that we use it to expand the ways in which we evaluate applicants, not reduce them; and second, that it is computer-scored rather than human-scored. To the first point, as I noted before, other schools tend to lessen their emphasis on grades and scores by shifting their focus to other, more subjective aspects of the application, which introduces greater inconsistency into the evaluation process. Waiving scores or going test-optional only exacerbates this shift.
The behavioral assessment, by contrast, is used to complement test scores and provide additional context, which heightens the predictive validity. Rather than shrinking the information we consider, we are expanding it. And we do so in a consistent way because the behavioral assessment is computer-scored, which minimizes subjectivity and creates a fairer and more equitable process. It’s not that we distrust human reviewers, but we know that we’re subject to a host of cognitive biases that can undermine the consistency of our decisions, so wherever possible, we try to remove that risk from the process.
Aside from the behavioral assessment, how else does Yale SOM try to reduce bias in the admissions process?
There are a whole host of things! Most simply, we’ve tried very hard to reduce the length of our application to include only those questions that we think will aid in our evaluation. Extraneous information can undermine the quality of decisions by focusing us on things that don’t matter. And we’re very deliberate about what information our reviewers see, and when. For example, our application, like most, has questions about academic discipline and criminal convictions. We suppress this information during the review process because we know that it can adversely affect a reader’s impression of an applicant—and that these disciplines and convictions are often unevenly and discriminatorily applied.
In addition, a number of years ago we transitioned to a structured interview format, where every applicant who’s interviewed receives the same questions in the same order. Research consistently shows that this format is far more predictive than other interview formats, and it is fairer and more consistent. We also have very structured reading and interview rubrics, and we calibrate our readers and interviewers throughout the application season to try to minimize inter-rater inconsistency. And we carefully train our readers, interviewers, and committee members on bias, leveraging the deep expertise of Yale University in this area. We also make decisions collectively as a committee—the full committee sits down together in person or on Zoom. It’s slower and more laborious to do it this way than, say, to use a single-point decision-maker or smaller committee, but I think it results in better decisions.
Are there other things Yale will do in the future to reduce bias and enhance the fairness of the admissions process?
We are constantly looking at additional things we can do—I don’t think there will ever be an endpoint to this process. For example, over the past few years, we’ve built out the information we ask about candidates’ backgrounds so that we can get a better sense of the context surrounding their applications. We were one of the first schools to ask in detail about this context, which is helpful in evaluating their academic and professional profile. But this is just a first step—there’s still much more we can do on this front, and we’re looking to partner with an external organization that will help us take a transformational leap in this regard.
In general, we try to lean on experts as much as possible in the work we’re doing. We’re always educating ourselves by talking to people in the field—academics and practitioners—to get a sense of what they think, then following those leads to see if they bear fruit. We lean especially heavily on our organizational behavior faculty and social psychologists to provide insights into what we should be looking at. We usually have three to four projects in the works at any given time. The pandemic has forced us to put a halt to these longer-term projects, but we’re hoping to resume them soon so that we can continue to evolve our processes.
We have a sense of where we’d like to get to, but we want to be thoughtful and diligent about how we do it. For example, ultimately we’d like to get to the point where we don’t need to rely on standardized test scores at all, but we want to make sure we do so without any loss of predictive fidelity. We’re currently still building the evaluative scaffolding to allow that to happen, but that takes time, and we don’t want to rush it.
Is all this work really worth the effort?
Applying to business school is a pivotal point in an individual’s life. We’re always trying to improve the quality of the decisions we make. And we owe it to the applicants to make the process as fair and accessible as possible. It’s so interesting to see what all these candidates have done in the past and what they want to do next. The application is an uplifting, aspirational time for them to reflect on where they’ve been, what they’ve done, and where they want to go, and I feel lucky to be part of that. Our students have very meaningful aspirations. It’s not just about success as defined by what they can accrue to themselves, but what kind of impact they can have, and that’s really inspiring to see.
Learn more about the Yale School of Management and apply here.
Bruce DelMonico is Assistant Dean for Admissions at the Yale School of Management in New Haven. He joined Yale SOM in October 2004 and has led the Admissions Office since November 2006. Before joining Yale, Bruce was an attorney focused on First Amendment, white-collar, and commercial litigation. He worked primarily on cases with exposure ranging from $10 million to $10 billion.
Bruce holds a BA in Honors English from Brown University, an MA in Literature from the University of Texas at Austin, and a JD from the University of Virginia School of Law. He has three children and in his spare time, he enjoys playing hockey and learning piano.
Comments or questions about this article? Email us.