Darden Dean Bruner on MBA Rankings et al

 

Robert Bruner, the dean of the University of Virginia’s Darden School of Business, has been blogging for some four years now. Under the rather unimaginative but direct title of “Robert F. Bruner, he posts quite imaginative thought pieces on a fairly regular basis. His posts are almost always thoughtful, provocative, and entertaining. They tackle meaningful topics, from business school rankings to how MBA education must change to keep up with the times.

But they’re especially helpful to business school applicants. He puts rankings into perspective, engages on the value of an MBA degree, and how to choose the best school for you.

Here are some of our favorite posts:

Choosing an MBA Program–Where Can You Do Your Best Work?

Published on May 6, 2010

Darden is in the talent discovery and development business. We look pretty far and wide for excellent inbound talent with which to fulfill our mission, “to improve society by developing principled leaders for the world of practical affairs.” And then we work very hard to develop that talent to have a transformational impact in the world. At this moment every year, we are well into the talent discovery cycle. Letters of admission have gone out for our MBA Full-Time Program and our MBA for Executives Program. About now, the candidates’ questions focus intensely on parsing out the differences among schools. Deposit deadlines are approaching. The candidates must decide, “Which school should I attend?”

As I told a group of admitted applicants at Darden recently, I have one consideration that dominates most others. Issues such as cost, convenience, geographic location, brand, alumni network, job placement, and others all arise in the candidates’ thinking, and justifiably so—but they are dwarfed in significance by the consideration I will tell. All too often, this consideration is a stealth issue that is overlooked entirely in choosing a school and then later discovered too late and with regret. My point is that it is far better to grapple with it now.

A few years ago, I was counseling a second-year student who had received two job offers and was trying to choose between them. The dilemma was stark: on one hand, high pay at a large, well-known firm in a big city to be part of a three-year leadership development program that would rotate him through many different jobs quickly. On the other hand, there was an offer for frankly low pay, to work for an unknown rapidly-growing small firm located in a dodgy neighborhood where the student would be a general manager from day one. The realities of student loans and economic uncertainty being what they were, the student was leaning toward the high-pay job offer. He seemed to be looking for my blessing on the choice.

Instead, I asked him, “where can you do your best work?” The look of astonishment on the student’s face told me that this was a new way of thinking to him. He offered some blah blah blah about hypothetical career progression. I politely suggested that he go away and think some more about my question. Not long after, he accepted the job with the small company. A few years on, he is successful in every way: happy, well-compensated, a big cheese in a much bigger company, and doing the work he feels ready and able to do. I’m glad that he and I had the conversation when we did. But the end of an MBA program is a little late to start thinking this way.

My advice to admitted applicants is to start wrestling with the question right now. “Where can you do your best work?” It’s deceptively simple and radically challenging. You must define for yourself two words in particular:

• Work. What “work” do you need to do or are you ready to do in the near future?  What “work” do you feel some passion for?  Do you hear a calling of some kind?  Students who enroll at the elite business schools aren’t there simply to bash through hundreds of tools, concepts, and buzzwords.  By and large, they are trying to work through a personal transformation of some kind.  A large majority of MBA students are contemplating the possibility of a career switch.  Virtually all of them feel ready for something bigger and want the kind of preparation that will accelerate them forward.  It is safe to say that the work you want to undertake at B-school consists of a complicated agenda that, to be achieved, requires some pretty sophisticated help.  It’s not easy; but that’s why they call it “work.
• Best. Truly transformational experiences almost always arise in engagement with others.  Thus, “best” usually entails some judgment about how you like to engage.  The best learning is active, not passive; creative, not rote; deep, not superficial; challenging, not easy; and collaborative and coached in various ways, not simply competitive—how you accomplish all that requires serious effort on your part and some ingenious design work on a school’s part.

Among the 12,000 schools in the world that award degrees in business, you can find an almost infinite range of choices around the definitions of “best” and “work.”  Where can you do it?  At the end of the day, you must choose.

From my perch as Dean, I see vast differences among the top schools—differences that seem invisible to rankings and guidebooks.  For instance, at Darden, we focus on the transformation of the student and therefore we design a total learning experience to include plenty of engagement, feedback, and individual choice.  As I have said on many occasions, what distinguishes Darden are “high touch” (a highly interactive, high-engagement learning approach), “high tone” (a focus on leadership development, not just the acquisition of tools and concepts), and “high octane” (a rigorous, transformative experience that is energizing and collaborative.)  We think that there is a great deal of “best work” to be done here.

The big point is that a life decision such as accepting a job offer or admission to an MBA program shouldn’t hinge just on the obvious criteria (dollars, title, location, etc.)  You must focus importantly on the work you want to do and how this choice can help you do it.  Where can you do your best work?

The Wolf at the Door

Published on December 29, 2008

As I wrote in my previous posting, (“The Wolf at the Door: A Parable about Ratings”) there are at least four tests that a good system of rating meets: it is objective and transparent; it tests a truly representative sample; it proves to be a valid predictor of some outcome of which we care; and its categories differentiate experience in a way that is statistically significant (could not be due to chance.)

Ratings simply bunch players into a category (AAA, AA, A, and so on.) And they are everywhere in society. Meat inspections, safety inspections, and T.V. viewer ratings probably meet the four tests. Grading of students by instructors is a form of rating–done well, it conforms to the four criteria. “Star” ratings on Amazon.com or Rottentomatoes.com flunk most of the criteria, as do online ratings of instructors by students.

Rankings list the players in some order of priority. We can apply the same four tests to rankings as we can to ratings. The New York Times list of best-selling books probably passes the tests of objectivity and representativeness; we could challenge it on the basis of significance and validity: the list is a measure of sales volume or popularity, not quality. Are the weight loss and self-help books that rise to the top of best-seller lists really the best literature that civilization affords?

Investment banking league tables are rankings too. Many bankers have issues with the way these league tables are constructed—for instance, how is credit awarded when there are two or more advisers? These league tables can be challenged on the basis of objectivity, significance, and validity. A recent critique in the Wall Street Journal noted that the league tables are measures of activity, not results.

Then we have business school rankings. In time, the story of the wolf and three pigs might apply here too. Take a moment to consider the four criteria:

**Objectivity and transparency: Very few of the B-school rankings are replicable by outsiders. Many of the rankings rely on arbitrary scoring of the schools on various criteria. And pity the poor school that fills in the questionnaire incorrectly or incompletely—in the history of school ratings, some of the raters have simply made up the data rather than collect accurate data from respondents.

**Representativeness: Virtually none of the B-school rankings warrant that the samples on which their surveys are based are representative of the larger population of alums, recruiters, or deans on whom they draw.

**Validity: What do the rankings measure? Is what they measure of any interest to those who care about excellence in management education? Do the rankings truly measure quality? Quality of what? Ideally, the rankings would measure the quality of the learning experience.

**Significance: None of the B-school rankings publish measures of variation on the underlying data, such as the standard deviation. In the absence of such statistics, it is impossible to tell whether the differences among the ranking categories are significant. For instance, is being ranked #16 significantly different from being ranked #10, or #1? We simply don’t know.

Apparently no one ^[2^] who has taken a deep dive into the B-school rankings thinks they meet the smell test. In an assessment of rankings, a task force of the AACSB concluded, “Measures used in media rankings are often arbitrary, selected based on convenience, and definitely controversial. Characteristics that are of little importance are often included, while important characteristics are excluded because they are more difficult to measure. Even when the measures do correlate with quality, media attempts to draw significant differences among similar programs are inappropriate. Indeed, weights that are applied to different characteristics to determine ranks are subjective and generally not justified. Two additional problems plague the rankings data. First, the data itself can be expensive for schools to provide. Schools can’t afford not to participate, and many have had to hire additional staff to respond to the increasing number of media requests for data. Although there is substantial overlap in the types of MBA data collected, each media survey requests some unique data and applies different definitions. The end result is that schools spend an extraordinary amount of time preparing data for media surveys. Second, the data reported to and published by the media are inconsistent. The lack of formal definitions and verification processes, combined with the highly visible and influential role of data in rankings, has been a recipe for highly implausible data. This task force believes that media rankings have had other more serious negative impacts on business education. Because rankings of full-time MBA programs are commonly presented under the label of “best b-schools,” the public has developed a narrow definition about the breadth and value of business education. This diminishes the importance of faculty research, undergraduate programs, and doctoral education and compels schools to invest more heavily in highly-visible MBA programs. Many schools have reallocated resources to activities that can enhance its ranking, such as marketing campaigns, luxurious facilities for a small number of MBA students, and concierge services for recruiters; but these gestures have little to do with quality. The result is an increase in the cost of delivering an MBA program, which generally translates to higher tuition for students. Rankings that rely on student or recruiter satisfaction can favor surface-level changes over substantive improvements. Similarly, rankings based on formulas that include student “selectivity” motivate schools to shrink entering classes and reduce diversity to “pump-up” statistics, such as average GMAT scores.”

The best we can say is that the rankings are simply data, not necessarily knowledge, wisdom, or absolute Truth. One might look at several rankings to gain a sense of the field. But as an insider to that field I see vast differences among the schools that the rankings don’t capture and that account for considerable variation in the learning experiences of students. My message to applicants: there is no substitute for on-the-ground research; you must do your own homework.

I pay attention to rankings because people I care about pay attention to them. All of Darden’s stakeholders want to be part of an enterprise of consequence. The rankings are one indicator of Darden’s impact. We steer the school by our mission and vision, not by our rankings. Steering by rankings would be like a CEO steering a business from each day’s closing stock price—what Warren Buffett calls “driving in the rear-view mirror.” We will not allow the rankings to dictate who we are. Fortunately, the rankings treat Darden relatively well. But I share the critics’ concerns. Any possible benefits from the rankings depend crucially on the quality of the measures. The publications have yet to persuade me about the quality of their measures and the meaningfulness of the rankings. Flawed metrics can lead to flawed decisions, as my previous posting (“The Wolf at the Door”) argues. Belatedly, we grieve the damage to the global economy from flawed debt ratings. Let us treat with similar caution the sustained effect of rankings on management education.

Education as Meeting

Published on January 1, 2009

“If you ask almost any adult about the impact of…school on his or her growth, he or she will not tell you about books or curriculum…or anything like that. The central memory is of the teacher, learning is meeting. ….We are learning slowly and late that education for competence without education as meeting promises us deadly values and scary options.” — Walter Brueggemann

I’ve been thinking about online education. University of Virginia is engaging the digital medium in a variety of ways to explore how it might enhance its educational impact. I note the existence of a number of for-profit providers of MBA degrees and spent some time last fall studying one of them closely. The business model of higher education is incredibly labor-intensive—shouldn’t it be possible to digitize some or all of the learning experience as a means of suppressing the relentless rise in costs? Clayton Christensen and co-authors in a new book, Disrupting Class, suggest that educators may have no choice: “disruptive innovation will change the way the world learns.” Easy access is nice. But here is a thought experiment: but what, if any, is the educational cost to the student of going completely online?

The answer is that one loses “high touch” learning and its benefits, such as the following:

Growth in social intelligence and emotional intelligence. These used to be called “street smarts,” how to get along in a rough-and-tumble world. Heightened awareness about yourself and your interactions with others is difficult to develop online. You build this awareness best in direct interaction with others, guided by counsel, coaching, and helpful feedback. The high engagement classroom is an outstanding arena in which to develop street smarts. And recruiters and executives tell us social and emotional intelligence matter. So many MBAs are technically competent, but behave in ways that impair the valuable impact of their insights. Effectiveness begins with wide-angle awareness about yourself and others. Simple technical competence is not enough as the basis for a high-impact career.

Learning to be present. At the start of each school year, I offer advice to the incoming class, a part of which is that “you must be present to win.” Being present is one of the casualties of modern life. We multi-task and are distracted from the things that matter; the urgent often crowds out the important. In-person classroom meetings teach one how to be present—it is more than simply attending. It is listening critically and speaking up respectfully at a moment of good impact. When one is present, one is taking in the whole scene: ideas, body language, and modes of expression. Most importantly, one is contributing oneself toward the success of the meeting. In an earlier posting (“ Reading and Reflecting on Information Technology”) I expressed my concern with the distractions of digital technology, with the filtering and selective engagement it affords, and with the narcissism that the electronic cocoon fosters. Actually meeting people busts the cocoon.

Learning from others. Direct observation suggests that successful executives differ from the rest on at least one important dimension: they are quick learners, particularly from the people around them. How, then, do you learn to learn rapidly? A high-engagement environment like Darden’s can show how. Here, the students probably learn as much from each other as from the instructors. We structure it this way because it is excellent preparation for professional life. A lot of this learning occurs outside of the classroom (or “offline” in digital parlance): collaboration on projects, preparation for class, mentoring, tutoring, club activities, dinners, parties, golf and sheer serendipity. The perspectives and wisdom of others rub off on you. As the Swiss psychologist, Carl Jung, said, “The meeting of two personalities is like the contact of two chemical substances: if there is any reaction, both are transformed.” In another article, Clayton Christensen disparaged learning from others “For highly selective schools, the other difficult-to-disrupt jobs have to do with networking, connections, and brand. Again, however, these jobs have nothing to do with the core missions of colleges and universities.” Learning to learn from each other and in-person is vitally important and much more durable than waves of new technology: it is at least as old as Socrates’ academy in ancient Athens.

Growth in communication skills. Seasoned executives and recruiters tell us that saying the right thing at the right time in the right way is crucially important to being an effective business person. Pure online learning might help you shape the content of your recommendations. But expressing them, particularly orally is only learned by doing. And what passes for written expression in many emails, online chats, and blogs is awful. The high engagement classroom exercises presentation skills: critiquing written work, thinking on your feet; persuading; presenting recommendations and defending them; and reading an audience. For international students, there is the opportunity to deepen one’s spoken mastery of another language. All of this is difficult to replicate in a pure online environment.

Growth in values. The astonishing frauds and collapses in the current financial crisis remind us urgently of the importance of trust and values in business practice. Ultimately, you learn these in close conversation with others. Anonymity breeds anomie. It is hard to lie (or be lied to) looking the other person in the eye. Walter Brueggemann got it right when he wrote that “education for competence without education as meeting promises us deadly values and scary options.”

Growth in leadership skills. The previous five points lead to the natural conclusion that one learns lessons of leadership more richly in direct interaction with others. To be sure, a great deal of leadership in business, government, and NGOs, is being conducted digitally. But learning to do this well is enhanced by structured exercise in person. Since leadership is a social activity, learning to lead cannot take place in isolation, like Einstein working out the theory of relativity.

Darden’s “high touch” teaching approach is the polar opposite of a totally online MBA program. ^[2^] Yet we have commenced a series of projects this year to study how digital technology can advance our educational mission. Darden will adopt the digital technology it needs to promote the deep transformational learning we offer. Indeed, our MBA Program for Executives is already employing technology in high-engagement ways. But my hunch is that the irreducible core of this kind of learning will entail education as meeting, or continued high-touch engagement.