Former Admissions Chief Calls For Audits

by Wendy Flynn on

In a commentary on the Tulane scandal, Wendy Flynn calls rankings an easily rigged carnival game

As a long-term MBA admissions professional, I read with sadness the recent suspicion of data discrepancies at Tulane University. Unfortunately, this is an issue that has been reported upon on more than one occasion and at more than one business school.

We all know that rankings are big business – for BusinessWeek, for U.S. News & World Report and other media outlets, but also for the business schools. It’s hard for us to resist the siren song of the rankings.  Everyone craves a neat snapshot that defines the quality of your own program compared to that of your competitors.

The pressure to perform well in the rankings is incredibly high. Prospective MBA students carefully read all of the data in the rankings and use it to make one of the biggest professional investments of their lives. Employers of MBA graduates decide which schools to recruit from based on the outcomes of the rankings.

But it goes further than that.  Deans report rankings results to university presidents, donors and alumni.  Admissions and other administrative leaders can be fired based on the outcomes of the rankings. Far too much relies on data that is subject to either human error or outright manipulation.

And, what is the penalty for falsifying data?  A few individuals are fired or reassigned, a short-term PR embarrassment that quickly fades away,  and the business of rankings continues as it always has.  The NCAA has stronger penalties for breaking the rules than does the academic community of business schools when responding to rankings.

How can journalists claim integrity when they are reporting on data that they can’t be certain is correct?  How can business schools, as an academic community, place so much value on an instrument that is not adequately validated?  This shabby research would not be tolerated in the academic world, yet all bets are placed on the outcomes of the business school rankings.

I call on BusinessWeek and U.S. News to follow the lead of the Financial Times and immediately begin auditing process to validate rankings data to ensure that what is submitted by business schools, and what is reported upon by your journalists, is accurate data. Until rankings data is validated through a formal external auditing process, accusations of falsifying data will arise from time to time.

And, sadly, business schools and prospective MBA students, will continue to place all their bets on a carnival game that is easily rigged.

From 1997 to 2012, Wendy Flynn led the admissions efforts for the Texas A&M MBA Program, most recently serving as the Director of MBA Admissions for the Full-Time, Executive, and Professional MBA Programs.

DON’T MISS: UC DAVIS PUTS ADCOM CHIEF ON LEAVE or WHO COOKED THE BOOKS AT TULANE?

  • Rachel Killian

    Please don’t assume the FT’s audit process is infallible. The School data is audited infrequently and the alumni returns are not audited at all. Both are therefore still vulnerable to manipulation. And the audit process itself has been questioned in research papers, including one by Free, Salterio and Shearer (2009): http://samirabid.com/wp-content/uploads/2009/02/behind-the-ft-mba-rankings.pdf

  • Juan Carlos

    A good co-ed piece by an adcom professional. I think there are a few structural problems.

    1) Incentive verses penalty: As Ms Flynn outlined, the penalty is negligible compared to NCAA sanctions or academic plagiarism. Just a slap on the wrist, no fine and a press release. Incentive for a ranking rise is massive combined with bonus and promotion.

    2) I concur that an independent third party is needed to audit the data as a regulatory body. Self regulation does not work and leads to abuse of the system.

    I wonder what are the most popular tricks in the trade to make BSchool look good? It is the Wild West out there. Do you accidentally exclude/lose problem data such as a few unemployed MBAs in the response rate ‘employment 3 months after graduation. Target aggressively response rate by alumni in high income sectors and ignore non-profit alumni?

    Some Tulane applicants were clearly misled when they applied with regards to ranking and quality of peers.

  • Jim Dixey

    While the Tulane case deals with admissions, business schools also submit detailed data on the employment results for the graduates of each year’s MBA class. We submit that data online through the MBA Career Services Council (MBACSC) and agree to an audit if selected. That audit is conducted by an independent outside firm.

    I can’t speak as to how the audit process selects which programs to audit, however in my 10 years as the Director for Graduate Business Career Services at Mays Business School at Texas A&M, we have been audited 3 times. The auditors I worked with were thorough; our data had to be supported by documentation from the graduates themselves or a reliable and verifiable source. Graduates were contacted to verify the information we reported. The auditors checked our formulas and required us to support our assumptions. Our data was confirmed as accurate.

    Wendy is right when she discusses the pressure of “the rankings” however it hurts all of our programs when a few decide to manipulate the data. It makes all of us suspects.

    It is true that someone committed to manipulating data can probably find a way. Setting aside the pro and con debate about the value of MBA rankings, users of this information should at least have confidence that all the information is accurate, however that is achieved.

    Based on my personal experience, I am confident in the accuracy of the employment data as a result of the oversight and audit process we have in place through MBACSC.

  • Jeff

    Not one that uses this data will ever accept the data to be validated for accuracy. When money and decisions rest on a number, the number becomes what is important, not what the number represents. As long as I get the number I want, I am happy. How I get that number is not important. And if the process is questioned, well some little guy down the line did it. I never asked them to cook the numbers, here, look at our policy! And if you change the system now, the numbers will look different than what they are and everyone will be accused of manipulation, not just the few who get caught. The change you should seek is a new ranking system that includes auditing by a third party. Start fresh,

Partner Sites: C-Change Media | Poets & Quants for Execs | Tipping the Scales