Advertisement
Supported by
After doubt about its data, the university dropped to No. 18 from No. 2. But now many are asking, can the rating system be that easily manipulated?
Send any friend a story
As a subscriber, you have 10 gift articles to give each month. Anyone can read what you share.
U.S. News & World Report likes to say that it is performing a consumer service when it puts out its annual college rankings. But on Monday, those ratings were again called into question after the publication demoted Columbia University to No. 18 from No. 2 in its newest annual list, after a monthslong controversy over whether the school had fudged its numbers.
The drop suggests that the highly influential rankings — which have been criticized for having an outsize influence on parents and college admissions — can be easily manipulated, since they rely heavily on data submitted by the universities that directly benefit from them.
Columbia’s No. 2 status was not questioned until one of its own math professors, Michael Thaddeus, in a February blog post, accused the school of submitting statistics that were “inaccurate, dubious or highly misleading.” Last week, the university said in a statement that it had miscalculated some data.
Columbia’s public humiliation raises questions for many parents and educational policymakers: Can the quality of a college be ranked by a single number, the way critics rate movies with stars? And should students choose where to go to college based on what has become a proxy for prestige?
Dr. Thaddeus said he would not draw conclusions about the quality of a Columbia education from the rankings, whether the No. 2 or the No. 18 spot.
“The broader lesson everyone should keep in mind is that U.S. News has shown its operations are so shoddy that both of them are meaningless,” Dr. Thaddeus said. “If any institution can decline from No. 2 to No. 18 in a single year, it just discredits the whole ranking operation.”
U.S. News, which has been rating colleges since 1983, says that given the cost and importance of education, it is ever more important that parents and students have some kind of guide to quality schools.
“For most of these students and their families — other than buying a home — attending college is the most consequential investment they will ever make,” Eric Gertler, chief executive of U.S. News, said in a statement.
Some experts say that though the numerical ranking system provides the satisfaction of a snap judgment, it exaggerates the differences among schools, and blurs more nuanced considerations, like whether a college is strong in certain fields or has good support systems and extracurricular activities. And, they say, the rankings encourage students to apply to a similar list of schools, regardless of their own personal interests.
“I don’t think there’s any reason that a student going to a school that’s ranked 60 versus one ranked 50 is going to have a meaningful risk for their lives,” said Mushtaq Gunja, a former official in the Obama administration’s Education Department and a senior vice president at the American Council on Education, which represents universities.
But students often apply to schools that they think will give them a leg up in life, enhancing their prospects for upward mobility, or at least for a satisfying career, solid earnings and the sense of accomplishment that comes with being educated.
The fixation with status that keeps the college rankings organizations — not just U.S. News but others like The Wall Street Journal, Forbes and Washington Monthly — in business may be overblown but it is not irrational, said Colin Diver, former president of Reed College, a rare school that does not participate in the rankings, and former dean of the University of Pennsylvania law school, which does.
“It’s based on a not-irrational premise that you’re more likely not only to get jobs, but you’re more likely to get noticed, you’re more likely to have good connections,” he said. “You’ll have a pedigree, and in America, a little of that is conferred by family, but most of it is conferred by education.”
As for the schools themselves, he said, “They have a love-hate relationship with U.S. News. Publicly, they may be reluctant to say, ‘We love this ranking system, anti-intellectual as it is,’ but in fact, when your ranking goes up you tend to brag about it.”
Mr. Diver argued that schools were far too complex to be properly reduced to a single number, even taking into account the 17 criteria and subcriteria used by U.S. News, including reputation (20 percent); student selectivity (7 percent, of which SAT and ACT scores are weighted at 5 percent); and debt held by graduates (5 percent).
Mr. Gertler of U.S. News countered that the rankings strove to be “the pre-eminent, objective resource to help high school students and their families make the most well-informed decisions about college and ensure that the institutions themselves are held accountable for the education and experience they provide to their students.”
Many critics of the rankings are especially troubled by the peer assessment, a survey of school reputation sent out to presidents and deans. They argue that it is impossible for anyone to know enough about hundreds of institutions to accurately rank their reputations, a survey that counts for 20 percent of the U.S. News score.
But schools continue to cooperate with the rankings because they are afraid that if they do not, U.S. News will use data from other sources that may be unfavorable to them, Mr. Diver said.
Mr. Gunja recalled that the Obama administration had created a college scorecard that compares institutions. “What you’ll see is not a ranking, but it does give important information about salary after graduation, graduation rate, field of study, demographic information,” he said.
“U.S. News tries to boil all of that stuff down to one number, and I get that — families are looking for some help here — but I don’t think it’s the answer,” he said. He said the scorecard had become increasingly popular among guidance counselors.
Columbia’s downfall began in February, when Dr. Thaddeus questioned the accuracy of the university’s data, saying he had compared it to publicly available sources and found discrepancies.
After originally defending its data, Columbia announced in June that it was withdrawing from the next set of rankings because of the questions raised by Dr. Thaddeus. U.S. News in turn announced that it was withdrawing Columbia from the rankings.
But on Monday, U.S. News announced the 2022-23 rankings, with Columbia restored to the list, at No. 18.
The statement said that Columbia’s new rank was calculated with data from the U.S. Education Department’s National Center for Education Statistics, the peer assessment survey conducted by U.S. News, and the government’s College Scorecard. Where there was no third-party data, U.S. News said, it “assigned competitive set values.”
Robert Morse, chief data strategist for U.S. News, said the formulas for calculating assigned values generally assigned a value below the average score for that indicator. Dr. Thaddeus said the values appeared to be “just a slightly more decorous way of saying they pulled these numbers out of the air.”
In the new rankings, Princeton ranks first, M.I.T. is second, and Harvard, Yale and Stanford are tied for third. Last year, Columbia was second to Princeton and tied with Harvard and M.I.T.
U.S. News regularly announces that it has found discrepancies in data submitted by universities. The consequences of misreporting usually involve being pulled from the list, but on occasion they have been harsher.
Last year, a former dean of Temple University’s business school was found guilty of using fraudulent data between 2014 and 2018 to improve the school’s rankings. In those years, the school’s online M.B.A. program was ranked best in the country.
This year, the University of Southern California pulled its education school out of the rankings because of inaccuracies that went back five years.
But Columbia, an Ivy League institution, is probably the most prestigious university in recent memory to be accused of providing incorrect data.
On Friday, just before the new rankings were released, Columbia admitted that it had submitted either “outdated” or “incorrect” data in two of the metrics that go into the ranking, class size and the number of faculty with the highest degrees in their field.
Columbia said the mistakes were a result, at least in part, of the “complexity” of the reporting requirements.
“We deeply regret the deficiencies in our prior reporting and are committed to doing better,” Columbia’s provost, Mary Boyce, said in a statement.
In last year’s rankings, Columbia claimed that about 83 percent of its classes had fewer than 20 students. On Friday, Columbia said that 57 percent of undergraduate classes had enrollments of fewer than 20 students in fall 2021.
Last year, Columbia said that 100 percent of its full-time faculty had “terminal degrees,” the highest in their field. On Friday, Columbia revised that to about 95 percent.
But as if to prove the potency of the U.S. News rankings, many schools sent out jubilant emails on Monday extolling their ratings. Kettering University in Flint, Mich., announced it had jumped six spots to No. 6 in the Midwest section of the rankings. And the University of California system sent out a news release celebrating that six campuses were among the country’s top public universities.
Advertisement
