Methodology behind U.S. News’s hospital rankings questioned
By Sean Greene
May 25, 2010 9:19 p.m.
U.S. News and World Report ranks Ronald Reagan Medical Center third in the nation ““ behind the Mayo Clinic in Rochester, Minn., and the top-ranked Johns Hopkins Medical Center.
These rankings serve as a seal of approval for the hospital, said Amir Rubin, chief operating officer of the UCLA hospital system.
Recently, however, critics have questioned U.S. News’s methodology in determining its rankings.
The rankings are based partly on reputation, in which a random sample of 200 physicians are surveyed based on 12 different specialties of care, which accounts for 32.5 percent of the hospital’s total score.
Mortality rate accounts for another 32.5 percent of the score, and the number of nurses and available technology accounts for 30 percent. Five percent of the score is based on patient safety, or how well the hospital minimizes harm, according to the U.S. News website.
Finally, the hospital must be a teaching institution, have at least 200 beds or have a certain number of new technologies. According to the website, 44 percent of hospitals met that criteria this year.
Four specialties ““ ophthalmology, psychiatry, rehabilitation and rheumatology ”“ are rated on reputation alone because the mortality rates are too low.
The ranking criteria used by U.S. News coincide with the areas the UCLA medical center continually works to advance, Rubin said. As a result, recognition comes in many forms from many places. While the hospital doesn’t rely on the rankings for funding, Rubin said they do speak to the hospital’s quality of care.
“The U.S. News survey is (important) in that it looks at the top tertiary, or very high-end hospitals and teaching hospitals,” he said. “We hope that this tool demonstrates that if you come here for complex conditions, that our outcomes are among the very, very best.”
A recent analysis by Ash Sehgal, a professor of medicine and bioethics at Case Western Reserve University, showed that U.S. News’s rankings correlated closely with the hospital’s reputation, but not with the other objective scoring measures. He also found no correlation between reputation and the quality of patient care.
“It would be like giving the Dallas Cowboys a trophy without making them play any games,” Sehgal said.
There are other rating systems that give smaller hospitals a chance to stand out against the UCLA medical system’s ability to take on difficult cases, said Dylan Roby, an assistant adjunct professor of public health.
One method is to compare hospitals according to Medicare data, which focuses more on day-to-day, procedural criteria, such as administering antibiotics prior to surgery to prevent infection, Roby said.
Routine tasks like these make up the “Swiss cheese model” of patient care, Roby said, in which there are small holes in every level of care. For that reason, many safeguards are put in place to catch errors and prevent bad outcomes for patients.
“It really does become simple and boring,” Roby said of the procedure. “Those are the (routine) things that make a system safe. Small hospitals might have those systems in place, but they don’t get to show off their skills in high-resource cases seen at UCLA.”
Although U.S. News is biased toward large facilities, he said, the smaller hospitals know the areas the Medicare system compares them on, so the staff is able to “teach to the test,” in a way.
Roby acknowledged that UCLA does a good job on the Medicare’s procedural rankings, and smaller hospitals tend to send their sickest of patients here.
Many prospective patients see the U.S. News rankings, which are also used by hospitals for marketing and influence policy makers.
Sehgal said he recommends that consumers look to multiple sources to find the most important aspect of care for their needs, instead of an overall ranking.
