Did your hospital make the list? Five things you didn't know about US News and World Report's best hospital rankings

US News & World Report describes itself as “the global authority in hospital rankings.” Its latest rating of best hospitals, now in its 28th year, included more than 4,658 medical centers nationwide.

For the second year in a row, the Mayo Clinic took first place at the top of the Honor Roll (a list of 20 hospitals that provide “unusual competence” across multiple lines of care). The Cleveland Clinic ranked second, followed by Johns Hopkins Hospital, and Massachusetts General Hospital.

The rankings also rated hospitals in 16 areas of specialty. University of Texas MD Anderson Cancer Center ranked number one in cancer, the Cleveland Clinic came in first in cardiology & heart surgery, and the Mayo Clinic was number one in diabetes and endocrinology.

The report also rated the best children’s hospitals and the best regional hospitals by state.

But this “top-line” information has been readily reported by a wide number of news outlets, and heavily promoted by the highly-ranked hospitals. Isn’t there more to the story?

Yes, there is.

Here are five things you probably didn’t know about the nation’s most respected rating of hospitals.

1. No military or VA hospitals made the ranks. There’s a good reason why zero military or Veterans Affairs hospitals made the list: because US News & World Report doesn’t rank them. And there’s a good reason why US News & World Report doesn’t rank them: because the federal government has never released performance data. Without performance data, US News has nothing to rank.

2. “It’s just a big popularity contest.” If by “popularity contest” you mean that the rankings are simply subjective—then no, they’re not. However, they’re not completely objective, either. Scores for most specialties were based on four factors: patient mortality; patient safety; care-related factors (such as nurse staffing, patient volume, and other measures related to quality of care); and “expert opinion” obtained through physician surveys.

The first three of these four factors are based on outcomes data and other objective measures. They each have different weights, too. Survival comprised 37.5% of the overall score, while patient safety only accounted for 5% of it. And “expert opinion”? That subjective measure made up 27.5% of a hospital’s overall score.

But in four specialty areas—psychiatry, ophthalmology, rehabilitation, and rheumatology—most care is delivered on an outpatient basis. So, risk-adjusted mortality rates, which are heavily weighted in the other specialties, don’t count for much in these specialties. Accordingly, hospitals in these specialties are ranked entirely on reputation.

US News & World Report defends its methodology, noting that a high reputation doesn’t necessarily mean a better rank—and vice versa for a low reputation.

“Many ranked hospitals have very low, even zero, reputational scores but are strong clinical performers,” the report stated.

Interestingly, an independent study on cancer centers found that US News & World Report’s designation of reputation—the subjective impressions of specialists in each field—was highly correlated with a hospital’s research productivity.

“Subjective impressions appear to approximate objective differences,” the authors concluded—although they emphasized that their results do not necessarily validate US News & World Report’s hospital rankings.

3. Teaching hospitals aren’t the only hospitals ranked.

A common misperception is that only teaching hospitals are eligible for ranking.

“Teaching status or medical-school affiliation are only two of the four ways in which a hospital can be a rankings candidate,” US News & World Report explained.

Of the two other ways, one is that a hospital must have at least 200 beds set up and staffed. The other way is that it must have at least 100 beds and offer at least four out of eight advanced technologies associated with high-quality care (such as a PET/CT scanner or precision radiation therapies). This year, 2,255 hospitals—about half of the 4,658 medical centers initially considered—met at least one of the four criteria.

4. Only senior citizens’ patient data were included. Patients 65 years of age or older at the time of care were the only ones included in the analyses. Why? Because the rankings’ primary measure of patient outcomes is 30-day mortality—a measure that’s derived from Medicare claims files. Thus, only patients receiving care under traditional Medicare who were 65 years of age or older were included. Moreover, “all-payer” data for younger patients is not publicly available for most hospitals. 

“This is not necessarily a significant omission,” US News & World Report noted. “How well older patients are treated is generally considered a good test of a hospital's capabilities.”

5. There is no list of ‘the worst hospitals.’

There’s a best hospitals list, but there is no worst hospitals list, according to US News & World Report. It doesn’t calculate such a ranking.

Click here for a closer look at the US News & World Report 2017-18 Best Hospitals rankings.