Accuracy of Model of the 2013 USN&WR Law School Rankings

As in every year since 2005, I’ve again built a model of the U.S. News & World Report ("USN&WR") law school rankings. This latest effort generated a record-high r-squared coefficient: .998673. More about what that means—and more about the one law school that doesn’t fit—below. First, here’s a snapshot comparison of the scores of the most recent (USN&WR calls them “2013”) law school rankings and the model:



As that graphical comparison indicates, the model replicated USN&WR’s scores very closely. Indeed, the chart arguably overstates the differences between the two sets of scores because it shows precise scores for the model but scores rounded to the nearest one for USN&WR.

As I mentioned above, comparing the two data sets generates an r-squared coefficient of .998673. That comes very close to an r-squared of 1, which would show perfect correlation between the two sets of scores. Plainly, the model tracks the USN&WR law school rankings very closely.

In most cases, rounding to the nearest one, the model generated the same scores as those published by USN&WR. In four cases, the scores varied by 1 point. That’s not enough of a difference to fuss over, given that small variations inevitably arise from comparing the generated scores with the published, rounded ones. Consider, for instance, that USN&WR might have generated a score of 87.444 for the University of Virginia School of Law and published it as “87.” The model calculates Virginia’s score in the 2013 rankings as 88.009. The rounded and calculated scores differ by 1.009. But if we could compare the original USN&WR score with the model’s score would get difference of only .565 points. I won’t worry over so small a difference.

You know what does worry me, though? Look at the far right side of the chart above. That red “V” marks the 4.48 difference between the 34 points USN&WR gave to the University of Idaho School of Law and the score that the model generated. Idaho showed a similar anomaly in last year’s model, though then it was not alone. This year, only Idaho does much better in the published rankings than in the model.

[Crossposted at Agoraphilia and MoneyLaw.]

U.S. News & World Report Improves Transparency of Law School Rankings

Huzzah for U.S. News and World Report! The most recent edition of its law school rankings includes the median LSAT and GPA of each school’s entering class. Finally. I have long argued that USN&WR should publish all of the data that it uses in its rankings. How else can the rest of us (read: rankings geeks) understand how—and, indeed, whether—the rankings work? Though USN&WR remains short of that ideal, disclosing median LSATs and GPAs represents a major step towards making the rankings more transparent and, thus, trustworthy.

USN&WR started the trend towards transparency last year, when it began publishing the “volume and volume equivalents” measures that it uses in its law school rankings. That input counts for only .75% of a school’s score, however. Median LSATs and GPAs together count for 22.5% of a school’s score, in contrast, making their disclosure by USN&WR all the more helpful.

There remain only two categories of data that USN&WR still uses in its law school rankings but does not disclose: overhead expenditures/student (worth 9.75% of a school’s score in the rankings) and financial aid expenditures/student (worth 1.5%). It isn’t evident why USN&WR declines to publish those inputs, too, though perhaps the financial nature of the data raises special concerns. If USN&WR cannot bring itself to publish overhead expenditures/student and financial aid expenditures/student, however, it should abandon those measures. They serve as poor proxies for the quality of a school’s legal education and if we cannot double-check the figures we cannot trust their accuracy.

[Crossposted at Agoraphilia and MoneyLaw.]