2009 U.S. News Law School Rankings--Peer Reputational Rankings

Yesterday I posted about the 2009 U.S. News & World Report rankings for law schools. Paul Caron at TaxProf Blog has posted a complete list of schools ranked only by their academic peer reputation. The results--located here--are extremely interesting, since rankings by peer reputation vary (sometimes significantly) from overall rankings. Remember that peer reputation is one of the most heavily weighted factors in the U.S. News rankings, so this particular variable matters a great deal.

In particular, check out the comments to Caron's post. A difference of one-tenth of a point can mean a huge move up or down with respect to ranking within this variable.

2009 U.S. News Law School Rankings

U.S. News & World Report has published its annual rankings of law schools, but the ABA Journal reports that bloggers (again) beat U.S. News to the punch with leaked rankings. The U.S. News rankings can be linked to here; an ABA Journal article on the rankings (and links to the leakers) is online here.

Much is made annually of the rankings. Many observers are critical, and some say they do not matter. But for better or worse, they do, since many current and potential students, current and potential faculty members, and current and potential donors pay attention to them.

My view is that the rankings can matter far less at the top than they do at the bottom. Harvard is not #1. Does that deter people from going to Harvard? No. NYU and Columbia traded places this year. So what? They are in the top of the top. A slip from the top 10 to the top 30 can be a crisis, but that happens not too often, I think. And as Theodore Seto has pointed out in his article Understanding the U.S. News Law School Rankings (available on SSRN here--I highly recommend it), much of what affects a law school's rankings is outside that school's control.

I also think that what matters more than year-to-year shifts are mid- or long-term trends. A school may misreport and fall from tier 2 to tier 3, or may have a temporary spike due to a new building, or some such thing that has a short-term impact for good or ill. But what really matters is a school's position over a period of years. It's like global warming in that sense. What matters is not the weather in any given year. What matters is climate change over a period of years. "Climate" can be defined as the "average of weather." Perhaps a law school's "real" ranking for U.S. News purposes can be defined as its average ranking over a period of years. So that in any given year, a school like George Mason's rise in the rankings might not mean much--but its climb in the rankings over the past decade and more is decidedly significant.

There's one other thing about these U.S. News rankings that is extremely interesting compared to years past: the online version can be used to rank schools in ALL tiers. In years past the 3rd and 4th tiers were listed alphabetically only. But now, schools in the lower tiers apparently can be ranked. And in my opinion that is where the rankings can really matter, and perhaps be the difference between life and death of a school, or good fundraising versus tuition-dependence, or strong recruiting versus weak recruiting (of both faculty and students). If you are #1, or #3, or #9, yes, that matters. But it matters much more, I think, whether your school is in the 3rd or 4th tier--and where in that tier. If you are in the 4th tier, you'd much, much prefer to be at the top than at the bottom. At the top, you can claim to be "on the cusp" of a move up. But at the bottom, or in the middle, that's a much harder argument to make.

2008 US News Law School Rankings Leaked

The internet is abuzz right now with unofficial, leaked versions of the 2008 U.S. News & World Report Law School Rankings. The rankings are to be officially released on Friday, March 30, 2007. (The official U.S. News rankings site can be linked to here.) The leaked rankings are available on the Law School Discussion website, LLM Guide, and Concurring Opinions. Law School Discussion and LLM Guide have the full rankings--both the top 100 and the 3rd and 4th tiers. TaxProf Blog has a comparison of the 2007 and 2008 rankings here. Brian Leiter weighs in on the topic here, and on the Volokh Conspiracy Orin Kerr provides his views on rankings (note: the post was at 9:21 p.m. on March 27, 2007; you have to scroll down).

I've got my own views of the rankings, which I can opine on at some later date, but in the meantime, what do readers think of the rankings? I'd like to hear your views.

Also, for those who are interested, there is a growing body of legal academic literature on the subject of law school rankings (now there's an irony for you). For starters, try Theodore Seto, Understanding the U.S. News Law School Rankings (available on the Social Science Research Network here) and Alfred L. Brophy, The Emerging Importance of Law Review Rankings for Law School Rankings, 2003-2007 (available on SSRN here). For those who do not know, SSRN is a free service for finding scholarship, although you have to register for it (you can register at the SSRN homepage at http://www.ssrn.com/.

More on the Law School Rankings Game

For those readers really interested in the debate over law school rankings and some of the alternative ranking methodologies offered by critics of the U.S. News and World Report rankings, the University of Illinois at Urbana-Champaign has a site listing and explaining some of the alternative approaches offered. This list is available online here.

My favorite? The Law School Ranking Game of the Indiana University School of Law-Bloomington. The game allows you to change the weighting of ranking criteria. This, of course, can alter the rankings significantly. And the weighting of criteria gets back to a point I made in my previous post on the U.S. News and World Report rankings--namely, that those rankings give great weight to peer evaluations, which means that a law school is rated "good" if other schools think it is good. Which might be valid (see my previous post), but is not, I expect, what many people think those rankings are based on.

Law Schools and the Rankings Game

My apologies to readers for not posting more frequently lately. There are many things worth talking about on this blog, but only limited time. And only one blogger.

Today's topic of choice is the subject of law school rankings, and in particular the annual rankings published by U.S. News and World Report. There is a lot of traffic on the internet about this, and a number of law professors who weigh in regularly on the subject, including Brian Leiter of the University of Texas at Austin, David Bernstein of George Mason University School of Law, and Tom Bell of Chapman University School of Law. There is also a very good article on the subject in the fall 2006 issue of preLaw. I recommend it for reading by any new or future law student. Unfortunately, that issue of preLaw is not currently available online, but the preLaw website is located here.

Much of the dialogue over law school rankings goes something like this:

(1) Most law schools say the rankings are bad, in part because they encourage schools to value their performance based on various proxy factors (e.g., size of library holdings or student-faculty ratio), rather than "actual" performance or quality of education.

(2) But often schools trumpet their success in the U.S. News rankings, even if they have been critical of them.

(3) So the rankings are not as useful as they might seem, and many schools are hypocritical about the process and are in fact damaging the quality of legal education by paying attention to them.

Here are my two cents:

(1) Whenever you try to measure an intangible by using objective (or even subjective) proxy values, you get distortion. You see it in all facets of life, from grades in school to job promotions. And in fact it is one of the most challenging aspects of law itself: to come up with legal rules that do not offer loopholes--that is, rules that do not inadvertently incentivize or permit unwanted behavior or results.

This of course oversimplifies the matter, and yet it is worth bearing in mind, especially since the alternative--having no measures of performance or ranking--is not very attractive, either. Without any rankings, are law schools going to be somehow more noble and work more for the public good, or are they going to be less accountable and more inefficient? I tend to think the latter. And this is the reason that some commentators, including Professor Leiter and others, have focused on trying to come up with other, ostensibly better, measures than the U.S. News rankings.

(2) In chasing the U.S. News rankings, or any rankings, law schools risk sacrificing their missions for a rise in the rankings. Some schools probably do this, while some admirably resist temptation. But again, this goes back to making sure your proxies are as good as they can be. If the proxies are good, then the ill effects of chasing rankings are minimized.

(3) Based on my own study of the rankings, the U.S. News rankings put great weight on what peers at other law schools think of a particular law school--this constitutes about 25% of a school's final score. Personally, I find this troubling. On the one hand, these peers are supposedly people who should know, since law schools are their business. But on the other hand, there is some troubling circularity to concluding that a school is good because a lot of people say it's good.

Specifically, doesn't this heavily weighted subjective factor lead to reputational lag? Aren't some schools likely overrated due to their prestige amongst other law schools, while others who offer very good legal education but promote themselves poorly remain underrated? And once people's minds in the legal academy are made up, isn't it hard to change these perceptions, no matter what you do? It's like going back to your 10th high school reunion: you may be very successful, but to many people in the room, you're still just a nerd.

Add to this the fact that scores regarding facilities and student-faculty ratios are only a small percentage of a school's overall score, and the U.S. News rankings start to look a lot like a popularity contest. But oddly enough, it's a contest that does add accountability. Perhaps, then, it comes down to adjusting the factors looked at and the weight given to them.

These are just a few of my thoughts. I'd like to hear what others think as well.