August 28th, 2009
In Signaling Value of Law Reviews, I noted an article by Al Brophy (North Carolina) cautioning that scholarship should be judged on its own merits. Paul Caron notes an empirical study of “the theory of cumulative advantage in science (Matthew Effect),” that controls for article quality. In The Impact Factor’s Matthew Effect: A Natural Experiment in Bibliometrics, Vincent Larivière & Yves Gingras conclude
The intrinsic value of a paper is thus not the only reason a given paper gets cited or not; there is a specific Matthew effect attached to journals and this gives to paper published there an added value over and above their intrinsic quality.
So, it’s not just a matter of the quality of the paper, but also of its placement. It follows that an author’s academic reputation is also enhanced by placement. Thus, the urge to “trade-up” in placement of articles. Presumably, a law school’s peer reputation follows (with a lag?) that of its faculty. Earlier, I discussed Jeff Lipshaw’s (Suffolk) thoughts on the penchant of ambitious young professors to “move up the food chain to a law school with a higher ranking.
Given the strong influence of peer reputation in the US News law-school rankings, should lower-tier law-schools try to move up in the rankings by using pay-for-placement bonuses to young professors that might be just moving through? Or does their ability to do that–or their earlier association with a school–also enhance the peer reputation of that school?
Gary Rosin
Posted in Law Professors, Law Schools, Rankings, Scholarship | No Comments »
August 24th, 2009
Over on TaxProf, Paul Caron notes in What’s Wrong with Law Schoola comment by Dean Erwin Chemerinsky (UC-Irvine) that his professors at Harvard weren’t interested in their students. Caron notes that the faculty at his son’s college voted down a proposal to reduce the teaching load, and wonders if has ever done that.
With the talk about alternative outcomes and assessment measures, it will be interesting to watch what happens. Will those be limited to clinics, or will all of the law professoriate have to start worrying about whether students are actually learning?
Gary Rosin
Posted in Law Professors, Law Schools, Student Learning, Teaching | No Comments »
August 24th, 2009
I haven’t looked at this yet, but this new article looks interesting: Alfred L. Brophy, The Signaling Value of Law Reviews: An Exploration of Citations and Prestige, 36 Fla. St. U. L. Rev. 229-243 (2009)(SSRN). His conclusion is that
the results here suggest that we should we wary of judgments about quality based on place of publication. We should also be wary of judgments about quality of scholarship based on number of citations and we should, therefore, continue to evaluate scholarship through close reads of it.
Gary Rosin
Tags: law reviews, Scholarship
Posted in Law Schools, Scholarship | 1 Comment »
August 24th, 2009
Tom Bell (Chapman) has another installment of his the components of US News law-school rankings: How Top-Ranked Law Schools Got That Way, Pt. 2. In this installments, he stacks the components of the top 22 schools in a bar-graph. Stip to come: top 41-51, top 94-100 and bottom 8.
Gary Rosin
Tags: US News
Posted in Law Schools | No Comments »
August 21st, 2009
Jeff Lipshaw has an interesting comment on the rankings game, Ambition and Rankings: “We Have Met the Enemy and He Is Us”. He begins by noting a WSJ article by Eric Felten, who is of the view that “the rankings are really about getting ahead.” He then notes an discussion with a colleague about the Big Law School game–moving up to more prestigious law schools. He continues
Yes, I think the rankings do have something to do with our subjective views of getting ahead, and I do think there’s something about the legal profession that makes OUR rankings so powerful. I used the phrase “progressing up the food chain” with my colleague, and in what industries or professions is the food chain as quantitative as the legal profession? * * *
* * *
… there’s a lot of self-selection in the process of becoming a lawyer, and even more in becoming a big law firm lawyer or a law professor. I suspect the first element of that self-selection is a particular orientation to progressing up the food chain…. There ain’t that much to distinguish us…. There are only dozens and not thousands of law schools. * * * In other words, it’s easy to see a well-defined food chain in the relatively small, homogeneous, and closed legal community.
Most of the blogging about law-school rankings focuses on the top law schools, and sometimes as far down as the top 100. Perhaps that’s because they are the only schools individually ranked, but I don’t think so. I’m not sure that professors at top law schools really care about what happens on the other side of the Great Divide in the legal academy (Tiers 3 and 4). If nothing else, the concerns of the lower-ranked schools are not the concerns of the elite.
For example, during the debate about ABA Interpretation 301-6 and minimum law-school Bar passage standards, the blawgosphere was largely (entirely?) silent. Was that because the elite law schools, and even the top 100, don’t worry about the Bar? Yes, the occasional Top 100 Dean gets toppled when Bar passage rates slip. But the top law schools don’t measure themselves by the proportion of the graduates that can meet the minimum standards to be come a lawyer. That’s taken as a given.
Gary Rosin
Tags: Rankings
Posted in Law Schools, Rankings | 1 Comment »
August 20th, 2009
The official (as opposed to the leaked) 2010 US News Law School rankings came out today. Over at MoneyLaw, Tom Bell has an interesting post, How Top-Ranked Law Schools Got That Way, Pt. I. He looks at the weighted standardized scores on each of the 12 components of the overall score. He then compares the amounts by which the component scores vary among the top 100, and the top 12, law schools. As expected, the peer reputation scores (PeerRep) vary (and count) the most. The surprising result is that the second highest variation is in overall expenditures per student (Over$/Stu):
[T]he Over$/Stu z-scores range quite widely, with Yale having more than double the score of all but two schools, Harvard and Stanford, which themselves manage less than two-thirds Yale’s Over$/Stu score. That wide spread gives the Over$/Stu score an especially powerful influence on Yale’s overall score, making it almost as important as Yale’s PeerRep score and much more important than any of the school’s remaining 10 z-scores. In effect, Yale’s extraordinary expenditures per student buy it a tenured slot at number one.
If would be interesting to see the relative component contributions for Tiers 3 and 4, as well as Tiers 2 and 3.
Gary Rosin
Tags: US News
Posted in Law Schools, Rankings | 1 Comment »
August 19th, 2009
Vault has released its Top 100 Law Firms rankings. Paul Caron has posted the top 25 on TaxProf. The Vault rankings are based on a survey of associates about law-firm prestige.
Gary Rosin
Posted in Law Firms, Rankings | No Comments »
August 19th, 2009
Over at Inside Higher Ed, Stephanie Lee, in Reputation Without Rigor, looks at the methodology behind the US News peer assessment survey. Inside Higher Ed obtained the peer assessment survey form submitted by 48 of the top100 public universities in 2009 US News university rankings. While she found some gaming, some “major oddities,” most respondents gave “honest, if imperfect” responses. Her overall conclusion:
the reputational survey is subject to problems, such as haphazard responses and apathetic respondents, that add to the lingering questions about its legitimacy.
Some of the persons who responded on behalf of universities complained of the difficulty of giving an overall evaluation of a university, as opposed to particular programs. Presumably, that’s less of a problem for the law-school survey. The real problem was time:
Ten hours. With 260-some colleges, giving each two or three minutes of attention, that’s how long it would take to adequately respond to the U.S. News survey, estimates Daniel M. Fogel, president of the University of Vermont. And he says that’s time no one like him can afford to spend.
With the number of law schools at 200 or so (and growing!), the time problem also affects the law-school peer assessment surveys.
Gary Rosin
Tags: peer assessment, US News
Posted in Rankings | No Comments »
August 18th, 2009
On September 11-13, 2009, the University of Denver will host a conference, Legal Education at the Crossroads, v. 3 – A Conference on Assessment:
This conference responds to the calls for better methods of student, teaching, and institutional assessment made in the Carnegie Report, Sullivan, et al., Educating Lawyers: Preparation for the Profession of Law(2007) and in Stuckey et al., Best Practices for Legal Education (2007). The conference will be particularly useful for law teachers and deans interested or engaged in developing and implementing outcomes measures.
Gary Rosin
Tags: Assessment, conference
Posted in Conferences, Student Learning | 1 Comment »
August 18th, 2009
Earlier this year, Paul Caron listed the peer assessment scores from the 2010 US News law school rankings. The rating scale runs from 1 (highest) to 5 (lowest). Here’s how the numbers fell out:
Average: 2.55
Percentiles
25th between 1.9 (22nd) and 2.0 (30th)
50th between 2.3 (47th) & 2.4 (55th)
75th about 2.9
The distribution (with a normal reference curve) looks like this:
Looking at the actual scores, the distribution is decidedly non-normal. Of particular interest are the “fat tails”–the distributions of the top and bottom 25 percent, which are much larger than would be expected with a normal distribution. The top 75% of law schools have peer assessments have a range of 1.5 points (1.4 to 1.9), while the for bottom 25%, peer assessments have a slightly larger range, 1.7 points (from 3.1 to 4.8).
Tags: peer assessment, Rankings, Statistics
Posted in Rankings | 1 Comment »