Attempt at a non-exhaustive typology: in the recent discussions of the PGR, I think we can find two main critiques (excerpts and comments below the fold):
1) the closed ecosystem critique (exemplified by Bharath Vallabha's piece; and
2) the restriction on method or "premature dogmatism" critique (exemplified by Jessica Wilson's piece).
I'd like to propose a third critique:
3) the class structure or "Moneyball" critique
A hypothesis: what the PGR has done is alert students with undergrad degrees from traditionally prestigious schools (Harvard, Yale ... those with a correlation but not guarantee of high SES for their students) to graduate school market opportunities undervalued by the traditional prestige markers (NYU, Rutgers ... -- this is the "Moneyball" angle). This is open to empirical testing, but I would guess that the UG prestige-school percentage of grad students at NYU, Rutgers and other non-traditionally prestigious but high-performing PGR schools does not resemble today what it was in the pre-PGR days.
Now of course there is co-evolution here: the faculty and friendly admins at those benefitting from the PGR's rankings could leverage that into more hires of those likely to further increase PGR ranking, and so on. Still, the above would be the outlines of the class structure / Moneyball critique.
Excerpts from Barath Vallabh's piece:
The "closed ecosystem":
The fact that it is really just the 50 ranked programs ordering themselves is not immediately apparent by looking at the list of evaluators. For in the list, besides the usual Princeton, Harvard, Rutgers and Oxford, one also sees the University of Alabama, Georgia State, Fordham, Texas A&M, and so on. The list thus gives the appearance, and the assurance, that the polling is capturing a broader landscape than just the ranked programs. It only takes a moment of looking closer to see this is an illusion. Yes, schools beyond the ranked programs are represented, but the people filling out the surveys at the unranked schools almost invariably got their PhDs from the ranked programs, and even usually from the top twenty or so ranked programs. Therefore what the representation of the unranked schools shows is not what the majority of philosophers at the unranked schools think, but rather the extent to which graduates of ranked programs have reached beyond the ranked programs and into the domain of the unranked programs....
Leading to "cultural imperialism" if exported:
One can easily imagine similar PGR rankings cropping up for India, South Korea, China, Mexico, Brazil and so on, especially as these countries seek to emulate American style liberal arts education and collaborate with prestigious American universities. Rankings which don't have to take into account the diverse philosophical ecosystems that already exist in these countries and which are vying for power and control of the emerging education markets within their country. Rankings which in effect can cut to
the head of the line by targeting five or ten schools in those countries, schools which are already familiar with the kind of philosophy done in America or which are open to being inculcated with the same philosophical culture as in the top ranked programs in America. Then through surveys it can be swiftly declared that those five or ten schools are seen to be the best philosophy departments in that country, and so therefore are the models to be emulated by all the other philosophy departments in
those countries.This is what cultural imperialism looks like. In the 1950s-80s, schools like Harvard and Princeton had the self-narrative that they are the very best departments of philosophy in the world. That they are the fount from which the most cutting edge results in philosophy flow, the departments with the all-star line ups of prized philosophers. With the rise of the internet in the 90s, PGR became the mechanism by which this narrative could be spread beyond the local networks of Harvard and Princeton and onto the broader American and world philosophical landscape. The genius of Harvard and Princeton, and now also the genius of NYU and Rutgers, spreading not only to the low-tiers of America but also to the whole world.
Excerpts from Jessica Wilson's piece:
Problems with ordinal ranking of multi-dimensional objects (see here for my take on this problem)
First, of all the disciplines, philosophy is arguably the most diverse in terms of topic, approach, and methodology: anything is potentially a topic of philosophical investigation; there are multiple canons and associated traditions or approaches to any given topic; and even among those taking a common approach to the same topic, there is often basic agreement about methodology. Such diversity is to be applauded and, with very few exceptions, encouraged---we are too far from the end of philosophical inquiry to be dogmatic (see my paper, 'Three Dogmas of Metaphysical Methodology'). That we can or should be trying to pull broadly linear rankings out of this wonderfully blooming buzz is, I think, ridiculous: one might as well try to rank flowers, cuisines, or cultures.
Leading to "premature dogmatism":
Second, ranking systems encourage premature dogmatism, whereby the favoured topics, approaches and methodology of those working at the top-ranked institutions (either absolutely, or relative to a given speciality) takes on the sheen of 'to be accepted' by others. Hence it is that so many talented philosophers of the past generation have spent their valuable time working within frameworks whose foundational presuppositions are clearly and immediately questionable, while more plausible and illuminating approaches to the topics at issue are neglected (again, see my paper). What goes for premature dogmatism concerning which methodology is correct also goes for premature dogmatism concerning which topics are worth working on. And the people most likely to think, falsely, that there is a reliable correlation between the "top" departments/philosophers and the "best" topics and methodology are... you guessed it, the very people who are invoked as most benefiting from the PGR rankings: graduate students.
Rutgers' website lists undergraduate degree (and MA if any) for almost all of their PhD students. I don't think it particularly strongly fits the moneyball hypothesis, though there may be more prestige schools listed here than in pre-PGR days. Here is the list (including only BA school, for a student listing both an MA and a BA, and not including 4 students listing only an MA school; if I counted right, three students do not list previous schools).
Rutgers 4 students
Harvard 2 students
University of British Columbia 2 students
Wesleyan
NYU
Brown
Oberlin
SUNY Buffalo
Calvin College
Stanford
Temple
Lafayette College
Miami
Yale
Universidad de los Andes
Dartmouth
Washington and Lee
Virginia Commonwealth
Wyoming
Amherst
USC
Univ of Washington
Emory
Ohio University
Spring Arbor University
Toronto
Belgrade
Minnesota
Biola U
Boston College
2 students list Master's degrees from St Andrews and 2 students list Master's degrees from Edinburgh
Posted by: Michael Kremer | October 18, 2014 at 11:17 AM
Hi Michael, yes, I had looked at that website, and at others, in making up the Moneyball hypothesis. We would need to do a lot more work to see what the historical trends are.
In any case, just eyeballing the list, the only (domestic) candidates for School of Hard Knocks places that jump out at me are Temple, Wyoming and SUNY Buffalo, but of course there could be high SES students there and low SES students from Harvard, Wesleyan, Brown, NYU, Stanford, Dartmouth, Amherst, and Emory.
Posted by: John Protevi | October 18, 2014 at 11:29 AM
One deeply problematic feature of graduate admission I have anecdotal evidence for is the value added for the testimony of known letter writers (personally, grad school class mate, or by visibility). Mentioning this at an admissions meeting is a pretty transparent status display on the part of the person who knows so and so, but it is presented as if knowing the letter writer increases reliability of the testimony.
Posted by: Owen Flanagan | October 18, 2014 at 11:52 AM