I print, in its entirety, a response I received from Simona Bizzozero, the Head of Public Relations at QS. I intersperse her response with some comments and corrections of my own, and then conclude with some questions that are still unanswered:
Dear Professor Leiter,
In response to the concerns that you raise about the QS academic reputation survey, we would like to clarify several serious inaccuracies in the piece. While we respect your right to disagree with the measures we employ to compile our rankings, we feel it is important that you base your conclusions on the correct information.
As we will see, there were no "serious" inaccuracies, just a single minor one, to which we will return shortly. In fact, the main allegations of my correspondents are actually now confirmed by QS.
First, some background: The QS academic reputation survey is one of six indicators used to compile the QS World University Rankings, and also feeds into various other QS research outputs and rankings. In 2012 over 46,000 valid responses worldwide were considered, making it the largest survey of its kind in the world.
QS's conception of a "valid" response will become clear shortly.
In May this year, QS became the first compiler of global and regional rankings to be awarded the “IREG Approved” label for three of its research outputs (including the QS World University Rankings). This followed a comprehensive independent audit of the methodology and all data collection processes (including the academic reputation survey), to which QS voluntarily submitted.
What is IREG, you ask? Here is its Executive Committee; it includes Bob Morse, who produces the notorious U.S. News rankings. Mr. Morse is a journalist turned editor, with no qualifications as an expert on higher education, statistical methods, or anything else one might think relevant. The presence of Mr. Morse on the Executive Committee, and several others involved in the business of world university rankings, raises doubts about the integrity of IREG--doubts made worse when one realizes that U.S. News now reprints the QS rankings under its own brand! In other words, the QS rankings are "approved" by an organization whose executive committee includes an editor who profits off the QS rankings.
The purpose of the audit, conducted by independent experts, was to verify that the rankings are compiled professionally and with a transparent methodology. Successful organisations also need to show that they observe good practices and respond to a need for relevant information from a range of stakeholders, in particular students, higher education institutions, employers and policy makers.
The QS World University Rankings were found to satisfy all of these criteria. We therefore feel justified in querying the use of the term “shady” in relation to our rankings, and also in pointing out that your claim that our rankings are “a fraud upon the public” is contradicted by the established facts.
So far, there are no "established facts" at all. There is an alleged audit by an organization that is not at all "independent" of QS claiming that the QS rankings "are compiled professionally and with a transparent methodology." Until the audit is published, and the independent "experts" named, this is all public relations, and nothing more. In fact, as we consider some examples of the complete lack of relevant transparency in the QS methodology, this will just raise further questions about IREG, which looks to me, at this point, like a front organization for legitimating bogus ranking systems.
An untenured philosopher at a religiously-affiliated university in the American South writes:
I’ve noticed that the (rather shady!) QS rankings have appeared on your blog several times – and thought it might be interesting to briefly share my experience as a QS ‘evaluator’.
One summer not all that long ago, I was short on cash and signed up for a paid survey site. Generally, this involved filling out online surveys about cell phones, toilet paper, and frozen vegetables for a reward of between $1 and $5. One of the stranger surveys that came up was something called the QS Rankings. I assumed that I was selected to participate because of my educational background and employment (PhD in philosophy, employed full time by a university), and completed the survey without much thought.
Here is how the survey went. First, I was asked to choose my discipline from a list of about 10 options as well as my continent and country; I chose ‘Humanities’, ‘North America’, and ‘United States’. Next, I was asked to type in the five best universities in the US for the study of humanities; then I was asked to type in up to 15 top universities for the study of humanities in North America other than the five I put in before. Finally, I was asked about my ‘sub-discipline’; I was asked to type in five top universities in the world for the study of philosophy.
The survey seemed awfully suspicious at the time, and I did not imagine that the results could have any sort of serious impact. Of course – I have not done much investigation into the QS rankings, and I would imagine that they use criteria other than paid surveys – but ranking universities by using a paid survey site seems to be poor methodology, for a number of reasons!
So the QS rankings are being completed by people who happen to sign up to be paid to complete surveys! (At least this individual was actually a philosopher!) I'd be interested to hear from others who may have been surveyed by QS, and if so, how. If they are really doing something as suspect as trolling fee-based survey sites for respondents, then this makes me wonder about the data bases they use for the purportedly 'objective' measures like citations and H-indices. Why doesn't QS disclose the faculty lists they use?
UPDATE: A philosopher in the Middle East writes:
As you mentioned you'd be interested to hear about how the QS company does the surveys, I thought I email you.
I received an email (actually, 12 emails consecutively) on April 16 this year asking me to register and complete the survey. I didn't do it, as it looked dubious. First, it came from a person from a "petroleum and minerals" university in Saudi Arabia, which has nothing to do with humanities whatsoever (it has no such fields of study): http://www.kfupm.edu.sa/default.aspx
Second, I could not find any info about the person who sent me the email. I googled his name, email, affiliation -- no result.
Below is the complete message.
Sub: Requesting your time for participating in the QS World Rankings Survey 2013.
The opinion of academics forms the very core of the QS World University RankingsR - the most widely circulated of the world rankings. As a leading
academic, and someone who has been associated with King Fahd University of Petroleum & Minerals (KFUPM), I am writing to request a few minutes of your
precious time to have your say in the 2013 Academic Peer Review Survey.
You will receive a message from QS providing you the link to complete the survey. If you have not received the survey, then I would like to request
Thank you for your time and please contact me if you have any questions.
Administrative Assistant to the Rector
King Fahd University of Petroleum & Minerals (KFUPM)
Dhahran 31261, Saudi Arabia
AND MORE: A law and philosophy scholar, not presently in a tenure-track position, writes:
A while ago (more than a year ago, but I can't remember for sure how long ago) I got an email like the one your correspondent got, about surveys of academic programs. I _think_mine was about philosophy, but it might have been law. I started filling it out, but pretty soon gave up- it was taking too long, and, given the odd way it was set up, I didn't really feel like I could answer well. As in the email you got, it asked for names of departments, didn't give lists of schools (let alone of faculty), and so on. It was obviously going to lean heavily towards names people could easily recall. And, while I think I'm competent to contribute to such surveys in some areas, this one and in this style left me pretty unsure that I was contributing anything, so I didn't finish it. I didn't think much of it at the time, but now I'm pretty sure it was the QS rankings. Apparently they more or less randomly email people and ask them to do a really poorly crafted survey, among other things!
Peter Fosl (Transylvania) posted this item on Facebook with the following apt preface (which he gave me permission to quote):
From rising costs, to post-tenure review, to the eliminating of tenure and the substitution of adjuncts, to hierarchical rankings (by the corporate/non-academic press), to the shift in power from faculty to administrators, to measuring scholarship quantitatively, to measuring teaching quantitatively, to thinking about education and scholarship as a product, to crushing student loan debt--it's all the bloody ravaging by Reagan-Thatcher and their minions of our ancient and hallowed academy.
Sure the schools rich in members got richer, but not as much as in prior years, and it seemed to me there was a wider distribution of academic institutions represented (which is as it should be, given the wide distribution of academic talent over the last thirty years); here are the schools ranked by the number of new Fellows elected in 2013:
1. Harvard University (10)
2. University of California, Berkeley (9)
3. Massachussetts Institute of Technology (8)
3. Princeton University (8)
5. Stanford University (7)
5. Yale University (7)
7. Columbia University (5)
8. Johns Hopkins University (4)
8. Northwestern University (4)
8. University of California, Los Angeles (4)
8. University of Chicago (4)
12. Brown University (3)
12. Cornell University (3)
12. New York University (3)
12. University of California, San Diego (3)
12. University of California, San Francisco (3)
12. University of Pennslyvania (3)
Schools with two faculty elected this year: Michigan/ann Arbor; SUNY-Stony Brook; Colorado/Boulder; UC Davis; Illinois/Urbana-Champaign; Boston Univ; Washington/Seattle; Rice; Rockefeller; Southern California; University of Florida/Gainesville.
Schools with one faculty member elected this year: Bard; UC Santa Barbara; Florida State; UC Irvine; Oregon; Case Western; Minnesota; Temple; Pittsburgh; Purdue; Barnard; Duke; Tufts; City College; Emory; Wash U/St. Louis; UT Southwestern; UT MD Anderson Cancer Center; North Carolina/Chapel Hill; Wisconsin/Madison; Souther Methodist; Georgetown; British Columbia; Baylor College of Medicine.
There are a number of unusual things about this incident (e.g., Mr. Blumenfeld et al. say that Mr. Rousselle "spontaneously" acted as a "mediator" for the volume), but it does seem to me the crux of the matter is this: someone who prepares a book proposal, and responds to the press's referees, which then results in a contract is an editor of the resulting book. This is what Mr. Rousselle reports doing, and as far as I can see, no one is denying he did that. Mr. Blumenfenfeld et al. now offer to add Mr. Rousselle to subsequent editions, perhaps acknowledging that point. Mr. Rousselle, in any case, takes issue with several claims in the reply from the listed editors.
If the SEIU (which is the most aggressive and progressive force in the American labor movement right now) succeeds in this initiative, it will change the economics of higher education for the next generation.
MOVING TO FRONT FROM LAST YEAR (SINCE TIMELY AGAIN--originanlly posted March 6, 2009)
Applicants to PhD and MA programs are now receiving offers of admission and, if they are lucky, are beginning to weigh choices between different departments. I want to reiterate a point made in the PGR, namely, that students are well-advised to talk to current students at the programs they are considering. There are often things you will want to know that you won't glean from familiarity with the excellence of the faculty's work, even if that remains the most important, if defeasible, reason for choosing a particular department. Here are some examples of information that no ranking, no departmental brochure, and no "official" departmental representive will tell you about; all of these are drawn from stories I've heard from students over the last few years about ranked departments (the departments will remain unnamed, obviously). You can think of them as representing "types" of problems you should be aware of before enrolling. I've tweaked some of the details to protect identities.
The Absent Faculty: Are the faculty who look so good on paper actually around and interested in working with students? I heard a story about a key senior person in one department who is an alcoholic, and who simply ignores his students. In another department, almost all the graduate students had to sign an open letter to the faculty a few years ago protesting the failure of faculty to return graded papers and their general lack of interest in mentoring the students. In yet another department, a well-known senior member of the faculty spent so much time travelling and lecturing around the world, that he rarely had time to review or discuss work carefully with students.
The Sexual Predator Faculty: Are women treated as young philosophers and aspiring professionals, or do faculty regularly view them as a potential source for dates and sexual liasons? It's a bit shocking to realize that this is still a live issue in some departments, but, sadly, it is. Are faculty-student sexual relations common in the department? What happens when the relations end? Are there repeated cases of sexual harassment complaints against faculty in the department? Do they ever result in discipline? I suppose it is possible this could be an issue for male students, but all the reports I've gotten over the years have been from women victimized by male faculty.
The Nasty Faculty: Talented philosophers and scholars often differ, dramatically, in how pleasant they are personally and professionally. I recall the story of one department where a member of the faculty was known to reduce students to tears in seminar. In another department, a faculty member regularly refuses to work with students, even those interested in his areas; he works only with those he deems "worthy," and there are not many of them! In another department, faculty openly express doubts about the competence of the graduate students and their ability. Make sure the philosophers who seem most interesting to you don't fall into these categories!
The Factionalized Faculty: Many faculties are "factionalized," in the sense that there are sub-groups that rarely see "eye to eye" about departmental issues, from appointments to admissions. Where this becomes worrisome, though, for a prospective student is when certain members of the faculty who share interests and approaches control all the key resources--fellowships, resources for speakers etc.--and use that control to define "in" and "out" groups of faculty and students: students with the "wrong" philosophical interests or who express an interest in the "wrong" faculty members are denied access to important perks and support. This kind of ugly factionalization is less common, but it exists.
I wish it were possible to meaningfuly measure and evaluate faculties along these important dimensions, but, alas, it is not. I have no doubt there are many programs that are really exceptional for how pleasant they are as places to do graduate study, and the way for a prospective student to discover them is to talk to lots of current students.
I don't know this guy or whether there is anything to his predictions, or he's just another pseudo-expert. But I imagine some readers are well-informed, so please share your knowledge. Does this guy have a sound analytical framework? What do you make of his prognosis? Signed comments will be strongly preferred.
It includes an explanation from the librarian's lawyer of Canadian libel law, which explains why Mellen did not bother to sue Kansas State University, the libarian's employer at the time of the original blog posting at issue. Canadian libel law is a bit closer to the insane British model than I had realized, for example, in putting the onus on the defendant to establish the truth of the facts asserted and the reasonableness of his opinions.
You can see the actual allegations in the "notice of action" here. It is, as Professor Green noted, "threadbare." If they filed such an action in the United States, they would be lucky to escape only with sanctions imposed upon them by the court. I do not know enough about Canadian libel law to comment on how they will fare there, but I am told that Canada, while not as speech-protective as the U.S., is not as libel-friendly as Britain, and that the action will fail there as well. (Many libel judgments in Britain are not enforceable in the U.S., because of the free speech implications.) Because Mellen's aim is clearly to discourage critical comment about the press, I think it is imperative that other blogs call attention to this sordid affair. Whatever the merits of their scholarly catalogue, the press does itself no favors by responding to fair critical comment this way. Indeed, they would help heal their image if they promptly withdrew the lawsuit, and simply issued a press release explaining why they think the criticisms unfair.
...are out in force because the Political Science Department and several other groups at Brooklyn College are sponsoring two speakers discussing the "BDS" (or boycott, divest, and sanction) movement in response to Israeli crimes against the Palestinians. John Protevi (LSU) has a good summary. And the President of Brooklyn College has made a good, public statement.
...to North Carolina! The "PhD in philosophy" is singled out for derisive remarks by these know-nothings, appraently unaware that UNC has one of the best philosophy programs in the United States. (The irony is William Bennett has a PhD in philosophy, though it didn't do him any good!)
At least with Springer! Philosopher David Hilbert (Illinois/Chicago) writes:
The email below seems to imply that Springer thinks I wrote a book, the original edition of which was published over 60 years before I was born. Although I'm flattered that they think I have made fundamental contributions to mathematics, I do have some doubts about their ability to successfully do the rights management that seems to be their major business these days. I wonder if they always just email some random person with the same name to obtain rights or if this is a special case.
We are writing today regarding your book The Theory of Algebraic Number Fields (ISBN: 978-3-540-62779-1), and to let you know about our plans for an electronic archive, the Springer Book Archives.
At this point you may be wondering what the Springer Book Archives are all about. The project is an effort at Springer to breathe new life into older books - specifically those published between 1842 and 2005 - by making them available in electronic formats. The archiving project includes about 40 renowned imprints. Many of these titles may even be out-of-print. With this initiative we will enable researchers all over the world to access a wealth of information via their libraries. The individual books will not only be made available digitally, but also in print. The Springer Book Archives affirms Springer's commitment to preserve valuable scholarly content, and your book plays an important role in this effort.
Your author benefits at a glance:
Your book will be digitized and become an eBook, published on SpringerLink, our online platform, and for e-reading devices such as the Kindle or iPad.
Your book can never go 'out-of-print' and will be preserved for future generations of scientists.
You will be provided with free access to the electronic version of your book once it is included in the archive.
You will receive royalties, or can choose to waive them in support of charitable organizations such as INASP or Research4Life, that help provide the developing world with access to scientific research.
Please go to the following website and select your preferred royalty option*: [omitted]
Peter Hendriks President STM Global Publishing & Marketing
UPDATE: Tad Brennan (Cornell) observes that, "It's a shame that the famous Aristotelian scholar, Hippocrates G. Apostle, died back in 1990. Springer could have given him royalties for the whole Corpus Hippocraticum, and the New Testament as well!"
Chris Bertram (Bristol) calls my attention to this analysis of the proposal, which does indeed seem worrisome along all the dimensions noted. How will this affect philosophy? Other perspectives? Comments are open, signed comments preferred.
...though the original analysis of the point in Forbeswas sloppy, but the resulting "controversy" tells us more about how little tenure-stream faculty know about what other jobs in the capitalist system are like than about the stresses of being a professor. The crucial facts are that tenure-stream faculty have considerable autonomy and considerable control over when and where they work, even if they are working fifty hours or more per week. The same can not be said for lawyers, most doctors, office workers, business men and women of all stripes, and so on.
...as we've noted before, but also, it turns out, for "quite suspicious," as in you can pay for a result you like. Of the three "world university" rankings, QS is clearly the most idiosyncratic and, it turns out, corrupt as well. THES is probably the best (THES severed its ties with QS a number of years ago, wisely), and the Shanghai rankings are fine, once you understand they are really rankings of schools by strength in the sciences, medicine, and engineering, and little else. All of them are skewed towards strength in the natural sciences, medicine and engineering to some degree by virtue of the use of citation measures.
Continental Philosophy Farhang Erfani, a philosopher at American University, provides a useful set of links to news, events, interviews, reviews, videos, etc. related to "Continental philosophy" (broadly construed)