I print, in its entirety, a response I received from Simona Bizzozero, the Head of Public Relations at QS. I intersperse her response with some comments and corrections of my own, and then conclude with some questions that are still unanswered:
Dear Professor Leiter,
In response to the concerns that you raise about the QS academic reputation survey, we would like to clarify several serious inaccuracies in the piece. While we respect your right to disagree with the measures we employ to compile our rankings, we feel it is important that you base your conclusions on the correct information.
As we will see, there were no "serious" inaccuracies, just a single minor one, to which we will return shortly. In fact, the main allegations of my correspondents are actually now confirmed by QS.
First, some background: The QS academic reputation survey is one of six indicators used to compile the QS World University Rankings, and also feeds into various other QS research outputs and rankings. In 2012 over 46,000 valid responses worldwide were considered, making it the largest survey of its kind in the world.
QS's conception of a "valid" response will become clear shortly.
In May this year, QS became the first compiler of global and regional rankings to be awarded the “IREG Approved” label for three of its research outputs (including the QS World University Rankings). This followed a comprehensive independent audit of the methodology and all data collection processes (including the academic reputation survey), to which QS voluntarily submitted.
What is IREG, you ask? Here is its Executive Committee; it includes Bob Morse, who produces the notorious U.S. News rankings. Mr. Morse is a journalist turned editor, with no qualifications as an expert on higher education, statistical methods, or anything else one might think relevant. The presence of Mr. Morse on the Executive Committee, and several others involved in the business of world university rankings, raises doubts about the integrity of IREG--doubts made worse when one realizes that U.S. News now reprints the QS rankings under its own brand! In other words, the QS rankings are "approved" by an organization whose executive committee includes an editor who profits off the QS rankings.
The purpose of the audit, conducted by independent experts, was to verify that the rankings are compiled professionally and with a transparent methodology. Successful organisations also need to show that they observe good practices and respond to a need for relevant information from a range of stakeholders, in particular students, higher education institutions, employers and policy makers.
The QS World University Rankings were found to satisfy all of these criteria. We therefore feel justified in querying the use of the term “shady” in relation to our rankings, and also in pointing out that your claim that our rankings are “a fraud upon the public” is contradicted by the established facts.
So far, there are no "established facts" at all. There is an alleged audit by an organization that is not at all "independent" of QS claiming that the QS rankings "are compiled professionally and with a transparent methodology." Until the audit is published, and the independent "experts" named, this is all public relations, and nothing more. In fact, as we consider some examples of the complete lack of relevant transparency in the QS methodology, this will just raise further questions about IREG, which looks to me, at this point, like a front organization for legitimating bogus ranking systems.