Ariew on the Controversy over the European Journal Rankings

Roger Ariew (South Florida) has kindly supplied some additional detail about the European initiative to rank journals, to which we alluded before.  Professor Ariew writes:

The ERIH (European Reference Index for the Humanities) is an attempt by the European Science Foundation to grade journals in the humanities – including “history and philosophy of science” and “philosophy.” The initiative proposes some lists of academic journals, with first, second, and third divisions (see ERIH initial lists here).

About sixty editors of journals in history and philosophy of science are protesting ERIH by publishing editorials in their first issues of 2009, requesting that the compilers of ERIH remove their journals’ titles from the lists (“Journals Under Threat: a Joint Response from History of Science, Technology and Medicine Editors”). According to these editors the initiative is entirely defective in conception and execution: there are major issues of accountability and transparency with ERIH. The process of producing the graded list of journals in science studies and philosophy was overseen by unrepresentative committees of four and five:

History and Philosophy of Science

Maria Carla Galavotti (Chair), Universitá di Bologna (IT)

Christopher Cullen, Needham Research Institute, Cambridge (UK)

Jaroslav Folta, National Technical Museum, Praha (CZ)

Juho Sihvola, University of Helsinki (FI)

Philosophy

François Recanati (Chair), Institut Jean Nicod, CNRS/EHESS, Paris (FR)

Manuel Garcia-Carpintero, Universitat de Barcelona (SP)

Diego Marconi, Universitá degli Studi di Torino (IT)

Kevin Mulligan, Université de Genève (CH)

Barry Smith, Birkbeck College, University of London (UK)

The great concern is that exercises such as ERIH can become self-fulfilling prophecies. 

If such measures are adopted as metrics by funding and other agencies, then scholars 

in our fields will conclude that they have little choice other than to limit their 

publications to journals in the first division. We will sustain fewer journals, much less 

diversity and impoverish our disciplines.

The protest has been gaining traction. A-HUG (The Arts & Humanities User Group) recently sent a letter to the chief Executive of the AHRC (Arts and Humanities Research Council — the UK's main funding agency for the arts and humanities), protesting ERIH, on behalf of such organizations as the British Philosophical Association and the British Society for the Philosophy of Science, and making use of the editorial (see A-HUG to AHRC).

Here is the A-HUG letter to AHRC:

A-HUG: The Arts & Humanities User Group

contact: Professor Robin Osborne, University of Cambridge ([email protected])

Dear Professor Esler

The Arts and Humanities community is increasingly concerned about the European Research Index in the Humanities. At the meeting which you kindly set up in February a very wide range of Arts and Humanities subject associations and learned societies were able to hear an explanation of the purposes which the ERIH is supposed to serve and to express their grave disquiet at the incoherent conception and mistaken assumptions that lie behind the ERIH and about the way in which the ERIH lists had been constructed. It is now six months later, and although Rudiger Klein of the ESF promised to take back these concerns to the board responsible for the ERIH, we have received no communication from him and there has been no public response to our calls for ERIH to be withdrawn until such time that it can be provided with a coherent rationale and competently executed.

Our concern is both with the failure of the ESF to show any sign of taking the criticisms seriously and with the failure of the AHRC to play an appropriate role in representing the UK Arts and Humanities Community to the ESF. We wish to make three points.

1. You have consistently, and most recently in your response to enquiries from the British Philosophical Association, repeated the ESF claims that ESF rankings are not qualitative. The Arts and Humanities Community have repeatedly drawn to your attention that when the ESF describes ‘A-rated’ journals as ‘High ranking, international level publication’ and ‘B-rated’ journals as ‘Standard, international level publication’ no ordinary reader can understand the distinction between 'high-ranking' and 'standard' as other than qualitative.

2. Journal editors themselves can see only harm to their interests in these rankings. There is a wide variation in the impact and quality of articles even in the very best journals. A good proportion of work rejected by the most highly rated journals and appearing in less highly rated ones is as good or better than work published in the most highly rated ones. Anything which suggests that the continuum of journal publication divides into distinct categories and gives those categories labels that can only be interpreted as a hierarchy is bound to coerce authors into aiming their submissions more exclusively at the category at the top of the hierarchy. But for editors of A-rated journal to be inundated with contributions which are an imperfect fit for the journals aims makes their task very much more difficult. The more editors are overwhelmed with contributions, the more difficult it becomes for them to set up appropriate peer review and the more likely it is that the quality of publication becomes more uneven. Nor is it in the interests of the research community that the desire to have publications in A-rated journals should lead to research being directed ever more exclusively at the sorts of topics which A-rated journals favour. It has been noted that in the social sciences, where some fields have themselves encouraged the idea that there is a hierarchy of journals in a field, the most influential papers turn out not to be published in the journals at the top of that hierarchy.

As you may be aware, the editors of journals in the History of Science have grouped together to request that the ESF withdraw their journals from the ERIH (we reproduce their joint editorial as Appendix A below). The Arts and Humanities community believes that if the ESF will not withdraw the ERIH, the only responsible action for the AHRC to take to ensure the continuing health of Arts and Humanities journals in the UK is to support and encourage such piecemeal withdrawals.

3. The aims of the ERIH, to give the Arts and Humanities an easily calculable measure by which it can indicate the impact of its research to governments, are very close to the aim of the UK's Research Assessment Exercise. The ESF aim is to give an indication of research impact at a national level, whereas the RAE aims to give an indication of research impact at the level of the individual university, but what they are concerned to measure is essentially the same. The discussions over successive RAEs and now over the REF have shown that both within and beyond the Arts and Humanities the academic community is convinced that any such assessment must be based on peer review not of journals in the abstract but of particular research outputs. The AHRC's own practices endorse this view by their dependence on research outputs. The Research Assessment Exercise panels have shown that it is possible for peer review to command the respect and confidence of the subject communities. Peer review is what both appointment to academic posts and academic promotions, both in this country and in the U.S.A. exclusively rely upon. The AHRC has been pioneering the construction of peer review networks, and the Arts and Humanities community fully supports the principle lying behind this construction.

            

We call upon the AHRC, therefore, to take the following courses of action:

a) to advise all Arts and Humanities subject associations and learned societies that, while the ERIH descriptions of journal categories A and B continue to indicate that A journals are of a higher standard than B journals it cannot support the ERIH and that it is requesting of the ESF that no journal edited in the UK be listed in the ERIH rankings;

b) to undertake a campaign within the members of the ESF to demonstrate that the aims of the ERIH would be better met by constructing a system for robust peer review based on an extension of the peer-review network which the AHRC is itself establishing.

We will offer our full support to the AHRC in undertaking both these courses of action.

Robin Osborne on behalf of:

Architectural Humanities Research Association

Association of Art Historians

AUDTRS (Association of University Departments of Theology & Religious Studies)

AUPHF (Association of University Professors of French and Heads of Departments of French in the United Kingdom and Ireland)

British Association of Slavonic and East European Studies

British Philosophical Association

British Society for the Philosophy of Science

Council for British Archaeology

Council of the British Society for the History of Science

Council of University Classical Departments

Linguistics Association of Great Britain

Media, Communications and Cultural Association

Modern Humanities Research Association (MHRA)

Philological Society

Royal Asiatic Society

Society of Legal Scholars

British Association for the Study of Religions

Appendix A

E D I T O R I A L

JOURNALS UNDER THREAT: A JOINT RESPONSE FROM

HISTORY OF SCIENCE, TECHNOLOGY AND MEDICINE EDITORS

We live in an age of metrics. All around us, things are being standardized, quantified, measured. Scholars concerned with the work of science and technology must regard this as a fascinating and crucial practical, cultural and intellectual phenomenon. Analysis of the roots and meaning of metrics and metrology has been a preoccupation of much of the best work in our field for the past quarter century at least. As practitioners of the interconnected disciplines that make up the field of science studies we understand how significant, contingent and uncertain can be the process of rendering nature and society in grades, classes and numbers.

                  We now confront a situation in which our own research work is being subjected to putatively precise accountancy by arbitrary and unaccountable agencies. Some may already be aware of the proposed European Reference Index for the Humanities (ERIH), an initiative originating with the European Science Foundation. The ERIH is an attempt to grade journals in the humanities — including “history and philosophy of science”. The initiative proposes a league table of academic journals, with premier, second and third divisions. According to the European Science Foundation, ERIH “aims initially to identify, and gain more visibility for, top-quality European Humanities research published in academic journals in, potentially, all European languages”. It is hoped “that ERIH will form the backbone of a fully-fledged research information system for the Humanities”. What is meant, however, is that ERIH will provide funding bodies and other agencies in Europe and elsewhere with an allegedly exact measure of research quality. In short, if research is published in a premier league journal it will be recognized as first rate; if it appears somewhere in the lower divisions, it will be rated (and not funded) accordingly.

                  This initiative is entirely defective in conception and execution. Consider the major issues of accountability and transparency. The process of producing the graded list of journals in science studies was overseen by a committee of four (the membership is currently listed at http://www.esf.org/research-areas/humanities/research-infrastructures-including-erih/erih-governance-and-panels/erih-expert-panels.html). This committee cannot be considered representative. It was not selected in consultation with any of the various disciplinary organizations that currently represent our field such as the European Association for the History of Medicine and Health, the Society for the Social History of Medicine, the British Society for the History of Science, the History of Science Society, the Philosophy of Science Association, the Society for the History of Technology or the Society for Social Studies of Science. Journal editors were only belatedly informed of the process and its relevant criteria or asked to provide any information regarding their publications. No indication has been given of the means through which the list was compiled; nor how it might be maintained in the future.

                  The ERIH depends on a fundamental misunderstanding of conduct and publication of research in our field, and in the humanities in general. Journals’ quality cannot be separated from their contents and their review processes. Great research may be published anywhere and in any language. Truly ground-breaking work may be more likely to appear from marginal, dissident or unexpected sources, rather than from a well-established and entrenched mainstream. Our journals are various, heterogeneous and distinct. Some are aimed at a broad, general and international readership, others are more specialized in their content and implied audience. Their scope and readership say nothing about the quality of their intellectual content. The ERIH, on the other hand, confuses internationality with quality in a way that is particularly prejudicial to specialist and non-English language journals. In a recent report, the British Academy, with judicious understatement, concludes that “the European Reference Index for the Humanities as presently conceived does not represent a reliable way in which metrics of peer-reviewed publications can be constructed” (Peer review: The challenges for the humanities and social sciences, September 2007: http://www.britac.ac.uk/reports/peer-review). Such exercises as ERIH can become self-fulfilling prophecies. If such measures as ERIH are adopted as metrics by funding and other agencies, then many in our field will conclude that they have little choice other than to limit their publications to journals in the premier division. We will sustain fewer journals, much less diversity and impoverish our discipline.

                  Along with many others in our field, this Journal has concluded that we want no part of this dangerous and misguided exercise. This joint Editorial is being published in journals across the fields of history of science and science studies as an expression of our collective dissent and our refusal to allow our field to be managed and appraised in this fashion. We have asked the compilers of the ERIH to remove our journals’ titles from their lists.

Michael Hoskin (Journal for the History of Astronomy)

Hanne Andersen (Centaurus)

Roger Ariew & Moti Feingold (Perspectives on Science)

A. K. Bag (Indian Journal of History of Science)

June Barrow-Green & Benno van Dalen (Historia Mathematica)

Keith Benson (History and Philosophy of the Life Sciences)

Marco Beretta (Nuncius)

Michel Blay (Revue d’Histoire des Sciences)

Cornelius Borck (Berichte zur Wissenschaftsgeschichte)

Geof Bowker & Susan Leigh Star (Science, Technology and Human Values)

Massimo Bucciantini & Michele Camerota (Galilaeana: Journal of Galilean Studies)

Jed Buchwald & Jeremy Gray (Archive for History of Exact Sciences)

Vincenzo Cappelletti & Guido Cimino (Physis)

Roger Cline (International Journal for the History of Engineering & Technology)

Stephen Clucas & Stephen Gaukroger (Intellectual History Review)

Hal Cook & Anne Hardy (Medical History)

Leo Corry, Alexandre Métraux & Jürgen Renn (Science in Context)

Brian Dolan & Bill Luckin (Social History of Medicine)

Hilmar Duerbeck & Wayne Orchiston (Journal of Astronomical History & Heritage)

Moritz Epple, Mikael Hård, Hans-Jörg Rheinberger & Volker Roelcke (NTM: Zeitschrift für Geschichte der Wissenschaften, Technik und Medizin)

Steven French (Metascience)

Willem Hackmann (Bulletin of the Scientific Instrument Society)

Bosse Holmqvist (Lychnos)

Paul Farber (Journal of the History of Biology)

Mary Fissell & Randall Packard (Bulletin of the History of Medicine)

Robert Fox (Notes & Records of the Royal Society)

Jim Good (History of the Human Sciences)

Ian Inkster (History of Technology)

Marina Frasca Spada (Studies in History and Philosophy of Science)

Nick Jardine (Studies in History and Philosophy of Biological and Biomedical Sciences)

Trevor Levere (Annals of Science)

Bernard Lightman (Isis)

Christoph Lüthy (Early Science and Medicine)

Michael Lynch (Social Studies of Science)

Stephen McCluskey & Clive Ruggles (Archaeostronomy: The Journal of Astronomy in Culture)

Peter Morris (Ambix)

E. Charles Nelson (Archives of Natural History)

Ian Nicholson (Journal of the History of the Behavioural Sciences)

Iwan Rhys Morus (History of Science)

John Rigden & Roger H Stuewer (Physics in Perspective)

Simon Schaffer (British Journal for the History of Science)

Paul Unschuld (Sudhoffs Archiv)

Peter Weingart (Minerva)

Stefan Zamecki (Kwartalnik Historii Nauki i Techniki)