Academics spend much time trying to assess the relative
merits of work in an area. There is no sure-fire way to do this of course. But
citation indices are one method to assess the impact work has had on
an area. Though philosophers are loathe to use them, they are widely used in other disciplines. Citation indices of course do not tell us everything we need to know
to make such a judgment. Much work is of very high quality, but sufficiently
specialized to be of interest to only a very few. Conversely, someone can write
a paper that sparks a great deal of interest for its obvious flaws. Nevertheless,
one can hope that citation indices could give us at least some sense of the
major themes in a subject area. My sense is that as philosophy has become more
specialized, more and more philosophers have simply lost contact with what is being
currently discussed in journals and books. One might hope that citation indices could provide a rough objective map of the terrain of an area that can be used in place of word-of-mouth.
Since I discovered Google Scholar about six months ago, I’ve
been comparing its citation results to my general sense of what is going on in
fields in which I work. Generally, it seems quite accurate – papers that have
had a significant impact in an area have had correspondingly greater hits on
Google Scholar than papers that have had smaller impacts. For example, some
test cases: two much-admired recent papers that have created significant
literatures in epistemology are Jim Pryor’s paper, “The Skeptic and the
Dogmatist” and Adam Elga’s “Self-Locating Belief and the Sleeping Beauty
Problem”. Google Scholar correctly reveals this; these are two of the most
cited papers in epistemology since 2000 (68 for Pryor’s paper, and 40 for Elga’s
paper). Keith DeRose’s “Solving the Skeptical Problem” is one of the most
influential papers in epistemology written in the past thirty years, and Google
Scholar again reveals this; it has 187 hits, despite being published as
recently as 1995. Ted Sider’s book Four-Dimensionalism and Timothy Williamson’s
Knowledge and Its Limits have been hugely important works, and Google Scholar
clearly reveals this (266 hits and 303 hits respectively, despite publication dates
of 2001 and 2000, respectively). One paper that has had no impact whatsoever in
its field is Jason Stanley’s 1998 contribution to the literature on personal identity,
“Persons and their Properties”. Again, Google Scholar correctly reveals this,
since this paper has no hits.
There are of course pitfalls to using Google Scholar. First,
one should refrain from comparing hit numbers across areas of philosophy. Some
areas of philosophy (e.g. philosophy of mind that borders on philosophy of
psychology and philosophy of language that borders on linguistics) are
cross-disciplinary, and so have created literatures in multiple fields. This
naturally increases the number of researchers reacting to these papers, and
correspondingly the number of hits. If one wants to compare the impact just in
philosophy of a certain work, this makes things difficult. Furthermore, some
areas of philosophy seem to involve more citation than others, or simply more
researchers. So one must take care to compare (e.g.) only work in history of
modern to other work in history of modern, or work in meta-ethics to other work
in meta-ethics. Finally, it takes a number of years for the impact of a work to
register on Google Scholar. The publication date of an article is a very large
factor, as the older an article or book is the more hits it will receive. It is
not yet possible to use Google Scholar to assess the impact of publications
from 2004 or after. So in judging the relative impact of work, it’s best to
compare work that was published at roughly the same time. Nevertheless, after
several months of procrastinating with it, in areas such as metaphysics,
epistemology, philosophy of language, and philosophy of mind, when used with appropriate
caution, it does deliver results that accord with my sense of what papers and books have
created some of the major debates in these areas.
Recent Comments