Philosophy Journals: Which Ones are Responsible, Which Ones Not?

A young philosopher in Australia writes:

For relatively junior people, the really useful and hard-to-get information is...which journals are well-run. For instance, it's useful to know things like:

1. Will they get back to you in a timely manner (within 3-4 months)? If not, how long will it take? (anecdotal evidence: 7 months at PhilRev, 12 months at Mind.)

2. Will they answer your emails or just ignore them?

3. Do they give reviews?

4. Will the review process be blind?

5. What's the quality of the reviewers?

6. Will revise and resubmit policies be arbitrary? (e.g. sending the paper to new reviewers who raise completely different objections)

7. Are you likely to run into arbitrary editorial decisions after the initial reviews? (Ethics looks like a culprit)

8. How long will it be before an accepted article appears in print ? (PPR: 3 years)

From what I can gather from my own experience and from word of mouth, most top journals perform shockingly on one or more of these questions.

In some cases, it seems to be systematic. It'd be useful to know how widespread these problems are at particular journals. Time to review can be a real problem for junior people, and it would be nice to have a more accurate understanding of the risks of delay and other vagaries you're letting yourself in for with different journals. And some journals are genuinely well run (e.g.. from what I can tell, Nous, PhilStudies, Phil Imprint, AJP, PhilQ seem on the ball).

I will note that my anecdotal evidence is the opposite of what is reported above with respect to Ethics, which I would have put in the "genuinely well run" category (and this prior to John Deigh, the editor, becoming my colleague).  In any case, I've opened comments; no anonymous postings, of course.

Comments

[Jonathan Schaffer]

The best journals, in my experience, have been Analysis, Phil Studies, and Nous.

The worst journal, by far, has been Mind (-based on about 4 submissions in the last 4 years, none accepted). Mind has taken an average of about 1 year for a decision. They have often ignored emails. And the referee reports have been low quality.

My worst experience with Mind involves a paper I had submitted to them, and was then invited to submit elsewhere. When I told Mind that I might withdraw the paper, they replied that the report had just come in, and that "the first referee gave a positive report." So I stuck with Mind. A month later I received the following report: "the referee’s recommendation went against the paper, and the Editor has not selected it for publication."

[Dan Kaufman]

I have to agree with Schaffer about Mind. I work in history and hence have not submitted anything to Mind. But many friends (who will go unnamed to protect the innocent) have submitted to Mind, and there are always complaints, especially about the review period. In one case, Mind took well over a year to review the paper (which was then rejected), and they ignored emails.

In my experience, the Australasian Journal has been excellent: quick review time, good referee reports, pretty timely publication. Also, I have noticed that PPR is very quick - especially if they are going to reject your paper! (6-8 weeks). At least they don't keep you waiting.

[Jason Stanley]

I've had good experiences with most philosophy journals, including those that have not accepted my papers. But I have to agree with the above comments about Mind. It is, in my experience, by some considerable margin the worst run journal in either field I've worked in (philosophy and linguistics). My experiences with them include many of the sins discussed in the young philosopher's post; horrific wait times, unresponsive editorial staff, and incompetent referees.

[Kai Wehmeier]

I've always had very positive experiences with both the Journal of Philosophical Logic and the Notre Dame Journal of Formal Logic. Synthese has been good in terms of highly competent referee reports and response times; their only drawback right now is the enormous backlog.

On the negative side, I must agree with everyone else that Mind is terrible, especially concerning the quality of referee reports. I've also had a ridiculous referee report from Phil Review, but am not sure whether that's an outlier, given my limited experience with the journal.

[Nick Treanor]

My complaint with Mind isn't about its response time, but about the fact that it advertises a response time that is significantly shorter than what anecdotal evidence suggests is typical. Its homepage says that its "target maximum" response time is 12 weeks, but, as I understand it, standard times are _at least_ twice this, and often three or four times longer.

On a more positive note, The British Journal for the Philosophy of Science has a long tradition of getting back to authors in six weeks. I have also had a good experience with Analysis (a rejection within a week!) and Phil Studies (an acceptance within three months, with great comments, although they didn't give me the chance to incorporate a response to the comments in the paper).

Are there good arguments against journals making public (via a homepage, or as part of their submission guidelines) the following information:

-details of reviewing process (initial vetting by author, blind reviewing, etc.)

-average response times, along with minimum and maximum

-typical time between acceptance and publication

~Nick

[Heath White]

Phil Quarterly appears to be an excellently run journal, both from my own experience and from other people's stories.

With regard to the original email: yea and amen. If some influential organization (the Gourmet Report?) could collect genuine information on journals turnaround times, etc., it would be of tremendous help for younger scholars. Knowing where to submit an article is right now in the same stage of anecdote-driven epistemic twilight that knowing where to apply to grad school was ten years ago.

[Sergio Tenenbaum]

Although Mind has already receive its share of negative assessments, I feel I must add my own horror story. I submitted a paper to Mind and 10 months later I got an R&R. I revised and resubmitted, naively asking myself "how long could it take to referee a revised version of a previously submitted paper?". 12 months later I got a rejection with a one paragraph useless referee report. Certainly not a journal for the wise junior Faculty.

Given the typical turnaround times in philosophy and the typical tenure clock, I wonder sometimes whether the unconditional prohibition against multiple submissions should not be reconsidered. It might be better to institute instead a policy according to which the author is committed to publish the paper in the journal only if the journal replies within, say, 3 months. After that the author is free to submit to another journal without withdrawing submission to the original journal.

[L. A. Paul]

I have had excellent experiences with Nous and the AJP, and I think my experience generalizes. Phil Review, on the other hand, takes way too long to review papers (9 months to a year in many cases, unless the paper is rejected without sending it out for external review). Worse, I know of two cases where people received revise-and-resubmits, dragging the process out for another 6 to 10 months, and then were ultimately rejected. Each of these cases involved excellent papers (it was crazy for PR to reject them) by tenured faculty at top departments, and were published in other top journals. As much as I'd like to publish something in Phil Review, I don't plan on sending them anything unless there is really no other viable option.

[Ralph Wedgwood]

With one notable exception, I've had excellent experiences with almost all the journals that I've submitted articles to (and I include some journals that rejected articles of mine with extremely useful referee's comments). This includes all the following journals: Nous, Phil Studies, Phil Quarterly, Philosophy and Phenomenological Research, the AJP, Ethics, the Phil Review, Philosophy and Public Affairs, the Journal of Political Philosophy, and the European Journal of Philosophy.

The notable exception, in my experience, was the Journal of Philosophy, which kept me waiting more than 7 months, and then sent me a rejection letter with no referee's comments at all.

[Timothy Bays]

I've had very nice experiences with JSL, AJP and even JPhil. I agree, though, that JPhil takes far too long to make decisions (my reports have always squeaked in just a day or two shy of their 6 month deadline).

I'll also join the general pile-on vis-a-vis Mind. I once sent them a piece in which I argued for two claims. After keeping it for almost a year, they rejected it with a reviewer's report which simply said that the two claims were false. Of course, the reviewer might have been right about this, but it would have been nice to get a brief explanation as to why the claims were false (or why my arguments for them failed, or etc.). But nope, just a brief "these claims are false".

[Keith DeRose]

The problems reported in the original e-mail are no doubt real & should be addressed. But it's also worth keeping in mind that it's those that have had some particularly bad experience who are most likely to report it, & so we may be collectively painting a more gloomy picture than the facts warrant.

My own general experience has been much like what Ralph reports -- all positive experiences but one. Well, positive in terms of response time, at least. Like Ralph, I'm here including journals that rejected my submissions, but did so in a timely manner (& rejections don't make for an over-all positive experience!). I'm not saying they all gave good comments: I don't really remember that for the most part. Response time is far more important to me. But in terms of response time & generally responsible handling of the submission, I've had very good to excellent experiences with submissions to: Analysis, AJP, CJP, F&P, JP, Nous, PPQ, PQ, Phil Studs, Phil Review, PPR, P&PA -- and probably several others that I'm now forgetting (& I'm not including under "submissions" cases where articles were more-or-less invited, like my SJP & PHIL PERSPECTIVES papers, which were good experience, but were invited). My one bad submission experience -- and it was really bad -- was with a not-very-prominent journal nobody's mentioned that was going through a rocky change of editorship at the time, and since that was more than 10 years ago, I won't even mention its name (& you can't figure it out by looking at my CV, because I sent the paper elsewhere).

The PHIL REVIEW has received several negative comments, so I should say that my experiences with it have all been excellent. Every time I sent them a submission, their first response was to ask me to R&R, so I have quite a bit of experience there. Every time, their first response either came within 3 months after I submitted, or at least soon enough after that that I din't notice the difference. And all the responses to the second versions came in much more quickly than that (generally a matter of a couple or a few weeks). And in the case of the PHIL REVIEW, I do remember the quality of the comments, because in every case they were truly excellent.

Perhaps part of what's going on is that I may have been fortunate enough to have my PR papers done "in-house". Maybe it's when they go to outside refs that problems are more likely to set in? I don't know.

[L. A. Paul]

Keith's suggestion about Phil Review and outside referees is a good one. The two people I mentioned in my comment above were writing somewhat technical papers in areas like metaphysics, logic and science. In the interest of full disclosure, I should say that neither of the authors of the revise-and-resubmits I mention above is me—but I did have a bad experience with Phil Review where it took them a year to reject a paper of mine. I didn't mind the rejection as much as I minded the long wait: for junior faculty, a year's delay is serious!

[Jason Stanley]

Keith,
I think you're somewhat delicately ignoring a clear consensus: everyone agrees, whether explicitly or implicitly (by not mentioning them, as you have done), that Mind is truly exceptionally remiss.

[Manuel Vargas]

Although my experience parallels much of what has been posted, I wanted to make mention of speed issues at the journals with which I've had experience:
Analysis is extremely fast (they usually take only 2-3 weeks to tell me no).
Phil Studies is pretty quick (in three out of three cases I got notification of an acceptance within three months of the original submission, and that includes an intermediate revise/resubmit step for at least two of those papers.) They have also always given me useful feedback, so I'm a big fan.
AJP, Phil Quarterly have also been pretty fast (under 3 months) when I've submitted.
Ethics can be pretty slow (8 months on a paper I submitted), but the referee reports were mostly helpful when they did finally come back.

For what it is worth, I've found that self-reporting by journals about their average time to decision tends to be significantly under what I've experienced and what I'm told by others about their own experiences (though Analysis may actually overstate their average time to decision). It would be interesting to try and build a database where people volunteered information about when a paper was submitted at a given journal and when they received a decision on that paper. Maybe someone with more coding skills than I have could give it a crack and see what happens.

[Michael J. Shaffer]

Isn't the ultimately important issue one of why the various journals mentioned have extra long response times, extra fast ones, etc.? Presumably, while it is important for junior faculty to publish relatively quickly, the more critical issue for our field is what to do about journals that have long response times and/or absent or poor commentary for authors. This sounds like an excellent thing for the APA and other organizations to investigate so that effective advice can be given to improve journals, especially as they are the medium in which much of our work is done.

[vm]

I sent something to Pacific Philosophical Quarterly once, and they seem to have lost the manuscript. A year later, they had no idea what had happened to it, although they reported receiving it initially.

[Clayton]

I've had excellent experience with Analysis rejections. While the post office assures me that it takes at least 7 days for submissions to make it to the editor's desk in England, he's managed to reject both of my attempts within 6 days. Remarkable efficiency.

[Alva Noe]

These horror stories confirm me in the view that journals play too big a role in the tenure process. Rejection from a strong journal does not reveal much, if anything, about the strength of submitted work. It's just the nature of the beast that there is no way to put constraints of fairness on the review process -- in philosophy, orientation and taste can't be factored out. And then there's inefficiency and tardiness in the process (often caused by the tardiness of the referees, I'd bet). The end result: a person's publication record at tenure time may not be a very good reflection at all of overall quality or productivity.

It's worth considering that the problem is not really with the journals. Sure, it would be nice if they were all smooth running machines. But they're not. For the most part, they are "projects" of their editors and why shouldn't the editors be somewhat arbitrary and relaxed? What is outrageous is the de facto role universities give them in deciding whether someone deserves tenure or not. (In the UK, with the RAE, the results are even more outrageous).

A practical suggestion: I publish quite a bit in psychology and cognitive science journals, and I have the sense that the mood is a bit different in those quarters. There's just a bit less angst attached to the review process. I think there's a sense that it's a valuable part of academic exchange. A lot of reviewing is not blind, which has the effect of making the review process more like a correspondence between author and critics, and tends to raise the level of civility without sacrificing critical value. In addition, review turnover is just much faster. I think that's partly because in the sciences everyone is racing to get to print so they can take credit for ideas. Whatever the reason, the result is a less stressful, more streamlined process, that is no less intellectually rigorous. My suggestion? (1) More non-blind reviewing; (2) one week deadlines to reviewers.

[ccs]

For those interested in the topic, the people at PEA Soup had a good post and exchange concerning the issue of the ethical obligations of journals vis-a-vis authors:

http://peasoup.typepad.com/peasoup/2004/06/the_ethical_obl.html

[Mark Moyer]

A friend relates the following:

"Among my three horror stories about Mind, here is my favorite: After 18 months and several emails checking up on the status of my submission, I received an email saying that the editor had decided not to publish my paper and that comments, along with a formal report, would soon be sent to me by mail. After 8 more months had passed, I received an email with the referee's comments. On the basis of the partly positive comments, the editor instead invited me to resubmit the paper."

[David Velleman]

I hope you won't mind my entering a plug for Philosophers' Imprint. Our response time has been very good thus far, and we offer the advantage of publication within weeks of acceptance. (Each paper is published as soon as it is ready, without waiting for space in a scheduled volume.) We occasionally have trouble reviewing a paper as quickly as we would like, especially if the paper is very long or is on topic for which referees are scarce. But we have provided referees' comments to the vast majority of the authors whose work we review, and our referees are (if I may say so) very astute. Junior faculty who have worries about how online publications will look in their tenure dossier should contact us for further information. A look at our contents will confirm that a number of junior faculty have regarded the Imprint as an advantageous venue for their work.

[Patrick Hawley]

Mind held one of my papers for 18 months, and then rejected it
without any reviewer comments at all. (They said that the reviewer
"has promised to deliver in the near future". Ha ha.)

[Jessica Wilson]

I had almost exactly the experience with Mind that Sergio had ("I submitted a paper to Mind and 10 months later I got an R&R. I revised and resubmitted [...] 12 months later I got a rejection with a one paragraph useless referee report"). One bright spot: the first referee report was excellent. Even so, on the basis of my and others' experience my advice to other junior faculty is to avoid (submitting to) Mind like the plague.

So far, I've had decent turn-around times (under 6 months), timely return correspondence, and excellent referee and editor's comments from Nous, Phil Quarterly, Philosophy of Science, and the Canadian Journal of Philosophy.

[David Sobel]

First five recommendations (shamelessly stolen from the Pea-Soup blog) then a plan of action:

(1) Journals should publish statistics on the average (actual, not hoped for) time between the receipt of a manuscript and the notification of the author of the initial decision.

(2) Journals should publish statistics on the time between the acceptance of a manuscript and its appearance in print.

(3) Journals should make public the details of the journal's editorial procedures, as Analysis, the Philosophical Review, and the Australasian Journal of Philosophy do on their respective web sites. (These sites describe, for instance, the various stages of the review process, how decisions are reached, and whether the process is blind or not.)

(4) Journals should keep the requirements in preparing manuscripts for submission to a bare minimum. Authors should not have to ensure that their manuscripts conform to the journal’s style at the submission stage. Of course, some requirements that will make the review of manuscripts easier on editors and referees -- such as, double-spacing and wide margins -- are in order.

(5) Journals ought to notify authors of receipt of their manuscripts and where possible indicate the approximate time needed for evaluation. After this period of time, inquiries from the author concerning the status of his or her submission should be met with a prompt and courteous reply.

I think we should seek to get a number of philosophers to sign a statement saying that we will, in one way or another, boycott journals that do not comply with a list of demands within a specified time frame. I believe we could get enough philosophers to sign such a thing that it could make a difference to how the journals conduct themselves.

Here is one specific proposal, but the general idea could take one of many forms. I say we come up with a list of the most basic and generally accepted demands to place on journals (I would urge using the 5 recommendations above). Then we ask a wide array of philosophers to sign a statement saying that if a journal does not comply with these demands, the people that sign will no longer referee papers for that journal. To be effective, we would have to get a lot of people to sign and if many of them were prominent, that would be even better. I think we should give journals something like 2 years to make the changes. One could, although I fear we would get fewer to sign, also pledge to no longer submit papers to the journals that do not comply.

In sum, comrades, I quite seriously urge that we take to the (metaphorical) streets. I urge that we not merely make a good case for change but find plausible ways to effect the warranted change.

[Keith DeRose]

In response to Alva Noe's comment (a few comments above) I'd like to stick up for (a) the use -- appropriately limited -- of a person's publication record for various purposes and (b) the practice of blind review. I'm pushing at least vaguely in the opposite direction as A.N., then, but wrt (a), we may not really disagree that much.

For some important purposes, there really is no substitute for actually reading a whole lot of a philosopher's writing. For instance, at least in the final stages of a hiring decision this should be done, and as well in making a tenure decision -- which, I should note, is the purpose AN specifically mentions. To the extent that publication record -- which I'm here taking to mean one's list of publications; how many & which journals: what one can tell by looking at the CV -- at all crowds out such a careful look, that's a very bad thing, and to the extent that's AN's point, I certainly agree with him. By the time you take a look at a tenure candidate's material that's as close as that situation deserves, the issue of what journals the candidate was able to get that work into should usually drop out as irrelevant. (In extreme cases, if the candidate consistently fails to publish at all or consistently publishes in very bad venues, even though the work is excellent, there's probably a problem there in publication strategy that should be addressed.)

But for other purposes, I think publication record should be given a larger role than it is presently given. For instance, in the hiring process, you simply can't (or at the very least, very few departments actually will) read a whole lot of writing by every applicant. There has to be some narrowing down process. And in this narrowing down process, publication record too often (in my view) gets trumped by other factors that are much worse guides than publication record, like, for an important example, institutional affiliation.

So, here comes my complaint about our profession. It really isn't sour grapes: I am very happy with how I personally have done as far as the jobs I've had. But I've known many (& have known of even very many more) very talented philosophers who have been put in a situation that's very tough to dig themselves out of. We as a profession aren't very good at discerning which 22-year-olds graduating from college are likely to be the best philosophers, and not that much better at discerning this when it comes to 28-or-so-year-olds coming out of graduate school. But which graduate school one gets into and what job one initially lands tragically does very much to determine how well one is likely to do, long-term. It often happens for instance, that extremely talented philosophers who deserve to do as well as those landing the great jobs instead end up at some low-prestige job with a heavy teaching load. Every now and then, one of them quite heroically overcomes the odds of having to write while teaching so much and puts out a bunch of excellent papers in really good journals (which at least often they're able to do largely b/c the journals use blind review!). But, too often, they can't get the people with the power in the profession (& who know that the candidate works at a low-prestige place) to take their work seriously. They loose out to candidates (the "chosen ones") who, despite their very cushy teaching loads, publish little in good journals but who have something that all too often proves more valuable on a CV: a high-prestige institutional affiliation. In my view, this is a very bad thing.

We have lots of extremely talented, but highly underemployed members of our profession. It's important to provide some way by which they might possibly dig their way out of that hole. One important way is to have lots of good journals with blind review, and for all of us to resolve to take publication records very seriously. This doesn't mean to hire someone based only on the quality of the journals that have published their work. But I am thinking it does mean something like this: If someone develops a good publication record, as you can tell simply by looking at their CV in a hiring situation, or as you should notice if they're publishing in your particular field, take that as strong prima facie evidence that they are doing excellent work, and then take a good look at their work. I'd also suggest, in a comparative vein, to take a strong publication record as a stronger reason to take a close look than that some other philosopher in a high-prestige job, and therefore well-connected with high-prestige friends, gets a lot of good word-of-mouth.

Despite the very real weaknesses in the process by which journals choose which papers to publish, reading around in the best journals (at least those papers in your area(s)) is a good way to read a lot of very good stuff, and if we all did this & became aware of such good work, that might also result in the goodies of our profession being distributed a bit more fairly.

That so many of our best journals use blind review, and that so many do as good a job as they do in the selecting process is one of the best things about our profession.

[Alva Noe]

In response to Keith DeRose: I agree almost entirely with what he writes. First, I actually think that a successful publication record is an important (of course defeasible) indicator of excellence. My point was that *absence* of a successful publication record in a younger scholar is not a reliable indicator of lack of excellence, other things being equal. I know of more than one case where a tenure situation was tipped one way or the other by the happenstance that a journal reached (or failed to reach) a decision by a certain deadline. My concern is not with what publication reveals, but with what failure to publish --especially within a pretty short time frame -- reveals.

Second, I also strongly agree that we should give much more attention to the quality of a person's work (for example, as expressed in his/her publication record). It is too rare that people are able to "move up" in the profession (or move down).

Is this last end best achieved by blind review? I can certainly see that there are arguments in favor of this. After all, if you don't know who you're reading, how can your judgment of what they write be affected by prejudice? In the best of all possible worlds, this would be the best way to do things.

But consider: we don't live in the best of all possible worlds. First, true double-blindness is harder to achieve than we might have thought, especially if you are active in the profession. Second, I find that referees are often protected by their concealed identity; it saves them from having "peer review" of their own evaluation of other's work. In journals where the process is open, there are checks and balances on arbitrary evaluations -- others learn what judgment you passed. (I realize the editor is supposed to serve this function -- but does he/she?) Third, I fear it creates the idea that secrecy and lots and lots of time are needed to justify a recommendation to the editor. I realize I'm not really giving much of an argument for this last point. I just have this sense that the "vibe" in the social science world is better and more open.

What prompted my comments about blind review is the concern that blind review actually serves to protect journals from taking the heat for their more or less arbtrary decisions.

[David Velleman]

Random comments.

Some of the discussion about hiring and tenure decisions strikes me as losing sight of the role that administrators play. Most departments make these decisions with one eye on the administrators who will have the ultimate say. Once in a long while, a department may be obligated to fall on its sword for a candidate who will never get past the college executive committee or the dean; but departments can't go falling on their swords every season. In general, then, they have to recommend the appointment and promotion of candidates whose qualifications can be measured in terms intelligible to non-philosophers. And publication in refereed journals is the generally accepted measure. It's a bad system -- except that all of the alternatives are worse. What other measures would you like the dean to use?

The exchange between Keith and Alva brings out an important distinction between author anonymity and referee anonymity. The former is essential the integrity of the process, in my opinion, but the latter may in fact be detrimental. As Alva suggests, the tone and constructiveness of referees' reports would be greatly improved if they were signed. Referees should be permitted to ask up front for anonymity, but signing their reports should be the default. (And editors should consider passing up a referee who asks to remain anonymous.)

I wouldn't support David Sobel's proposal for a boycott of journals that do not accept some pre-defined terms. People should choose their venues and their refereeing assignments with an eye to the relevant qualities of a journal; but different people will regard different qualities as relevant. Myself, I refuse to referee for any commercially published journal; and I try very hard never to publish in commercial venues, either. (The latter policy can be awkward in the case of conference proceedings, symposia, and the like, since other authors are involved.) In order to support non-profit and open-access publications, I'm willing to put up with suboptimal editorial policies. Others are willing to deal with commercial publishers in order to get better treatment from editors. To each, his or her own.

Finally, although I have had my share of bad experiences with Mind and the Journal of Philosophy, I'm reluctant to condemn the editors. Having taken on the task of editing a journal, I've discovered that it is very difficult to do well. One needs the cooperation of many people -- referees, above all -- and one cannot be more prompt or thorough than the general run of referees.

[Michael Cholbi]

I and the other contributors to PEA Soup had a discussion about this some time ago, which resulted in something akin to an 'author's bill of rights.' (http://peasoup.typepad.com/peasoup/2004/06/the_ethical_obl.html#comments)

My greatest frustration with journals is the lack of sympathetic and objective consideration of submissions, and a general lack of editorial judgment or care. I once received back two sets of reviewer's reports, one positive, one mostly negative, and the associate editor said she agreed with the first reviewer that the paper was "original and carefully argued" and with the second reviewer that the paper addressed its topic "in a way that is wholly familiar from the literature." In another instance, a managing editor rejected my submission because it argued against a claim he had argued for in his published work. A reviewer at another journal later described that same claim as "extremely problematic at best." (Both of these were with Ethics, btw.) I'm untenured, and I get the impression from my senior colleagues that journals at one time took more seriously their duty to further the professional development of those who submit manuscripts by providing extensive, constructive feedback that responds not only to a paper's weaknesses but also to its strengths. My feeling is that publishing competition is now so intense that journals no longer see themselves as able to fulfill that duty, or even as having such a duty. Perhaps the blind review process encourages this trend?

Let me also plug for the Southern Journal of Philosophy; it's not a top-tier journal in terms of prestige, but in my experience they review quickly and fairly, and publish expeditiously as well.

And to Keith: Here here. Why should the 5-7 years that mark the beginning of a philosopher's career have such a great impact on the trajectory of a career that will likely last 40 years?

[Keith DeRose]

The exchange between Keith and Alva brings out an important distinction between author anonymity and referee anonymity.

That is an important distinction. When I wrote of "blind review," above, I was thinking only in terms of author anonymity. As for referee anonymity (weren't there some letters a while back in the APA Proceedings about this?), I agree that there are good reasons to have signed reviews, but have this major worry about it:

I don't find refereeing to be enjoyable. I'd much rather have others be the gatekeepers, & I could just read the papers that pass the test in the journals. I do a good deal of refereeing, but I do it out of duty: I think doing my share of that important task is part of being a good citizen. I suspect that most referees share my feelings here. If so, we really have to worry about what would happen if we added a substantial cost to a task that people are already doing out of the goodness of their heart. And in the cases where your decision is that a paper shouldn't be published, there is a real cost to being an identified referee. It's a good way to make enemies in the profession, and who needs that? I fear it would become even harder to get qualified refs if they were identified.

[Jamie]

On anonymous refereeing...

Here's a compromise solution that might address Keith's worries: a rejection can be anonymous, but an acceptance cannot.

Geoff Pullum once suggested (in one of the"Topic... Comment" essays, reprinted in The Great Eskimo Snow Vocabulary Hoax) that every published paper should have the name of each referee who accepted it included at the end. One effect would be to reward the good citizens who referee a lot of papers, especially insofar as they accept very good ones. Another would be to cast some light on academics who recommend their friends' papers for publication -- though I have no reason to suspect that such a thing happens in philosophy, Pullum apparently did suspect that it happened in linguistics.

I understand that this second suggestion might tend to discourage positive verdicts, but maybe that's not such a bad idea.

[Robert Northcott]

I would guess that the bottleneck in the system is often a paper sitting neglected at the bottom of a referee's in-tray for months. The weak link here may be the reliance on referees' goodwill, a reliance that might work fine on a small scale but that is likely to become increasingly problematic with the ever stiffer competition for publication and hence the ever more refereeing required. Indeed previous posts, and the general interest in this thread, suggest that it has become problematic already. In our imperfect world it seems to me there will always be a limit to how much even the best of editorial intentions can achieve on their own without also addressing the situation from the point of view of referees. (Of course this is no reason not to push for better editorial behavior too.)

Here's one thought. To improve matters, maybe referees should be paid? Payment presumably conditional on reporting within a certain time period (6 weeks?) and also up to a certain standard, as judged by the editor.

Of course, an immediate problem is where the money would come from. A 50 dollar submission fee would at least generate 25 dollars for each of 2 referees - OK not much, but perhaps enough to make some difference or to prod a referee's conscience. And if nothing after 6 weeks, a new referee could be used. Judging by some of the sentiment expressed here, writers might consider such a submission fee to be money well spent if in return they could feel confident of a quick and informative response, in which case it would not be a discouragement to submit.

Admittedly this particular proposal sweeps various details under the carpet, but I think the underlying point remains true - that there is a limit to what can be done until we address the position of referees more deeply than just by making moral appeals to their goodwill. Or am I (as an ex-economist) too cynical?

Incidentally, my own anecdotal impression is that in other humanities and in (non-psychological) social sciences, the situation is worse than in philosophy.

[Andrew Jorgensen]

RE possibly charging per submission.

The trouble with this idea is that money is not an issue for some (like faculty and graduate students from well funded departments who would bear the cost as a research expense on behalf of their staff and students) and is for others (like poorly paid adjunct faculty at teaching focused institutions). So it would contribute to some of the social inequalities mentioned by Keith De Rose by bringing in a factor that's really irrelevant to philosophical quality.

[Tad Brennan]

It is very important that authorship remain anonymous. Being able to judge a paper solely on its merits, without any clue as to authorship, has several times led me to reject a paper that, I later found out, was written by a friend or colleague. Had I known who the author was, my decision would certainly have been complicated.

Had I been required to sign my rejection, I would have refused the commission outright. As Keith says, refereeing is thankless and joyless enough already. Requiring signed rejections would dry up the referee pool overnight, I think.

I am more open to the possibility of signed acceptances. Something like this has occurred in my case both when I have been the author and when I have been the reviewer, i.e. a paper is submitted, the referee writes extensive and helpful comments, the author responds to them, an anonymous correspondence ensues through the editor, and at some point by mutual agreement the author and referee begin corresponding directly. This also allows the author to acknowledge the referee's assistance by name. Because it happens in the context of an acceptance, all is sweetness and light.

It has been usefully noted by some recent posts that refereeing is the bottleneck in the submission time-line. One point that has not come up yet, so far as I can see, is that referees are now over-loaded by an excess of bad or prematurely submitted papers.

We can see how this happens by reading the comments on another of Brian's posts, which advocate that students should start submitting their papers in graduate school. It may be good for some grad students, but this proposal takes no account of the social cost to the profession of having the journals flooded by lot of bad grad student papers.

I find it irritating to referee papers that read as though they were written for a grad seminar, received a passing grade, and were then mailed straight to a journal. When I find that I have agreed to referee one, I still feel bound to give it a full read and write out a full account of why it should not be published. (And of course I can never *know* that I am refereeing a grad seminar paper as opposed to the work of Herr Professor X). But as I am writing my page or more of comments, I sometimes feel like the earnest TA who spends one hour grading a paper that the freshman spent a half-hour writing.

In the old paternalistic system, the solution would be obvious: journals would allow submissions from grad students only with the permission of the student's supervisor. No rule like that would find general acceptance nowadays, and for good reason. Advisors have blind-spots and partialities, and students are free agents. But I still think that in some cases, perhaps a majority of cases, the submission of half-baked papers represents a failure of supervision. Because you did not give your student sufficiently frank advice, you shift the burden onto me as a referee.

It is very easy to tell students to begin submitting early. "...and if your paper isn't accepted, well, no harm done--it's a valuable learning experience either way!" Well, perhaps no harm is done to the student--but in aggregate it leads to exactly the harm that has been detailed in the complaints that have filled up this thread: referees are overwhelmed and fall behind, begin to write shorter and shorter comments, and make their judgements with less thorough study and greater impatience. So journals look inefficient and arbitrary.

Could it be that some of the posters who have complained about these deficiencies in the journals really ought to have been advised not to submit that paper to begin with?

[James Lenman]

By miles, my best experience with a journal recently was with PPR in virtue of an exemplary turnround time for a decision (less than three months) and a referee's report of exceptional sharpness that resulted in very substantial improvements to the paper.

Like others I don't have good experiences with 'Mind'. Rather further down the pecking order the Israeli journal 'Philosophia' does not appear to be well managed.

'Philosophy and Public Affairs', from whom I've had a mixture of good and ill fortune recently on the acceptance front, though it doesn't usually seem to give much by way of comments on rejections, seems to be pretty efficient and has never kept me waiting too long.

I suspect there is a tendency for some the few journals at the very top of the pile - the sort of journals you try desperately to get into if you want tenure at a relatively top-drawer US university or a permanent post somewhere good in the UK - to rather rest on their laurels taking their contributors for granted and be very much less considerate of them than might be hoped. Others - I suspect this is true of the Phil Quarterly widely praised here - try to raise their own relative standing by cultivating a reputation for the opposite behaviour. WHile accepting David Velleman's point about the difficulties of a journal editor's life, it would be thoroughly satisfying to see the former losing some ground to the latter - especially as very high status journals tend to make a fair bit of money and can well afford to give their editors very good administrative support. So I think there's something rather healthy about the naming and shaming (or the opposite as the case may be) afoot here.

(Charging for submissions would constitute an enormous and profoundly depressing change in the whole culture of philosophical publishing and is surely a council of despair to be shunned execpt in circumstances far more desperate than can at all plausibly be said to prevail.)

JL

[Matt]

Somewhat late now, but let me put in a plug for the new Journal of Moral Philosophy, which I recently had experience with. They were quick, easy to deal with, the editor, Thom Brooks, was helpful and responded quickly to emails, and the comments were at least somewhat helpful. It seems like a good place to try.

[David Velleman]

Combining the comments of Robert and Tad:

One might try to encourage some self-selection on the part of submitters by using the medical "co-pay" model -- a fee just high enough to provide a disincentive to frivolous use but not so high as to constitute a barrier. Maybe $10?

Of course, splitting that $10 between two referees would be ridiculous. Clearly, then, the money should go into the pockets of the editors.

[Alva Noe]

Many interesting comments. I return to David Velleman's a few posts ago. What other measures would I like the dean to use in evaluating tenure cases? I certainly wouldn't say that no weight should be given to, say, the absence of publications, or the absence of a lot of publications at good places. But I would have thought primary weight should be given to the evaluation of the quality of the work by established philosophers participating in the tenure review (an evaluation that is likely to shed light on the significance of the publication record). Here I echo Keith's comments from a while back. Once the work has been read and evaluated by a sizable number of reviewers, and then interpreted by the candidate's faculty, it's hard to see how it can be permissable to allow administrators to place real weight on the "publications list" as a free-standing criterion.

What motivated my comment on this is the evidence -- assembled anecdotally in this discussion -- that the process whereby articles are selected for journals is frequently unfair. I am skeptical whether we can change the behavior of journals -- or whether we are entitled to demand that they change -- but we certainly can have a more clear-eyed conception of what a publication record tells us.

I'll go back to the drawing board on the question of blind review. I'm impressed by the objections that have been raised against my suggestion. I will repeat one thing, though: light needs to be thrown on the process. I fear that, in addition to arbitrariness and tardiness and unresponsiveness, there's a fair bit of insiderism and nepotism.

[David Sobel]

David Velleman wrote above:

"I wouldn't support David Sobel's proposal for a boycott of journals that do not accept some pre-defined terms. People should choose their venues and their refereeing assignments with an eye to the relevant qualities of a journal; but different people will regard different qualities as relevant. Myself, I refuse to referee for any commercially published journal; and I try very hard never to publish in commercial venues, either. (The latter policy can be awkward in the case of conference proceedings, symposia, and the like, since other authors are involved.) In order to support non-profit and open-access publications, I'm willing to put up with suboptimal editorial policies. Others are willing to deal with commercial publishers in order to get better treatment from editors. To each, his or her own."

I understand David's unwillingness to sign on for the sort of coordinated attempt to have journals adopt the sort of policies I outlined in my post (again, stolen from a Pea Soup post by Dan Boisvert et al.). He has other priorities that are more important to him in such matters. I think David's priorities are sensible and well worth promoting.

But I am not sure I yet understand why he would be against others signing on for the sort of coordinated recommendation/threat to journals of the kind that I outlined. Perhaps he is not, and all he is telling us is that he will not sign, but not speaking against others signing on. But perhaps he is saying that his priorities are more important than the one's I outlined and we should not sacrifice that cause for the cause I outlined. But I doubt he means this as then he would not end with "To each, his or her own." Perhaps he is saying that the priorities I championed are not sensible or worth promoting, but I have not heard him say that yet.

Thus if David means to speak against others signing on for the sort of plan I mentioned above, I would like to hear more about why he is against this.

I take it we are generally agreed that it would be desirable, other things equal, if journals would tell us how long we can expect it to take for them to make a decision on papers, etc. Yet journals frequently do not do this, despite it being easy to do so. I imagine in some cases they fail to do this at least partly because an accurate assessment of time to decision would lead us to send our papers elsewhere. This creates real problems, especially for untenured people seeking to get enough papers published for tenure. I am seeking a way we might make the situation better in this respect. If others have better plans for how to accomplish the goals that most of us seem to share, please speak up.

[Eric]

I just stumbled upon this extremely helpful conversation one week too late! I just mailed a paper off to Mind, and I've got about a year before my tenure case comes up. Oops. I don't think the paper's publication is necessary for promotion, but every little bit helps.

Advice, anyone? Should I pull it now? Pull it in three months?

I can also attest that PPR, PQ, and Analysis (since they changed editors about five or six years ago) are model journals. Phil Studies rejects reponsibly!

[David Velleman]

David Sobel writes:
I take it we are generally agreed that it would be desirable, other things equal, if journals would tell us how long we can expect it to take for them to make a decision on papers, etc. Yet journals frequently do not do this, despite it being easy to do so.
In my (admittedly limited) experience as an editor, there is no way to predict how long it will take to referee a given paper. Sometimes I can get a paper refereed within a few weeks. Sometimes I can't get it refereed for months -- more months than it should take.

Often the delay is due to the editor's hope of serving some of the goals mentioned by Michael Cholbi. Here's a typical case. A paper is submitted by a junior faculty member from an out-of-the-way department. The paper looks rough around the edges but promising -- potentially publishable if revised with the right assistance. The editor asks two highly qualified experts to referee the paper, hoping for the sort of insightful commentaries that will help the author to make the paper publishable. The experts accept the assignment, one of them reports within two weeks, but the other sits on the paper for three months, repeatedly promising to report "next week" but never following through. Should the editor start over with a new referee? The new referee might sit on the paper for another three months, whereas the original referee may now feel guilty enough to get moving at last. But maybe the original referee is a lost cause and starting over with someone else would be best. Then again, the advice of the original referee really could be of value to the author -- maybe worth waiting for just a bit longer?

Sometimes the editor guesses right: he's able to give the author useful feedback in a reasonable amount of time, the paper is revised successfully, and it gets published. Sometimes the editor guesses wrong. In either case, the editor is guessing.

I would be happy to tell authors in advance how long it will take to referee their papers -- if only someone would tell me how to predict the behavior of my fellow philosophers!

[David Velleman]

In response to Alva Noe: I don't think we disagree. The opinions of outside referees should of course be central to a tenure case. But administrators are often on guard against a department's stacking the deck of referees; they want some additional, independent confirmation. One way that administrators try to prevent stacked decks is by requiring an absurd number of outside letters -- a practice that places a temendous burden on the profession, since refereeing a tenure case is far more time-consuming than refereeing an article. (If every university required as many outside letters as Stanford, the entire discipline would probably grind to a halt.) The peer-review process of refereed journals offers a preferable way of double-checking the judgment of outside referees. But I completely agree that it should be a secondary check, not the main basis of judgment.

[Robert Allen]

Professor DeRose,

I see that you not only have a very charitable view of how things should work in heaven- "The Really Good News"- but also here on earth:

"I've known many (& have known of even very many more) very talented philosophers who have been put in a situation that's very tough to dig themselves out of. We as a profession aren't very good at discerning which 22-year-olds graduating from college are likely to be the best philosophers, and not that much better at discerning this when it comes to 28-or-so-year-olds coming out of graduate school. But which graduate school one gets into and what job one initially lands tragically does very much to determine how well one is likely to do, long-term. It often happens for instance, that extremely talented philosophers who deserve to do as well as those landing the great jobs instead end up at some low-prestige job with a heavy teaching load. Every now and then, one of them quite heroically overcomes the odds of having to write while teaching so much and puts out a bunch of excellent papers in really good journals .... But, too often, they can't get the people with the power in the profession (& who know that the candidate works at a low-prestige place) to take their work seriously. They loose out to candidates (the "chosen ones") who, despite their very cushy teaching loads, publish little in good journals but who have something that all too often proves more valuable on a CV: a high-prestige institutional affiliation. In my view, this is a very bad thing.

We have lots of extremely talented, but highly underemployed members of our profession. It's important to provide some way by which they might possibly dig their way out of that hole."

[Saul Smilansky]

The consistency of people's experiences with the various journals in the postings is very large, and the more such knowledge is spread around, the better. Australasian and Phil. Studies deserve, in my experience, the positive reviews they have been getting, and have the added benefit of accepting papers electronically, a practice which surely could be more common. Analysis is the most amazing journal and puts others to shame in terms of efficiency. APQ is also fairly quick, and I would like to put in a good word for two journals which haven't been mentioned, the British journals Philosophy and Ratio (although the first tends not to give comments).

[Keith DeRose]

Professor DeRose, I see that you not only have a very charitable view of how things should work in heaven- "The Really Good News"- but also here on earth

Alas, here on earth, things are not all sweetness and light. If things went the way I suggest, different philosophers would get the good jobs. Focus on those who would do better, and the suggestions may look charitable, but those who would do worse would not find them so.

[John Lamont]

On the question of the role of publications in hiring and tenure decisions - does anyone know whether citations of publications are taken into account much, as I gather they are in the sciences? I thought this would provide some reflection of the quality as opposed to the quantity of a person's work. (I say this despite the fact that the few things that have been published about my own stuff have reflected intense irritation about the theses I have maintained, which is not a very good sign with respect to the quality of my work.)

[Robert Allen]

Prof. DeRose,

Those who would do worse would have no reason to complain if the facts are as you maintain. Also, Universalism is consistent with there being an order of rank in heaven. But those of us who are underemployed are not necessarily looking to supplant those holding prestigious positions. I for one would be happy just to be able to support my family and write more. (Who needs prestige if you have philosophy?) A full-time job at the lowliest community college in the country would suffice. What I was trying to say is that your belief that every philosopher who is qualified to teach philosophy should have a full-time teaching position at some institution of higher learning was charitable and of a piece with your view on salvation.

[Tad Brennan]

re John Lamont's query: I have done a lot of hiring, no tenure review. I can tell you that I have never heard any numerical discussion of how often an article has been cited. This for the simple reason that there is no centralized count of citations in philosophy, as there is in the sciences and social sciences with their citation indices. Informal and anecdotal impressions do of course enter in--that paper made a big splash, people are talking about her book, that piece took a lot of flak, and so on. But numbers to back up these impressions simply are not collected--so I assume the picture must look the same at the tenuring stage, too.

In fact, I assume this is part of why tenure reviews need to collect so many letters--they attempt to measure the same thing citation indices attempt to measure, i.e. the impact of publications on the profession. Would philosophy be better off with a citations index measuring actual footnote-occurrences? I don't know. A former colleague at a former job once tried to convince us of the greater importance of senior candidate A over senior candidate B by telling us that A's name produced many more google hits than B's name did. On this basis we almost voted out an offer to Britney Spears, but there was some concern about her teaching record--who says teaching doesn't count at the senior level?

On Keith's soteriology--I was teasing Keith in a separate email about the implications of his saying that some philosophers are "underemployed". He has come clean now: he may be a universalist about post-mortem salvation, but when it comes to good jobs he is a zero-sum man. (And fair enough--there's a premise in the theological position about the existence of a perfectly benevolent being, and no such beings are to be found in academic hiring). So now that you've come out to the public, Keith, I put it to you again: what's your top five list of the *over-employed* philosophers who ought to make way for those other deserving souls who are struggling through purgatory? Since we have been trashing journals by name, certainly we can trash colleagues by name?
Oh--and Brian, could you change this thread to allow anonymous posts?

[Tom Hurka]

About citation counts for philosophy: they do exist, in the Humanities and Social Sciences Indexes compiled by the ISI, accessible at isi6.isiknowledge.com. And though I doubt this is very commmon, I've had departments look them up in my case, when doing senior hiring. I actually think they have some use at this level. But they're much less appropriate for junior hiring and tenure decisions, just because citations take a fair amount of time to appear. Consider: X publishes an article in 2004. Y has to read it and then write his own article citing it (the ISI tracks only citations in journal articles, not in books), which can take into 2005; then Y has to get the article accepted by a journal, which can take into 2006 (see posts above); and then the journal has to print it, which can take another year. So only articles published relatively early in a person's tenure review period have much chance of being cited by the time of the tenure review. Citation counts are informative for senior philosophers -- for fun, look up Rawls's count -- but much less so for junior people. That said, at least at the senior level they probably deserve more attention than they now get. (The point is a little like Keith DeRose's about blind review: the counts track impact on all philosophers everywhere, not just your cronies in a few name departments who will say how great you are.)

[Benj]

Here's my Mind war-story: the first review took eight months. To its credit, this generated some excellent comments from Mike Martin and some, ah, less-than-excellent comments from the second reviewer (apparently Martin got his comments done speedily).

At roughly the end of the tenth month of the second review, my monthly prodding email to editorial secretary Rachel Carter caused her assistant Lin Fou to notice that the second reviewer had, at some time (they wouldn't say what) during the ten month interval, come to instantiate a certain property F (which I won't disclose to protect the identity of the guilty) such that no one with F would ever complete a review for Mind. My paper would have to be reassigned.

The epistemic failure here was not a failure to notice that the second referee had become F: the editor of Mind always knows exactly who is F. Nor was it to notice that someone who is F will never complete a review: this is obvious. Rather, the failure was to notice that my paper (among, perhaps, others) had been assigned to a person who had become F (I advise anyone who has a paper out at Mind to email Carter or Martin and ask for assurance that their paper will at some---far future, of course---point be reviewed). I think it fair to label this an instance of organizational bungling/indifference verging on the immoral.

That makes 18 months, and going back to a third reviewer who is looking at it cold. Would the editor assure me of some date by which a verdict would be reached? The editor would of course not personally respond to requests at all, but the editorial secretary was perfectly willing to respond with chilly boilerplate about asap and all that. A second request for greater specificity has now gone ignored for two business days.

Fortunately this thread has given me a good sense for where to send the paper now that I've steeled my resolve to divorce Mind.

[Thomas Hofweber]

Benj, that all sounds terrribly familar to me. I have a paper at MIND which was resubmitted in March 2003. That is 21 months ago. No word on a decision, standard email replies with no details, etc. In fact, the paper has been at MIND for close to 3 years overall. It seems almost comically absurd to me now that I think about how long that is. I hardly even remember what I said in it, and I even need to be reminded from time to time that it still is sitting there somewhere. I guess I have been easy on them since this happens to me frequently. I would say at least half of the time I submit something it takes a year or more before I first hear back, and in conversation I hear that this happens a lot to others to, so I guess I just got used to that being normal. I think it really is time that I pull it and send it to the Philosopher's Imprint instead....