Now that David Wallace (Oxford) has destroyed the purportedly data-driven analysis in the Bruya critique of the PGR (and here), confirming with damning details the crux of my initial criticisms and adding several other important criticisms, what are the takeaways from this sorry episode?
I begin by observing that anyone who actually read the first few pages of the Bruya paper would have immediately heard the sound of an axe grinding, a clear sign that one needed to examine carefully what's to come. As Prof. Wallace's analysis has now established, whenever Bruya had a choice to make, he opted for massive inflation of the categories into which he put the data for analysis: doubling the size of the "metaphysics & epistemology" category, for example, by arbitrarily incorporating all of the specialties in philosophy of the sciences, math and logic into that category; and treating institutional affiliation as meaning either the school at which the evaluator currently teaches or the school from which s/he received the PhD, thus massively inflating the institutional affiliation numbers for schools that have granted lots of PhDs over the past several decades. In both cases, the fabrications were not acknowledged clearly in the text or even sensibly defended; the fabrications were buried in appendices and footnotes. And in both cases, the fabrications biased the purportedly "data driven" analysis of bias, as Prof. Wallace has shown. Between the fabrications, the mathematical mistakes (noted by Prof. Wallace) and the defamatory racism smear, it was fairly obvious that something was, shall we say, amiss. But it's quite clear most of those enthusing about the paper on social media had not read it, and simply repeated its self-advertising as a "data-driven" critique, which we now know was a bit of false advertising.
So here are a few lessons I think we can learn from all this:
1. Peer review isn't all that it's cracked up to be, at least at some journals. Metaphilosophy is now on notice that something went very wrong, and the journal's reputation now hangs in the balance. Between the defamation and the fabrications and the mathematical mistakes, someone wasn't really paying attention--or perhaps the referee (assuming there really was one, I don't know) shared the relevant bias so was only too happy to give the paper a pass, despite its incompetence.
2. If philosophy is still a Wissenschaft, then it should be possible for philosophy papers to end up on Retraction Watch. A purportedly interdisciplinary paper, in which misrepresentations and mathematical errors abound, is surely a rather good candidate.
3. I know many delicate flowers in cyberspace don't like my sharp tone, but in this case I was clearly correct to be irritated by Bruya's piece, as everyone now knows thanks to Prof. Wallace's detailed analysis.
4. Confirmation bias is, of course, a familiar psychological phenomenon, which we see in the political realm, but also in philosophy. Social media amplifies the phenomenon, as the usual suspects race to tweet their instant approval of anything critical of me or the PGR, even though they clearly haven't any idea of its content. If anyone ever had doubts about the "herd animal" tendencies of "professional" philosophers, social media has provided overwhelming evidence of their existence.
5. Justin Weinberg (South Carolina) will post just about anything--as long as it has the appearance of being substantive--if it represents an opportunity to attack me (he did it last month too with the Lebron nonsense). The sequence of events here was amusing. Weinberg initially added a link to Bruya's piece to his "heap of links" (his long and indiscriminate list of links to anything vaguely philosophy-related), but then he pulled it from there and created a separate post, with lengthy excerpts, even going out of his way to include the fact-free smear of me and the Advisory Board as "racists" and "sexists." After I commented on the Bruya piece, Justin then hosted 14-single-spaced pages of unresponsive, posturing nonsense from Bruya--and even praised it as "patient" and "measured" (Justin is a sucker for tone, as opposed to substance). Now in this instance, I suspect Weinberg was especially irritable because of the recent discussion of PhD programs of dubious merit, which included South Carolina. And while others, like faculty at Tulane and Arizona State, responded effectively to the issue raised, Weinberg's colleagues did not help their cause. I really can't blame him for wanting to tear down the PGR at this moment, but he should have picked a better vehicle for doing so than the Bruya mess. Despite Weinberg's unhappy habit, I want to reaffirm that I think his blog generally has lots of interesting and enjoyable content. When I think about retiring from blogging (and I do for a host of reasons), I am glad that there are now so many other generally useful blogs out there, including his. I do wish Weinberg had more mature judgment, however, and I hope this fiasco will be a learning experience for him.
ADDENDUM: A graduate student writes to report that the following comment on the 14-page Bruya "response" did not make it through Justin's comment moderation, but it is sufficiently funny (and rather mild) that it's worth sharing:
Personally I welcome more high quality philosophical work that engages with non-western traditions. But let's be honest: this is yet another (failed) attempt to go after Big Bad Brian Leiter. This attempt is especially (though not uniquely) galling, in that Bruya concludes, based on the shoddy reasoning exposed by David Wallace's response, that Leiter is motivated by sexism, racism, ethnocentrism and xenophobia.
Morally and intellectually serious people do not make such charges without compelling evidence. Too often Leiter's critics are neither.