Prof. Wallace has updated his critique here. Section 6, which was added in the second version of his critique, also raises, as Wallace notes, an "academic malpractice" issue. Wallace also offers a useful summary of his take on the issue in light of Bruya's purported response:
Here’s where I am coming from. Methodologically this is not a philosophy paper: it’s a quantitative social-science paper, dominated by statistical analysis, claims that various results are demonstrated by data, discussion of correlation coefficients, presentation of regression analysis, and the like. (It describes itself as a “data-driven critique”.)
And all of this is fatally flawed by the standards of the quantitative sciences. A central methodological assumption is relegated to appendices and described in a way that naturally invites confusion. Another is not stated at all, to be inferred only indirectly, and the quantitative analysis and the discussion rely on incompatible versions of that assumption. A mathematical tool is used in a context where it is normally inapplicable and no discussion is given of its validity. Classifications of obviously uneven sizes are treated as demographics. And so on. Nothing of this kind would ever have got past peer review in a serious science journal; indeed, science papers get withdrawn for much subtler methodological mistakes.
As I have repeatedly noted, I don’t hold Prof. Bruya blameworthy for this. I had hoped that when I pointed it out in some detail he would have responded by withdrawing the article or at any rate publishing a correction that acknowledges the flaws in most of the “data-driven” part of the paper. (That’s entirely compatible with continuing to criticise the PGR on the other grounds that Prof. Bruya raises.) Instead he has doubled down on his analysis. In this situation I didn’t, and still don’t, see any way forward less drastic than a call for withdrawal, if this issue is to be clearly identified as a simple failure of scientific methodology, and not just treated as one more philosophical dispute where reasonable people can differ, and/or annexed to the interminable disputes about the PGR.
You say: “Given the profound and widespread disagreement among philosophers on virtually everything philosophical, our relatively high fallibility regarding such matters, and the increasing reliance on the part of many of us on “data,” I strongly urge charity, humility, caution, and of course rigor and more rigor.”
The point is that these are not *philosophical* matters, and they ought not to be controversial. The only reason to have a high fallibility regarding them is lack of understanding of statistics and scientific methodology. As for a reliance on data, *mostly* when philosophy engages with data it does so at one remove, through philosophical engagement with statistical analysis in the science literature. (I have no objection to people quoting R-squared values without much methodological discussion, if they’ve got those R-squared results from a published paper). But if we as philosophers are going to roll our own analysis, we absolutely have to hold ourselves to the standards of science here.
I have taken a large amount of time and energy to engage with this issue at an inconvenient time for me not because I particularly want to get involved in a debate about the pros and cons of the PGR but because it is important to me that philosophy, as a discipline, is able to engage with science properly and in a way that can reasonably ask to be taken seriously. If we’re prepared to let this kind of thing appear uncorrected in our research literature, and to suppose that straightforward and widespread error on elementary matters of methodology is just one more form of philosophical controversy, we will make fools of our discipline. We will appear to others as if we are children playing with tools that we do not understand.
Recent Comments