This is probably my last post here since Brian is now back in action. I’d like to thank him for the opportunity. I had in mind to write more posts, but I did not realize how time consuming this can be.
In any case, here is the follow up post I promised a few days ago.
John Doris and I have recently written a paper examining simple ways of improving philosophers’ use of the behavioral sciences. You can read the near final draft there. (Comments are welcome!) It’s written as a letter to our students, the kind of letters that we wished we would have received when we were still graduate students.
Some pieces of advice in this paper are fairly obvious, although that does not mean you won’t find many cases where these obvious suggestions are not heeded. We don’t name names. The paper would have been spicier if we had, but also perhaps uselessly polemical and confrontational!
Other pieces of advice will be somewhat less obvious, although none is absolutely surprising.
Here are some of my favorite pieces of advice, which are elaborated at great length in the paper:
1. Here’s an obvious suggestion: read the science! That would seem obvious, but we suspect philosophers sometimes get their science from summaries in scientific articles (Scientific American), pop science books, journal articles (the Science pages of the NYT) or blogs, and those turn complex, intricate results into talking points.
2. If you focus on one finding or on a few findings in a scientific area, you’re doing it wrong. You are likely to cherry pick, to engage in motivated reasoning (particularly if the science has a political or moral import), etc. Rather, consider many studies and rely on meta-analyses (critically of course).
3. Be wary. Scientists hype their results (don't believe the hype); some do shoddy work; falsified results or results that can’t be replicated keep being positively cited. So, investigate your sources. It helps to be able to talk to scientists to learn about the reputations of scientists and about what scientists think of various experimental results.
4. There are a few simple rules to assess experimental results in the behavioral sciences. Examine sample size and effect size (size matters); look for hypotheses that are obviously post hoc (be particularly concerned of hypotheses that predict an effect, but only for a particular demographic – good sign that the hypothesis is post hoc.).
Again, obvious suggestions, but as we say in the paper about one of those, "Once again, that advice is obvious does not mean it is bad. Nor is obvious advice always followed." There's much more in the paper. Check it out!