I am periodically asked why I don't include a criterion like membership in the American Academy of Arts & Sciences as a factor in the philosophy rankings. You can see a list of current philosophy fellows of the American Academy here. (Note that philosophy is lumped with religion and theology in this category, and that no distinction is drawn between emeritus and non-emeritus faculty.) There are several difficulties with AAAS membership as a marker of departmental quality. First, fellows are usually elected quite late in their careers, yet departmental quality is often equally a function of much younger philosophers. Second, the selection process for the AAAS is a bit incestuous, both with respect to particular departments and particular genres or sub-fields. Third, and as a consequence of the second point, AAAS membership tends to be underinclusive with respect to quality (it's less often overinclusive). Take a look again at the list of philosophers who are currently fellows, and now ask yourself whether there is a qualitative explanation why the following philosophers aren't also currently fellows: Kit Fine, Christopher Peacocke, Crispin Wright,
Arthur Fine, James Higginbotham, Alan Code, Alvin Goldman, Gilbert Harman, Nuel Belnap, Anil Gupta, Timothy Williamson, Laurence BonJour, Ruth Millikan, Frank Jackson, Frances Kamm, Philip Pettit, Peter van Inwagen, and so on. Of course, I would wager that all these philosophers will be elected, sooner or later, but that just underlines the point that the current roster of membership is not a reliable measure of departmental quality.