Jenny Saul (Sheffield) declared Prof. Jennings "a professional data guru." How professional? This exchange between her and David Wallace (Oxford) is instructive. Prof. Wallace starts by quoting Jennings's attempt to attack me with her data:
David Wallace said...
Carolyn Dicey Jennings said in reply to David Wallace...
Here is another way of looking at this: 32 of the 97 (33%) programs have neither a higher percentage of women graduates than the overall percentage nor a higher percentage of non-white graduates, whereas only 2 of the 11 (18%) programs listed by Brian Leiter have this quality. Conversely, 67% of all programs have one or the other feature, whereas 82% of the programs from this list have one or the other feature. To me, this is a striking difference.
David Wallace said...
"To me, this is a striking difference."
To me, it's noise. Or to be more accurate, those numbers by themselves don't obviously suggest anything but noise. (11 is a small sample.) You'd need to look at the actual scale of the deviation, institution by institution, to show there's something more interesting going on.
At this point I'd come back to my comment at Feminist Philosophers: this is driven by 3 of the 11 that really do have a statistically significant excess of women and non-whites. The other 8, collectively, are below average.
Carolyn Dicey Jennings said in reply to David Wallace...
I don't think it is noise. Check out the second tab of the linked spreadsheet above. If we compare the 11 programs mentioned by Brian Leiter to the rest of the programs (omitting the 11 from that set), then the difference is highly statistically significant for the percentage of women graduates (p=.002) and very nearly statistically significant for the non-white graduates (p=.06).
As to the 8.25 number: if the whole has 67% programs with one or the other of these features, a sample of 11 should have around 7. This sample has 9.
David Wallace said...
To 11: (sorry, this Is getting a bit interleaved and hard to follow)
8.25 is just 75% of 11, which is what you'd predict on the crudest of null hypotheses: mean and median coinciding, no correlation between number of women and number of non-whites. Both of which may be false, but I had the impression that you thought 9 out of 11 was striking in itself, not that it was striking given some particular features of the overall distribution - hence my reply.
You've now given the actual population statistics, on which basis a randomly-selected 11 programs would have 7.33 programs with one or other feature, with a standard deviation of about 1.6. So 9 out of 11, by itself, looks pretty uninteresting - barely one standard deviation from the prediction. (All calculations done late enough at night - UK time - that I welcome corrections!)
For more on Professor Jennings's "data analysis" recall this and this.
UPDATE: The dialogue continues.
"9 of the 11 programs mentioned by Brian have a higher percentage of either women graduates or non-white (citizen/permanent resident) graduates than the percentage for all of the covered programs for this time period. "
If you picked 11 programs at random, on average 8.25 would have a higher percentage of either women graduates or non-white (citizen/permanent resident) graduates than the percentage for all of the covered programs for this time period. So by itself that isn't a persuasive argument.