Following up on this, more amusement:
"But if one accepts Brian Leiter's position that they should close *except* for being concerned about harming representation, the right conclusion would be to close the other 8 and leave those 3, not to reject the overall position and leave all 11 open. (If one rejects Leiter's position on grounds independent of representation, then the data here is irrelevant in any case.)"
I am not sure if this accurately describes Brian's view, since I am not sure if he has presented the full list of variables that could matter to him just yet. In any case, I would say that programs that especially support either women or racial/ethnic minorities have value, and so 9 of the 11 would have an argument in their favor on these grounds. They may also have others. I would want to see a systematic review of all programs based on all relevant variables to say more here.
To Carolyn at 36:
"In any case, I would say that programs that especially support either women or racial/ethnic minorities have value, and so 9 of the 11 would have an argument in their favor on these grounds."
For 6 of those 9, there is no reason at all to think their "support" is anything other than random noise.
On the one hand I am not invested in using statistical significance to determine the matter. On the other I would be more confident in agreeing or disagreeing with your statement if it came from a professional statistician with some neutrality on the issue and the time/energy to think carefully about it.
EDIT: This could have been clearer. What I mean is that I would be more confident in agreeing with your statement if it came from a professional statistician with some neutrality on the issue and the time/energy to think carefully about it but ALSO that I would be more confident in disagreeing with your statement if this disagreement came from a professional statistician with some neutrality on the issue and the time/energy to think carefully about it.
The point is just that I don't think your claim has been established but this is also not a point that I find important enough to dig in to the details about right now. I think the way I wrote it above could come across as disrespectful, and I want to be clear that I mean no disrespect.
(a) I'm fairly sure I am neutral on the issue - and it does not take a lot of time / energy (let alone being a professional statistician) to do elementary things with the binomial distribution and its normal approximation. (The statistics component of my physics qualifications will do fine, if credentialing is really necessary here.)
Look: of those 9, 3 clearly are statistically significant deviations *by themselves*. The others, collectively and individually, are not statistically significant at all. (p>0.1 in all individual cases that are above the average; the remaining 8 collectively are *below* average in both categories.)
(b) You say "I am not invested in using statistical significance to settle the matter". Are you seriously saying that it doesn't matter to your view whether the effect you're discussing is random noise?
I am just going to let David have the last word on this issue and set it aside, per my comments above.
