This study by some political science and public policy types is being widely cited as showing that there is "liberal bias" in the media. Here is what the authors write (I'm not making this up, follow the link above and read the first page):
"Our results show a very significant liberal bias [emphasis added]. All of the news outlets except Fox News’ Special Report received a score to the left of the average member of Congress. Moreover, by one of our measures all but three of these media outlets (Special Report, the Drudge Report, and ABC’s World News Tonight) were closer to the average Democrat in Congress than to the median member of the House of Representatives. One of our measures found that the Drudge Report is the most centrist of all media outlets in our sample. Our other measure found that Fox News’ Special Report is the most centrist. These findings refer strictly to the news stories of the outlets. That is, we omitted editorials, book reviews, and letters to the editor from our sample."
And what "methodology" produced these surprising results?
"To compute our measure, we count the times that a media outlet cites various think tanks. We compare this with the times that members of Congress cite the same think tanks in their speeches on the floor of the House and Senate....As a simplified example, imagine that there were only two think tanks, one liberal and one conservative. Suppose that the New York Times cited the liberal think tank twice as often as the conservative one." The authors then identify which member of Congress has a similar citation pattern. The Americans for Democratic Action score of that politician then determines how liberal or conservative the news outlet with the same citation pattern is.
The word, "liberal," alas is being used a bit oddly here. When the authors are being more careful, and less polemical, this is how they correctly state their actual conclusion:
"All the media outlets except Fox News’ Special Report and the Drudge Report have a score that is left of the center of Congress [emphasis added]. And this is true whether one uses citations or sentences as the level of observation or sentences, or whether one defines center of Congress by means or medians, or by House or Senate."
You would think that political scientists--who presumably have had to study, at some point, political theory--would think to mention that being to the "left of the center of [the United States] Congress" is not the same thing as being liberal. Indeed, the giveaway on how absolutely misleading the billing of this study is comes from a section which the authors actually have the gall to title "Digression: Defining the 'Center.'" But this is no digression, this is the essence of why the study does nothing to show "liberal bias" in the media! They write, apparently without being ironic, as follows:
"In discussing left- or right- wing biases of the media, one should be careful how he or she defines center. We think the most appropriate definition refers to a central voter, as opposed to a central member of Congress. Accordingly, we think that it is more appropriate to compare media scores to the House as opposed to the Senate, since the Senate disproportionately represents small states. Next, we think it is more appropriate to use the median House member, instead of the mean."
Unless one had any reason to think that being to the left of the median member of the Republican-dominated House (with its large contingent of conservative Southern Democrats) made one a "liberal"--and there is no reason to think that, and the authors give no reason to think that--then even without quarrelling with any other aspect of the methodology, it is starkly obvious that the study does not show that there is a "liberal bias" in the media. It shows that the major media are to the "left" of the extremely conservative Congress, while right-wing news outlets like Fox and Drudge are right in sync with the conservative Congress.
Didn't we know this before this pointless exercise?
But what's really scandalous here--what confirms the widely shared suspicion that "political science," despite its pretentious name, is an ideologically driven field--is that the authors have framed their results in a way that suggests they have shown "liberal bias in the media," when they have shown nothing of the kind, even accepting their entire methodology.
The authors conclude:
"Although we expected to find that most media lean left, we were astounded by the degree. A norm among journalists is to present 'both sides of the issue.' Consequently, while we expected members of Congress to cite primarily think tanks that are on the same side of the ideological spectrum as they are, we expected journalists to practice a much more balanced citation practice, even if the journalist’s own ideology opposed the think tanks that he or she is sometimes citing. This was not always the case."
But this argument presupposes that truth and objective information is evenly distributed between the think tanks all along the political spectrum. But why think that? If journalists aspire to objectivity and factual accuracy, then they will disproportionately quote those who are objective and factually accurate. Only if there were some reason to think those characteristics were evenly distributed across the political spectrum--Stalinists, Maoists, Hayekians, Bushites, Clintonites, etc. all have an equal share--would there be any reasonable expectation of "bipartisan" citation practices.
What a disservice these two scholars have done to honest public discussion. Shame on Professors Groseclose and Milyo!
UPDATE: My colleague Frank Cross, far more expert on methodological matters than me, has written with some informative doubts about the methodology itself (quite apart from the misleading use of the words "liberal" and "bias"):
"The study has other, more serious methodological problems, however. Using
thinktank citations to study for bias seems an odd approach and it creates some real problems for drawing conclusions from the data. Consider these alternative explanations for their results:
"a) Suppose conservative thinktanks have greater resources than liberal thinktanks and can therefore better produce and disseminate their reports. This would create an across the board conservative bias regardless of report accuracy. But suppose that congresspersons just grab what's available, relatively uncritically, while the media gives a critical eye to the reports, investigating them for truth, relevance, etc., looking for evidence from the other side, etc. This is presumably what a good, unbiased media would do. Yet this scenario could produce exactly the results that the authors found! In other words, good unbiased reporters would falsely appear to be biased in this methodology, if Congress were less critical in using the reports. The study results could actually be evidence for unbiased media quality, if these alternative presumptions were true.
"b) Suppose that conservative thinktanks focus their reports on Congress, because they control Congress and can get action taken. Liberal thinktanks, realizing that Congress is a hopeless case, don't disseminate much to Congress. Indeed, perhaps liberal thinktanks focus their attention on the press, to go over the head of Congress. In this scenario, you can't really say anything about media bias, except as an artifact of thinktank strategy.
"The use of thinktank proxies has another methodological problem. The authors don't examine the particular ideological bent of the individual thinktank reports, just the general reputation of the thinktank. Because liberal sources sometimes produce conservative reports, and vice versa, this could easily cause the authors to have spurious results (a variant of something called the ecological fallacy).
"One other concern about the study. The authors excluded the ACLU, after
finding unexpected reports about how often it was cited by conservatives. They found that this was due to the ACLU's position on McCain-Feingold and therefore concluded that their results on the ACLU were unreliable. This is inappropriate. One can exclude cases from statistical studies but there are accepted procedures for doing so, which the authors apparently did not use. At minimum, they should have checked whether this same problem existed with other thinktanks, but they apparently did not do so. By excluding the ACLU, they made the media results look more biased than they would with the ACLU included. To me, this smells like authorial bias, manipulating the data to produce a strong result in the direction intended. While it was fully disclosed, as appropriate, it makes me wonder about the other discretionary decisions that the authors took."
Recent Comments