Positive and negative associations with a particular candidate would almost certainly have influenced the raters. I wanted to know whether the headlines themselves were negative or positive, which meant I had to minimize this influence. I took the headlines, went through and redacted the names and any clearly identifying information, like “Republican†or “Democrat,†and replaced names and info with single letters, not O or M, not R or D.
Then I printed them out on two sheets of paper, front and back. After a section in two composition classes discussing connotation and denotation, I passed the list of redacted headlines out to the students. This was in late September, when the elections were still going on. I told them that I was simply trying to determine whether the headlines themselves were positive, negative, or neutral. I also told them that their choices would have no ramification on their grade, but that there would be a grade for finishing the work.
These students were from a community college in Texas, a significantly red state. The school itself is in a strong Republican suburb of Houston. However, many of the students are from surrounding small towns. Most of the students were not politically aware. Most had not voted in the primaries. I did not feel comfortable asking for their political affiliations, so I do not know how the classes’ affiliations fell, though I do know that there were supporters of both candidates in the classes based on comments made at other times.
After I received all this information, the headlines were divided by referents. There were three different groups: Obama, McCain, and both.
To determine the “score,†I took the number of raters who gave a particular headline a rating, such as neutral. Then I added all those up and divided by the number of articles. This gave an average rating. There were 34 raters, though occasionally someone rated a single headline two ways. In order not to put my own interpretation of which they meant, I took them both each time that occurred.
So, in the four days of campaign headlines which I examined, there were 85 possible headlines. Nineteen of those were about both candidates. Twenty-one of the headlines were about McCain. Surprisingly, for the perception of FoxNews.com headlines about Obama were the most common. Of the 85 headlines, 45 were about Obama. Obama headlines were more than twice as common as McCain. In sheer numbers of headlines, FoxNews.com does not appear to be biased to the right. Having seen that, the question becomes, are the titles themselves biasing?
Out of the 85 headlines, there were 19 which referred to both of the candidates, either by name or by identifying information such as “’08 Candidates.†These headlines mostly received negative and neutral responses. Relatively few of the raters gave these dual-candidate headlines a positive rating. In fact, the positive score for the headlines referring to both candidates was a 6.5. There were almost double the number of neutral ratings as positive ones, with a total of 240 neutral ratings for an average score of 12.5. The dual-candidate headlines were viewed by the raters as more negative. There were 267 negative ratings total, for an average score of 14. I wondered if this were a comment on politics in general.
The next highest number of headlines were those which referenced McCain. There were 21 headlines of the 85 which either referred to McCain by name or whose articles were about McCain. As for the dual-candidate headlines, the lowest rating by percentage was positive. Positive ratings totaled 150 for an average score of 7.1. The neutral ratings were double that, coming in with 302 for 21 for a final average rating score of 14.3. For headlines about McCain, with his name or other identifying information redacted, negative ratings fell between the positive and negative, with an average score of 11.9 in the negative ratings. So for headlines for articles about McCain, the most common rating was a neutral, with the negative ratings second, and the positive ratings at half of the neutral ratings.
Out of the 85 headlines rated, 45 were about Obama. This includes the one possibly skewed headline, so I will give the numbers with and without this headline included.
This paper was presented at the national conference of Popular Culture on 8 April 2009.