Openness and Replicating Studies

Brian Keegan’s article argues for the need to have openness in data journalism by replicating a study by Walt Hickey on movies using the Bechdel Test.

Hickey says the study “analyzed 1,615 films released from 1990 to 2013 to examine the relationship between the prominence of women in a film and that film’s budget and gross profits.”

Taken from Hickey's original article.
Taken from Hickey’s original article.

Keegan discusses a lot of different findings (and shows his data and programming runs). One of the most interesting findings is on when we can expect to see American movies regularly passing the Bechdel Test.

Extrapolating this linear model forward in time, on Tuesday, August 30, 2089, the average movie will finally pass the Bechdel test. Just 75 years to go even the average summer blockbuster will have minimally-developed female characters! Hooray!

This is a quote from an article on repeating analysis of published studies for data journalism. It specifically repeats research on movies that pass/don’t pass the Bechdel test.

His findings on movies that pass the Bechdel test:
Receive budgets that are 24% smaller
Make 55% more revenue
Are awarded 1.8 more Metacritic points by professional reviewers
Are awarded 0.12 fewer stars by IMDB’s amateur reviewers

In addition to replicating (as far as possible given inadequate information on methods) Hickey’s study, Brian Keegan added some analysis of the date regarding:
IMDB ratings
professional critic ratings

He also controlled for additional variables, including:

MPAA Rating. People dislike G-rated movies that happen to pass the Bechdel test more, perhaps.
Runtime. Instead of people hating “feminist” movies, maybe movies passing the Bechdel test are just longer and people don’t like 2-hour marathons.
Genre. Maybe some genres like romantic comedies or dramas have an easier time passing the Bechdel test.
Year. There may be a nostalgia effect of movies in the past that pass the test being rated differently than movies released more recently that pass the test.
Week. Summer and holiday blockbusters are different animals than awards vehicles that are released in the fall and winter.
English language. “Seriously, who likes strong female leads and subtitles? Get me a Bud Light Lime and let’s fire up Michael Bay’s magnum opus Transformers!”
USA. As bad as it may be here, other countries may have it worse.

Is this discrimination?

These four points point to a paradox in which movies that pass an embarrassingly low bar for female character development make more money and are rated more highly by critics, but have to deal with lower budgets and more critical community responses. Is this definitive evidence of active discrimination in the film industry and culture? No, but it suggests systemic prejudices are contributing to producers irrationally ignoring significant evidence that “feminist” films make them more money and earn higher praise.

This is an excellent and interesting article. I appreciate the time Dr. Keegan took to work on it and my husband for passing it on to me.

Leave a Reply

Your email address will not be published. Required fields are marked *

CommentLuv badge