First published on Charlie’s Diary (reproduced here with permission).
By Hugh Hancock, Artistic Director at Strange Company
Image from Hugh Hancock’s new film ‘Death Knight Love Story’
One of the Hot Topics in the big media / tech crossover world recently has been data-driven storytelling. Wired breathlessly reported that Hollywood gurus have reverse-engineered a “formula for success” from audience data. Netflix has revealed its ability to data-mine genres that they already know their audience will like.
The most common reaction to all this Big Data Story stuff is horror. Even less spontaneity. Even more focus group driven cookie-cutter movies. Artists straitjacketed by polling data.
Cats and dogs living together. It’s the end of the world as we know it.
And yet, I feel fine.
Truth is, as a Web-based storyteller, I’ve been making use of data analysis, A/B testing, focus groups, and lots of other terribly un-artisty things for a long while now. I love ’em.
Data is good. Information is useful.
In fact, data saved my most recent film, Death Knight Love Story.
Key fact before I begin this tale: I suck at openings. I’m pretty darn good at entertaining endings, and once a story’s rolling I can certainly keep it moving along, twist the plot, build up subtext and develop my characters. But opening a movie is something I always struggle with.
In the case of Death Knight Love Story, a dark fantasy love story loosely set in the world of the computer game World of Warcraft, we had a few choices. We could go with the classic fantasy Deep-Voiced Narrator opening. We could go for something fast and scrappy, Lock Stock meets Warcraft. Or we could go with a fast-paced in media res opening with our adventurer heroes battling evil right from the outset.
My strong belief was that we needed to go for the In Media Res option. Death Knight Love Story was targeted at a Web audience, and there’s plenty of evidence that Web based viewers turn off after thirty seconds max if they’re not hooked. We were targeting a weird mix of people who were and people who weren’t familiar with the universe in which the film is set, and it seemed pretty reasonable to assume that WoW players would be bored to death if we started by explaining the world of the film.
So we animated and shot a fantastic high-energy opener. And then, thank goodness, Big Scary Data stepped in.
The first signs that not everything was well came when we screened the film to an informal focus group – something that a lot of movie-makers would argue was already polluting my artistic vision. The focus group sorta liked the film, but we noticed that just about everyone was confused by the plot on some level. And a few people said that it took them a couple of minutes into the movie before they really started to get into it.
We had an inkling that our introduction might not be ideal. Under normal filmmaking circumstances, we’d have debated it, I’d probably have made some minor changes but stuck with my “vision” of the opening, and we’d have released.
Instead, I channeled my inner Tim Feriss, and decided on a large-scale test using that most familiar of filmmaker’s tools – Google Adwords.
We put a version of the first five minutes of the film online, engineered with analytics to show us exactly how far through viewers watched it. Then, we ran an advertising campaign targeted at phrases we figured would represent our core audience, pointed the ads to the film, and waited.
Some $200 or so of testing budget later (statistical significance being kind of important in this test), we had our result: the In Media Res intro suuuuuucked.
Almost no viewers were getting through the first minute and a half, let alone the whole of the first five minutes. For whatever reason, my intuition about my audience – founded on 15 years of filmmaking experience – was completely wrong.
One simple Adwords test had prevented me from totally tanking my film.
We subsequently tested some more openings, and finally, after some more Adwords, some anonymous user testing (via UserTesting.com, who are rather awesome), more focus groups, and various other techniques that would make any auteur filmmaker explode from spontaneous Muse-based rage, settled on the opening we have now – a classic narrator, albeit a female one. Turns out, cliches are cliches for a reason.
Thanks to that change, more people finish watching the entire 20 minutes of Death Knight Love Story part 1 than made it through the first minute and a half of my In Media Res version.
But didn’t I end up ceding control of my story? Didn’t I end up cheapening it and producing something that might have been what the audience wanted, but didn’t reflect my inner views on the world?
Nope.
There are a lot of elements of Death Knight Love Story that I feel very strongly about. If Adwords had dictated that our heroine be thinner, less muscled, and possessed of revealing armour and gravity-defying boobs, for example, I would have gleefully told Adwords to fuck right off. (Actually, we didn’t even do this test – because I’m 95% sure that a more fantasy-game conventional Miria would have tested better, and 100% sure that I don’t give a shit.)
But the opening of the movie’s just a story delivery system. We needed to have one, and it needed to get my audience into the interesting bits of the movie, but I had no particularly strong feelings about it artistically. So the best opening was the one that got the largest number of my viewers to watch the movie as a whole – the bits I did care about.
And that’s what data is useful for. It’ll tell you if your audience thinks your character’s chin looks weird. (Fixed. Improved.). It’ll tell you if you’re better shunting the act break along one scene. (Fixed. Improved.)
Hell, it’ll tell you if the front page of your movie’s website would be better with or without a trailer on it.
We tested that: twice as many people finish watching Death Knight Love Story pt 1 if they’re presented with a front page just containing the movie itself, not a trailer. You know what? If changing the look of my website doubles my audience, I’m 100% OK going with “soulless data” over auteur theory.
There are lots of ways in which blindly following test data could have ruined Death Knight Love Story. It could have told me to drop the slightly twisted romance / sexuality elements of the movie. It could have told me to make my characters more two-dimensional. It could have told me to use crappy Hollywood fight choreography or reverse the genders of our protagonists so that the our hero was male and our damsel in distress was female.
But those choices wouldn’t have been the fault of the data. They would have been the fault of the person who made the choice to go with “popular” over “good”.
Every successful author I know makes use of sophisticated audience analysis, based on a carefully selected, semi-representative focus group. They’re called “beta readers”, “editors”, “writers’ groups”, and more colloquially, “any of my mates who owe me a favour large enough I can persuade them to read my unfinished novel”. I’ve spent many hours participating in Charlie’s natural language-based data-mining projects, otherwise known as “getting drunk and talking about his next novel”.
And that approach seems to work.
So why should we fear a different way of collecting data, just because it involves less hangovers and more graphs?