How President Obama's campaign used big data to rally individual voters, part 1 and 2.
By Sasha Issenberg on December 17, 2012
Sasha Issenberg is the author of The Victory Lab: The Secret Science of Winning Campaigns.
http://www.technologyreview.com/featuredstory/508851/how-obama-used-big-data-to-rally-voters-part-2/
Two years after Barack Obama's election as president, Democrats suffered their worst defeat in decades. The congressional majorities that had given Obama his legislative successes, reforming the health-insurance and financial markets, were swept away in the midterm elections; control of the House flipped and the Democrats' lead in the Senate shrank to an ungovernably slim margin. Pundits struggled to explain the rise of the Tea Party. Voters' disappointment with the Obama agenda was evident as independents broke right and Democrats stayed home. In 2010, the Democratic National Committee failed its first test of the Obama era: it had not kept the Obama coalition together.
But for Democrats, there was bleak consolation in all this: Dan Wagner had seen it coming. When Wagner was hired as the DNC's targeting director, in January of 2009, he became responsible for collecting voter information and analyzing it to help the committee approach individual voters by direct mail and phone. But he appreciated that the raw material he was feeding into his statistical models amounted to a series of surveys on voters' attitudes and preferences. He asked the DNC's technology department to develop software that could turn that information into tables, and he called the result Survey Manager.
That fall, when a special election was held to fill an open congressional seat in upstate New York, Wagner successfully predicted the final margin within 150 votes—well before Election Day. Months later, pollsters projected that Martha Coakley was certain to win another special election, to fill the Massachusetts Senate seat left empty by the death of Ted Kennedy. But Wagner's Survey Manager correctly predicted that the Republican Scott Brown was likely to prevail in the strongly Democratic state. "It's one thing to be right when you're going to win," says Jeremy Bird, who served as national deputy director of Organizing for America, the Obama campaign in abeyance, housed at the DNC. "It's another thing to be right when you're going to lose."
It is yet another thing to be right five months before you're going to lose. As the 2010 midterms approached, Wagner built statistical models for selected Senate races and 74 congressional districts. Starting in June, he began predicting the elections' outcomes, forecasting the margins of victory with what turned out to be improbable accuracy. But he hadn't gotten there with traditional polls. He had counted votes one by one. His first clue that the party was in trouble came from thousands of individual survey calls matched to rich statistical profiles in the DNC's databases. Core Democratic voters were telling the DNC's callers that they were much less likely to vote than statistical probability suggested. Wagner could also calculate how much the Democrats' mobilization programs would do to increase turnout among supporters, and in most races he knew it wouldn't be enough to cover the gap revealing itself in Survey Manager's tables.
His congressional predictions were off by an average of only 2.5 percent. "That was a proof point for a lot of people who don't understand the math behind it but understand the value of what that math produces," says Mitch Stewart, Organizing for America's director. "Once that first special [election] happened, his word was the gold standard at the DNC."
The significance of Wagner's achievement went far beyond his ability to declare winners months before Election Day. His approach amounted to a decisive break with 20th-century tools for tracking public opinion, which revolved around quarantining small samples that could be treated as representative of the whole. Wagner had emerged from a cadre of analysts who thought of voters as individuals and worked to aggregate projections about their opinions and behavior until they revealed a composite picture of everyone. His techniques marked the fulfillment of a new way of thinking, a decade in the making, in which voters were no longer trapped in old political geographies or tethered to traditional demographic categories, such as age or gender, depending on which attributes pollsters asked about or how consumer marketers classified them for commercial purposes. Instead, the electorate could be seen as a collection of individual citizens who could each be measured and assessed on their own terms. Now it was up to a candidate who wanted to lead those people to build a campaign that would interact with them the same way.
Dan Wagner, the chief analytics officer for Obama 2012, led the campaign's "Cave" of data scientists.
After the voters returned Obama to office for a second term, his campaign became celebrated for its use of technology—much of it developed by an unusual team of coders and engineers—that redefined how individuals could use the Web, social media, and smartphones to participate in the political process. A mobile app allowed a canvasser to download and return walk sheets without ever entering a campaign office; a Web platform called Dashboard gamified volunteer activity by ranking the most active supporters; and "targeted sharing" protocols mined an Obama backer's Facebook network in search of friends the campaign wanted to register, mobilize, or persuade.
But underneath all that were scores describing particular voters: a new political currency that predicted the behavior of individual humans. The campaign didn't just know who you were; it knew exactly how it could turn you into the type of person it wanted you to be.
The Scores
Four years earlier, Dan Wagner had been working at a Chicago economic consultancy, using forecasting skills developed studying econometrics at the University of Chicago, when he fell for Barack Obama and decided he wanted to work on his home-state senator's 2008 presidential campaign. Wagner, then 24, was soon in Des Moines, handling data entry for the state voter file that guided Obama to his crucial victory in the Iowa caucuses. He bounced from state to state through the long primary calendar, growing familiar with voter data and the ways of using statistical models to intelligently sort the electorate. For the general election, he was named lead targeter for the Great Lakes/Ohio River Valley region, the most intense battleground in the country.
After Obama's victory, many of his top advisors decamped to Washington to make preparations for governing. Wagner was told to stay behind and serve on a post-election task force that would review a campaign that had looked, to the outside world, technically flawless.
In the 2008 presidential election, Obama's targeters had assigned every voter in the country a pair of scores based on the probability that the individual would perform two distinct actions that mattered to the campaign: casting a ballot and supporting Obama. These scores were derived from an unprecedented volume of ongoing survey work. For each battleground state every week, the campaign's call centers conducted 5,000 to 10,000 so-called short-form interviews that quickly gauged a voter's preferences, and 1,000 interviews in a long-form version that was more like a traditional poll. To derive individual-level predictions, algorithms trawled for patterns between these opinions and the data points the campaign had assembled for every voter—as many as one thousand variables each, drawn from voter registration records, consumer data warehouses, and past campaign contacts.
This innovation was most valued in the field. There, an almost perfect cycle of microtargeting models directed volunteers to scripted conversations with specific voters at the door or over the phone. Each of those interactions produced data that streamed back into Obama's servers to refine the models pointing volunteers toward the next door worth a knock. The efficiency and scale of that process put the Democrats well ahead when it came to profiling voters. John McCain's campaign had, in most states, run its statistical model just once, assigning each voter to one of its microtargeting segments in the summer. McCain's advisors were unable to recalculate the probability that those voters would support their candidate as the dynamics of the race changed. Obama's scores, on the other hand, adjusted weekly, responding to new events like Sarah Palin's vice-presidential nomination or the collapse of Lehman Brothers.
Within the campaign, however, the Obama data operations were understood to have shortcomings. As was typical in political information infrastructure, knowledge about people was stored separately from data about the campaign's interactions with them, mostly because the databases built for those purposes had been developed by different consultants who had no interest in making their systems work together.
But the task force knew the next campaign wasn't stuck with that situation. Obama would run his final race not as an insurgent against a party establishment, but as the establishment itself. For four years, the task force members knew, their team would control the Democratic Party's apparatus. Their demands, not the offerings of consultants and vendors, would shape the marketplace. Their report recommended developing a "constituent relationship management system" that would allow staff across the campaign to look up individuals not just as voters or volunteers or donors or website users but as citizens in full. "We realized there was a problem with how our data and infrastructure interacted with the rest of the campaign, and we ought to be able to offer it to all parts of the campaign," says Chris Wegrzyn, a database applications developer who served on the task force.
Wegrzyn became the DNC's lead targeting developer and oversaw a series of costly acquisitions, all intended to free the party from the traditional dependence on outside vendors. The committee installed a Siemens Enterprise System phone-dialing unit that could put out 1.2 million calls a day to survey voters' opinions. Later, party leaders signed off on a $280,000 license to use Vertica software from Hewlett-Packard that allowed their servers to access not only the party's 180-million-person voter file but all the data about volunteers, donors, and those who had interacted with Obama online.
Many of those who went to Washington after the 2008 election in order to further the president's political agenda returned to Chicago in the spring of 2011 to work on his reëlection. The chastening losses they had experienced in Washington separated them from those who had known only the ecstasies of 2008. "People who did '08, but didn't do '10, and came back in '11 or '12—they had the hardest culture clash," says Jeremy Bird, who became national field director on the reëlection campaign. But those who went to Washington and returned to Chicago developed a particular appreciation for Wagner's methods of working with the electorate at an atomic level. It was a way of thinking that perfectly aligned with their
Obama's campaign began the election year confident it knew the name of every one of the 69,456,897 Americans whose votes had put him in the White House. They may have cast those votes by secret ballot, but Obama's analysts could look at the Democrats' vote totals in each precinct and identify the people most likely to have backed him. Pundits talked in the abstract about reassembling Obama's 2008 coalition. But within the campaign, the goal was literal. They would reassemble the coalition, one by one, through personal contacts
The Experiments
When Jim Messina arrived in Chicago as Obama's newly minted campaign manager in January of 2011, he imposed a mandate on his recruits: they were to make decisions based on measurable data. But that didn't mean quite what it had four years before. The 2008 campaign had been "data-driven," as people liked to say. This reflected a principled imperative to challenge the political establishment with an empirical approach to electioneering, and it was greatly influenced by David Plouffe, the 2008 campaign manager, who loved metrics, spreadsheets, and performance reports. Plouffe wanted to know: How many of a field office's volunteer shifts had been filled last weekend? How much money did that ad campaign bring in?
But for all its reliance on data, the 2008 Obama campaign had remained insulated from the most important methodological innovation in 21st-century politics. In 1998, Yale professors Don Green and Alan Gerber conducted the first randomized controlled trial in modern political science, assigning New Haven voters to receive nonpartisan election reminders by mail, phone, or in-person visit from a canvasser and measuring which group saw the greatest increase in turnout. The subsequent wave of field experiments by Green, Gerber, and their followers focused on mobilization, testing competing modes of contact and get-out-the-vote language to see which were most successful.
The first Obama campaign used the findings of such tests to tweak call scripts and canvassing protocols, but it never fully embraced the experimental revolution itself. After Dan Wagner moved to the DNC, the party decided it would start conducting its own experiments. He hoped the committee could become "a driver of research for the Democratic Party."
To that end, he hired the Analyst Institute, a Washington-based consortium founded under the AFL-CIO's leadership in 2006 to coördinate field research projects across the electioneering left and distribute the findings among allies. Much of the experimental world's research had focused on voter registration, because that was easy to measure. The breakthrough was that registration no longer had to be approached passively; organizers did not have to simply wait for the unenrolled to emerge from anonymity, sign a form, and, they hoped, vote. New techniques made it possible to intelligently profile nonvoters: commercial data warehouses sold lists of all voting-age adults, and comparing those lists with registration rolls revealed eligible candidates, each attached to a home address to which an application could be mailed. Applying microtargeting models identified which nonregistrants were most likely to be Democrats and which ones Republicans.
The Obama campaign embedded social scientists from the Analyst Institute among its staff. Party officials knew that adding new Democratic voters to the registration rolls was a crucial element in their strategy for 2012. But already the campaign had ambitions beyond merely modifying nonparticipating citizens' behavior through registration and mobilization. It wanted to take on the most vexing problem in politics: changing voters' minds.
The expansion of individual-level data had made possible the kind of testing that could help do that. Experimenters had typically calculated the average effect of their interventions across the entire population. But as campaigns developed deep portraits of the voters in their databases, it became possible to measure the attributes of the people who were actually moved by an experiment's impact. A series of tests in 2006 by the women's group Emily's List had illustrated the potential of conducting controlled trials with microtargeting databases. When the group sent direct mail in favor of Democratic gubernatorial candidates, it barely budged those whose scores placed them in the middle of the partisan spectrum; it had a far greater impact upon those who had been profiled as soft (or nonideological) Republicans.
That test, and others that followed, demonstrated the limitations of traditional targeting. Such techniques rested on a series of long-standing assumptions—for instance, that middle-of-the-roaders were the most persuadable and that infrequent voters were the likeliest to be captured in a get-out-the-vote drive. But the experiments introduced new uncertainty. People who were identified as having a 50 percent likelihood of voting for a Democrat might in fact be torn between the two parties, or they might look like centrists only because no data attached to their records pushed a partisan prediction in one direction or another. "The scores in the middle are the people we know less about," says Chris Wyant, a 2008 field organizer who became the campaign's general election director in Ohio four years later. "The extent to which we were guessing about persuasion was not lost on any of us."
One way the campaign sought to identify the ripest targets was through a series of what the Analyst Institute called "experiment-informed programs," or EIPs, designed to measure how effective different types of messages were at moving public opinion.
The traditional way of doing this had been to audition themes and language in focus groups and then test the winning material in polls to see which categories of voters responded positively to each approach. Any insights were distorted by the artificial settings and by the tiny samples of demographic subgroups in traditional polls. "You're making significant resource decisions based on 160 people?" asks Mitch Stewart, director of the Democratic campaign group Organizing for America. "Isn't that nuts? And people have been doing that for decades!"
An experimental program would use those steps to develop a range of prospective messages that could be subjected to empirical testing in the real world. Experimenters would randomly assign voters to receive varied sequences of direct mail—four pieces on the same policy theme, each making a slightly different case for Obama—and then use ongoing survey calls to isolate the attributes of those whose opinions changed as a result.
In March, the campaign used this technique to test various ways of promoting the administration's health-care policies. One series of mailers described Obama's regulatory reforms; another advised voters that they were now entitled to free regular check-ups and ought to schedule one. The experiment revealed how much voter response differed by age, especially among women. Older women thought more highly of the policies when they received reminders about preventive care; younger women liked them more when they were told about contraceptive coverage and new rules that prohibited insurance companies from charging women more.
When Paul Ryan was named to the Republican ticket in August, Obama's advisors rushed out an EIP that compared different lines of attack about Medicare. The results were surprising. "The electorate [had seemed] very inelastic," says Terry Walsh, who coördinated the campaign's polling and paid-media spending. "In fact, when we did the Medicare EIPs, we got positive movement that was very heartening, because it was at a time when we were not seeing a lot of movement in the electorate." But that movement came from quarters where a traditional campaign would never have gone hunting for minds it could change. The Obama team found that voters between 45 and 65 were more likely to change their views about the candidates after hearing Obama's Medicare arguments than those over 65, who were currently eligible for the program.
A similar strategy of targeting an unexpected population emerged from a July EIP testing Obama's messages aimed at women. The voters most responsive to the campaign's arguments about equal-pay measures and women's health, it found, were those whose likelihood of supporting the president was scored at merely 20 and 40 percent. Those scores suggested that they probably shared Republican attitudes; but here was one thing that could pull them to Obama. As a result, when Obama unveiled a direct-mail track addressing only women's issues, it wasn't to shore up interest among core parts of the Democratic coalition, but to reach over for conservatives who were at odds with their party on gender concerns. "The whole goal of the women's track was to pick off votes for Romney," says Walsh. "We were able to persuade people who fell low on candidate support scores if we gave them a specific message."
At the same time, Obama's campaign was pursuing a second, even more audacious adventure in persuasion: one-on-one interaction. Traditionally, campaigns have restricted their persuasion efforts to channels like mass media or direct mail, where they can control presentation, language, and targeting. Sending volunteers to persuade voters would mean forcing them to interact with opponents, or with voters who were undecided because they were alienated from politics on delicate issues like abortion. Campaigns have typically resisted relinquishing control of ground-level interactions with voters to risk such potentially combustible situations; they felt they didn't know enough about their supporters or volunteers. "You can have a negative impact," says Jeremy Bird, who served as national deputy director of Organizing for America. "You can hurt your candidate."
In February, however, Obama volunteers attempted 500,000 conversations with the goal of winning new supporters. Voters who'd been randomly selected from a group identified as persuadable were polled after a phone conversation that began with a volunteer reading from a script. "We definitely find certain people moved more than other people," says Bird. Analysts identified their attributes and made them the core of a persuasion model that predicted, on a scale of 0 to 10, the likelihood that a voter could be pulled in Obama's direction after a single volunteer interaction. The experiment also taught Obama's field department about its volunteers. Those in California, which had always had an exceptionally mature volunteer organization for a non-
Alex Lundry created Mitt Romney's data science unit. It was less than one-tenth the size of Obama's analytics team.
With these findings in hand, Obama's strategists grew confident that they were no longer restricted to advertising as a channel for persuasion. They began sending trained volunteers to knock on doors or make phone calls with the objective of changing minds.
That dramatic shift in the culture of electioneering was felt on the streets, but it was possible only because of advances in analytics. Chris Wegrzyn, a database applications developer, developed a program code-named Airwolf that matched county and state lists of people who had requested mail ballots with the campaign's list of e-mail addresses. Likely Obama supporters would get regular reminders from their local field organizers, asking them to return their ballots, and, once they had, a message thanking them and proposing other ways to be involved in the campaign. The local organizer would receive daily lists of the voters on his or her turf who had outstanding ballots so that the campaign could follow up with personal contact by phone or at the doorstep. "It is a fundamental way of tying together the online and offline worlds," says Wagner.
Wagner, however, was turning his attention beyond the field. By June of 2011, he was chief analytics officer for the campaign and had begun making the rounds of the other units at headquarters, from fund-raising to communications, offering to help "solve their problems with data." He imagined the analytics department—now a 54-person staff, housed in a windowless office known as the Cave—as an "in-house consultancy" with other parts of the campaign as its clients. "There's a process of helping people learn about the tools so they can be a participant in the process," he says. "We essentially built products for each of those various departments that were paired up with a massive database we had."
The Flow
As job notices seeking specialists in text analytics, computational advertising, and online experiments came out of the incumbent's campaign, Mitt Romney's advisors at the Republicans' headquarters in Boston's North End watched with a combination of awe and perplexity. Throughout the primaries, Romney had appeared to be the only Republican running a 21st-century campaign, methodically banking early votes in states like Florida and Ohio before his disorganized opponents could establish operations there.
But the Republican winner's relative sophistication in the primaries belied a poverty of expertise compared with the Obama campaign. Since his first campaign for governor of Massachusetts, in 2002, Romney had relied upon
Such techniques had offered George W. Bush's reëlection campaign a significant edge in targeting, but Republicans had done little to institutionalize that advantage in the years since. By 2006, Democrats had not only matched Republicans in adopting commercial marketing techniques; they had moved ahead by integrating methods developed in the social sciences.
Romney's advisors knew that Obama was building innovative internal data analytics departments, but they didn't feel a need to match those activities. "I don't think we thought, relative to the marketplace, we could be the best at data in-house all the time," Romney's digital director, Zac Moffatt, said in July. "Our idea is to find the best firms to work with us." As a result, Romney remained dependent on TargetPoint to develop voter segments, often just once, and then deliver them to the campaign's databases. That was the structure Obama had abandoned after winning the nomination in 2008.
In May a TargetPoint vice president, Alex Lundry, took leave from his post at the firm to assemble a data science unit within Romney's headquarters. To round out his team, Lundry brought in Tom Wood, a University of Chicago postdoctoral student in political science, and Brent McGoldrick, a veteran of Bush's 2004 campaign who had left politics for the consulting firm Financial Dynamics (later FTI Consulting), where he helped financial-services, health-care, and energy companies communicate better. But Romney's data science team was less than one-tenth the size of Obama's analytics department. Without a large in-house staff to handle the massive national data sets that made it possible to test and track citizens, Romney's data scientists never tried to deepen their understanding of individual behavior. Instead, they fixated on trying to unlock one big, persistent mystery, which Lundry framed this way: "How can we get a sense of whether this advertising is working?"
"You usually get GRPs and tracking polls," he says, referring to the gross ratings points that are the basic unit of measuring television buys. "There's a very large causal leap you have to make from one to the other."
Lundry decided to focus on more manageable ways of measuring what he called the information flow. His team converted topics of political communication into discrete units they called "entities." They initially classified 200 of them, including issues like the auto industry bailout, controversies like the one surrounding federal funding for the solar-power company Solyndra, and catchphrases like "the war on women." When a new concept (such as Obama's offhand remark, during a speech about our common dependence on infrastructure, that "you didn't build that") emerged as part of the election-year lexicon, the analysts added it to the list. They tracked each entity on the National Dialogue Monitor, TargetPoint's system for measuring the frequency and tone with which certain topics are mentioned across all media. TargetPoint also integrated content collected from newspaper websites and closed-caption transcripts of broadcast programs. Lundry's team aimed to examine how every entity fared over time in each of two categories: the informal sphere of social media, especially Twitter, and the journalistic product that campaigns call earned press coverage.
Ultimately, Lundry wanted to assess the impact that each type of public attention had on what mattered most to them: Romney's position in the horse race. He turned to vector autoregression models, which equities traders use to isolate the influence of single variables on market movements. In this case, Lundry's team looked for patterns in the relationship between the National Dialogue Monitor's data and Romney's numbers in Gallup's daily tracking polls. By the end of July, they thought they had identified a three-step process they called "Wood's Triangle."
Within three or four days of a new entity's entry into the conversation, either through paid ads or through the news cycle, it was possible to make a well-informed hypothesis about whether the topic was likely to win media attention by tracking whether it generated Twitter chatter. That informal conversation among political-class elites typically led to traditional print or broadcast press coverage one to two days later, and that, in turn, might have an impact on the horse race. "We saw this process over and over again," says Lundry.
They began to think of ads as a "shock to the system"—a way to either introduce a new topic or restore focus on an area in which elite interest had faded. If an entity didn't gain its own energy—as when the Republicans charged over the summer that the White House had waived the work requirements in the federal welfare rules—Lundry would propose a "re-shock to the system" with another ad on the subject five to seven days later. After 12 to 14 days, Lundry found, an entity had moved through the system and exhausted its ability to move public opinion—so he would recommend to the campaign's communications staff that they move on to something new.
Those insights offered campaign officials a theory of information flows, but they provided no guidance in how to allocate campaign resources in order to win the Electoral College. Assuming that Obama had superior ground-level data and analytics, Romney's campaign tried to leverage its rivals' strategy to shape its own; if Democrats thought a state or media market was competitive, maybe that was evidence that Republicans should think so too. "We were necessarily reactive, because we were putting together the plane as it took off," Lundry says. "They had an enormous head start on us."
Romney's political department began holding regular meetings to look at where in the country the Obama campaign was focusing resources like ad dollars and the president's time. The goal was to try to divine the calculations behind those decisions. It was, in essence, the way Microsoft's Bing approached Google: trying to reverse-engineer the market leader's code by studying the visible output. "We watch where the president goes," Dan
Obama's media-buying strategy proved particularly hard to decipher. In early September, as part of his standard review, Lundry noticed that the week after the Democratic convention, Obama had aired 68 ads in Dothan, Alabama, a town near the Florida border. Dothan was one of the country's smallest media markets, and Alabama one of the safest Republican states. Even though the area was known to savvy ad buyers as one of the places where a media market crosses state lines, Dothan TV stations reached only about 9,000 Florida voters, and around 7,000 of them had voted for John McCain in 2008. "This is a hard-core Republican media market," Lundry says. "It's incredibly tiny. But they were advertising there."
Romney's advisors might have formed a theory about the broader media environment, but whatever was sending Obama hunting for a small pocket of votes was beyond their measurement. "We could tell," says McGoldrick, "that there was something in the algorithms that was telling them what to run."
Tomorrow: Part 3—The Community
