Well the American election has come to an end...almost... and Donald Trump will be the next President of the United States.
Many people, across the ideological spectrum, have asked me: "What happened?"
Right up to election night, predictions showed that Hilary Clinton would win the Electoral College.
Clearly the polls had it wrong. We have seen this trend in other places: the provincial election in British Columbia in 2013 when Adrian Dix was the predicted to be premier, the federal Liberal sweep in 2015 that saw Justin Trudeau become prime minister; and the Brexit vote that shook Britain and the European Union.
In the last few days, pollsters have poured over their models and the numbers trying to figure out what happened. Most conclude that it is really too soon to know exactly why the models were wrong but there are a few explanations that we can give.
I would like to point out two observations made by Steve Lohr and Natasha Singer of the New York Times in their article: "How Data Failed Us in Calling an Election."
First, they say: "...data science is a technology with trade-offs. It can see things as never before, but also can be a blunt instrument, missing context and nuance. All kinds of companies and institutions use data quietly and behind the scenes to make predictions about human behavior. But only occasionally - as with Tuesday's elections results - do consumers get a glimpse of how these formulas work and the extent to which they can go wrong."
The fact is that the analysis of data is as much an art as it is a science.
The numbers that are generated by polls have to be interpreted. Until there is an analysis of the numbers they are simply digits on a page and have little meaning.
Interpretation begins with a set of assumptions that the researcher applies to the data.
So, as Lohr and Singer put it, data can miss the "context and nuance."
If the assumptions of the researcher, which are imbedded in the model, miss some key variable then the analysis can be wrong. For example one may make assumptions about who is likely to vote and whether or not respondents are being truthful about their voting intentions.
As Lohr and Singer write: "[...] data scientists said the inherent weakness of election models might have caused some forecasting errors. Before an election, forecasters use a combination of historical polls and recent polling data to predict a candidate's chance of winning. Some may also factor in other variables, such as giving higher weight to a candidate who is an incumbent. But... it is difficult for forecasters to predict accurately a candidate's chance of winning [...] months or even weeks ahead of time."
Analogous to weather models, the article says, "it is difficult to predict the weather more than 10 days out because there are so many small changes that can cause big changes...."
What the polls did not see were the nuances and the context that was brewing in states that were supposed to vote for Clinton.
Early in the evening on Tuesday, I saw two startling poll questions that came together to clearly predict the outcome of the election.
The questions went something like: "What is most important outcome you want from this election?"
Overwhelmingly, the answer was "change."
"Who is the candidate most likely to deliver change?" Again, overwhelmingly, the answer was "Donald Trump."
From the outset, this was not a normal election cycle. We knew that all bets were off when Trump won the nomination. Pollsters were trying to predict the way that this unusual candidate would fare by focusing on issues of character and scandal and past performance by the Republicans and the Democrats in other elections.
The media was aiming to turn the election into a statement about American values that had been articulated and accepted in 2008 and 2012 when Obama won the White House. But a repudiation of those liberal values combined with a rejection of globalization was the context of the Trump win.
These nuances were also missed in the predictions about the Brexit vote.
The election is not quite over yet. The Electors still have to vote. It is not likely that Electors will be faithless, i.e. will vote against the state's nominee and so Donald Trump will be president in January of 2017.
