Nate Silver, elections and SPORTS
By Cam Charron
11 years ago
Math kicked ass.
Not to say that Nate Silver’s models are completely infallible, of course, the guy got lucky. Every single state of the union voted as Silver forecasted it to vote, within a slim margin of error he gave simply by aggregating polls over the last year or so.
I don’t want to write an election-themed post, but I do think it’s worth it for me to note Silver in writing somewhere. Silver, once fed up with mainstream electoral coverage during the 2008 election and began writing a blog. He correctly forecasted the winners of 49 of 50 states. That’s not a huge achievement on its own, but Silver had gained notoriety in the political world for the statistical approach on his blog fivethirtyeight and his work with Baseball Prospectus in developing PECOTA, a system used to find historical matches of baseball players, and use those matches to forecast a range of outcomes for a player later in his career.
Neither the fivethirtyeight model or PECOTA are flawless, and Silver would be the first to tell you that. However, his success did land him a job with the New York Times and some controversy leading up to the election, particularly when he began to forecast the likelihood the president would be re-elected. It was 72.9 per cent eight days before the election. By election day, the likelihood had increased to over 90 per cent. A pundit who did not appreciate Silver’s mathematical methods said this:
“Nate Silver says this is a 73.6% chance that the president’s going to win. Nobody in that campaign thinks they have a 73.6% — they think they have a 50.1% chance of winning.Anybody that thinks that this race is anything but a tossup right now is such an ideologue [that] they should be kept away from typewriters, computers, laptops, and microphones for the next ten days, because they’re jokes.”
That’s a man on MSNBC named Joe Scarborough, who as of midnight Pacific Wednesday morning, has not apologized to Silver despite getting practically everything right that there was to get right. Not only did Silver’s models correctly predict the states, but the margins of victory were stunning. I don’t want to dwell too much on this because this is a hockey blog, not a political blog. Scarborough’s prediction? As transcribed by The Colbert Report…
Well, it was considerably off, let’s just say.
But is there a familiar pattern? A pundit taking issue with the forecast of a man who never played in the game? Who hasn’t covered a campaign close enough to have a permanent coffee dependency?
In Silver’s new book The Signal and the Noise, he doesn’t exactly show a lack of disdain towards pundits and TV commentators. In an analysis of a panel on a Sunday talk show The McLaughlin Group, Silver found that when panelists were making predictions that had a simple “yes” or “no” answer, “the panel may as well have been flipping coins. I determined 338 of their [733] predictions to be either mostly or completely false.”
Sports and politics are similar beasts in the way they’re covered by the media, particularly on television. I stopped watching TV newscasts, particularly political debates, because the commentary was just so annoying. Sports debates shows are just as bad, and men who make a number of bold predictions that turn out to be false in the end, never seem to lose their job or take heat from their co-workers.
I often wonder why this is. The Internet now, thankfully, allows us to record failed predictions in infamy. I’m not sure what my own record is. For every time I’ve forecasted the demise of the Minnesota Wild, it seems like I’ve also forecasted the rise of a young two-way threat who never panned out. I’d like to think that I’m batting over .500 when it comes to testable hypotheses, but going back and re-reading some of the 1000 or so thoughts I’ve posted online in the last year or so seems tedious.
SPROTS
Silver’s sports background is in baseball, which doesn’t lend itself to hockey quite the same way. Baseball is composed of a small number of events between individuals. Hockey is a game composed of a much larger number of events taking place between as many as ten players on the playing surface at the same time.
But the two also have a lot in common. The key difference from a statistical perspective is that there’s more time between pitches or at-bats to record the previous results. Trying to capture everything that happens in a hockey game is maddening. You have scoring chances, missed shots, zone entries, zone exits, puck battles… it’s like basketball in the respect Michael Lewis noted in a New York Times feature: “[For] most of its history basketball has measured not so much what is important as what is easy to measure”.
In hockey, that’s usually goals, assists and points. The overwhelming impact of Big Data on our society allows pundits in sports to dig up any amount of numbers, and attribute meaning to them and making poor, poor predictions based on a misunderstood concept. A hockey pundit might say “the Los Angeles Kings won just 17 of 46 one-goal games this season! They don’t have what it takes when it counts, and will never be successful in the playoffs.”
Of course, that same pundit ignores research that suggests that one-goal games are not necessarily a predictive quality of a hockey teams’ ability. The general idea, that a team needs to be able to come through in important situations if they can ever hope to succeed in the postseason, sounds reasonable enough, but when looking at small pockets of small margin games that can literally be determined by something as random as a coin flip, the pundit is paying attention to far more noise than signal. Not all information available is important, and I like to disregard a lot of current hockey stats kept because I don’t see their use.
“The key to making a good forecast,” Silver writes, “is not in limiting yourself to quantitative information. Rather, it’s having a good process for weighing the information appropriately. This is the essence of Billy Beane’s philosophy: collect as much information as possible, but then be as rigorous and disciplined as possible when analyzing it.”
Later:
“It’s easy to adopt a goal of “exploit market inefficiencies.” But that doesn’t really give you a plan for how to find them and then determine whether they represent fresh dawns or false leads. It’s hard to have an idea that nobody else has thought of. It’s even harder to have a good idea—and when you do, it will soon be duplicated.”
Sports forecasting is a more rigorous job than political forecasting. In the end, when all the votes are tallied, everything looks really nice and clean in hindsight (thank you, XKCD):
But in sports, every general manager is weighing the available information, whether it comes from statistics or traditional scouts, to make calls on whether a player will be productive or valuable in the upcoming seasons. Unlike polls, we don’t have small hints of Phil Kessel’s scoring futures cropping up as much as a year before the start of the season, so it’s natural that sports forecasters lag behind on political forecasters.
That said, we’re getting there. This extended summer has given us a lot of new things to work with. A fifth year of Behind the Net statistics is good for finding closer comparables to the players we’re researching. We know also know the effects of something as simple as entering the zone with the hockey puck. I think as our information improves, we can get closer to mirroring the success that Silver has had with his electoral model.
There is no one way to do things, if you attack the same problem, figuring out how many wins the Toronto Maple Leafs will get next season, many different ways, we’ll come closer and closer to the truth.
Still, though, what a fun night for math geeks everywhere.
Recent articles from Cam Charron
