Nate Silver, elections and SPORTS

Cam Charron
November 07 2012 02:55AM

Math kicked ass.

Not to say that Nate Silver's models are completely infallible, of course, the guy got lucky. Every single state of the union voted as Silver forecasted it to vote, within a slim margin of error he gave simply by aggregating polls over the last year or so.

I don't want to write an election-themed post, but I do think it's worth it for me to note Silver in writing somewhere. Silver, once fed up with mainstream electoral coverage during the 2008 election and began writing a blog. He correctly forecasted the winners of 49 of 50 states. That's not a huge achievement on its own, but Silver had gained notoriety in the political world for the statistical approach on his blog fivethirtyeight and his work with Baseball Prospectus in developing PECOTA, a system used to find historical matches of baseball players, and use those matches to forecast a range of outcomes for a player later in his career.

Neither the fivethirtyeight model or PECOTA are flawless, and Silver would be the first to tell you that. However, his success did land him a job with the New York Times and some controversy leading up to the election, particularly when he began to forecast the likelihood the president would be re-elected. It was 72.9 per cent eight days before the election. By election day, the likelihood had increased to over 90 per cent. A pundit who did not appreciate Silver's mathematical methods said this:

"Nate Silver says this is a 73.6% chance that the president's going to win. Nobody in that campaign thinks they have a 73.6% -- they think they have a 50.1% chance of winning.

Anybody that thinks that this race is anything but a tossup right now is such an ideologue [that] they should be kept away from typewriters, computers, laptops, and microphones for the next ten days, because they're jokes."

That's a man on MSNBC named Joe Scarborough, who as of midnight Pacific Wednesday morning, has not apologized to Silver despite getting practically everything right that there was to get right. Not only did Silver's models correctly predict the states, but the margins of victory were stunning. I don't want to dwell too much on this because this is a hockey blog, not a political blog. Scarborough's prediction? As transcribed by The Colbert Report...

Well, it was considerably off, let's just say.

But is there a familiar pattern? A pundit taking issue with the forecast of a man who never played in the game? Who hasn't covered a campaign close enough to have a permanent coffee dependency?

In Silver's new book The Signal and the Noise, he doesn't exactly show a lack of disdain towards pundits and TV commentators. In an analysis of a panel on a Sunday talk show The McLaughlin Group, Silver found that when panelists were making predictions that had a simple "yes" or "no" answer, "the panel may as well have been flipping coins. I determined 338 of their  [733] predictions to be either mostly or completely false." 

Sports and politics are similar beasts in the way they're covered by the media, particularly on television. I stopped watching TV newscasts, particularly political debates, because the commentary was just so annoying. Sports debates shows are just as bad, and men who make a number of bold predictions that turn out to be false in the end, never seem to lose their job or take heat from their co-workers.

I often wonder why this is. The Internet now, thankfully, allows us to record failed predictions in infamy. I'm not sure what my own record is. For every time I've forecasted the demise of the Minnesota Wild, it seems like I've also forecasted the rise of a young two-way threat who never panned out. I'd like to think that I'm batting over .500 when it comes to testable hypotheses, but going back and re-reading some of the 1000 or so thoughts I've posted online in the last year or so seems tedious.

SPROTS

Silver's sports background is in baseball, which doesn't lend itself to hockey quite the same way. Baseball is composed of a small number of events between individuals. Hockey is a game composed of a much larger number of events taking place between as many as ten players on the playing surface at the same time.

But the two also have a lot in common. The key difference from a statistical perspective is that there's more time between pitches or at-bats to record the previous results. Trying to capture everything that happens in a hockey game is maddening. You have scoring chances, missed shots, zone entries, zone exits, puck battles… it's like basketball in the respect Michael Lewis noted in a New York Times feature: "[For] most of its history basketball has measured not so much what is important as what is easy to measure".

In hockey, that's usually goals, assists and points. The overwhelming impact of Big Data on our society allows pundits in sports to dig up any amount of numbers, and attribute meaning to them and making poor, poor predictions based on a misunderstood concept. A hockey pundit might say "the Los Angeles Kings won just 17 of 46 one-goal games this season! They don't have what it takes when it counts, and will never be successful in the playoffs."

Of course, that same pundit ignores research that suggests that one-goal games are not necessarily a predictive quality of a hockey teams' ability. The general idea, that a team needs to be able to come through in important situations if they can ever hope to succeed in the postseason, sounds reasonable enough, but when looking at small pockets of small margin games that can literally be determined by something as random as a coin flip, the pundit is paying attention to far more noise than signal. Not all information available is important, and I like to disregard a lot of current hockey stats kept because I don't see their use.

"The key to making a good forecast," Silver writes, "is not in limiting yourself to quantitative information. Rather, it's having a good process for weighing the information appropriately. This is the essence of Billy Beane's philosophy: collect as much information as possible, but then be as rigorous and disciplined as possible when analyzing it."

Later:

"It's easy to adopt a goal of "exploit market inefficiencies." But that doesn't really give you a plan for how to find them and then determine whether they represent fresh dawns or false leads. It's hard to have an idea that nobody else has thought of. It's even harder to have a good idea—and when you do, it will soon be duplicated."

Sports forecasting is a more rigorous job than political forecasting. In the end, when all the votes are tallied, everything looks really nice and clean in hindsight (thank you, XKCD):

But in sports, every general manager is weighing the available information, whether it comes from statistics or traditional scouts, to make calls on whether a player will be productive or valuable in the upcoming seasons. Unlike polls, we don't have small hints of Phil Kessel's scoring futures cropping up as much as a year before the start of the season, so it's natural that sports forecasters lag behind on political forecasters.

That said, we're getting there. This extended summer has given us a lot of new things to work with. A fifth year of Behind the Net statistics is good for finding closer comparables to the players we're researching. We know also know the effects of something as simple as entering the zone with the hockey puck. I think as our information improves, we can get closer to mirroring the success that Silver has had with his electoral model.

There is no one way to do things, if you attack the same problem, figuring out how many wins the Toronto Maple Leafs will get next season, many different ways, we'll come closer and closer to the truth.

Still, though, what a fun night for math geeks everywhere.

63811cbf517d2d685ea09e103488ea3a
Cam Charron is a BC hockey fan that writes about hockey on many different websites including this one.
Avatar
#1 Kent Wilson
November 07 2012, 10:34AM
Trash it!
0
trashes
+1
0
props

Great stuff cam.

Avatar
#2 ItsTheBGB
November 07 2012, 11:20AM
Trash it!
0
trashes
+1
0
props

You didn't play the game.

Avatar
#3 RexLibris
November 07 2012, 01:09PM
Trash it!
0
trashes
+1
0
props

I especially like the separation of statement from ability (exploit market inefficiencies) as so often management, or any observer, can come out with a bold comment about how they are going to be smarter and better than the competition.

Then they have to actually do it.

Fans can be guilty of the same thing at times - we'll just draft better like Detroit or Philadelphia and find value players in later rounds - if it were that easy then everybody would do it.

Great article. Go Team Math!

Avatar
#4 mikeH
November 07 2012, 02:20PM
Trash it!
0
trashes
+1
3
props

That Obama, he's so clutch.

Avatar
#5 The Oilers Shot Clock
November 07 2012, 02:36PM
Trash it!
0
trashes
+1
2
props

Math is creepy stuff. I dont trust it. I trust that its accurate, I just worry about its intentions.

Avatar
#6 Pizzy
November 07 2012, 02:54PM
Trash it!
0
trashes
+1
0
props

YAWN! And the point of this article was????

Avatar
#7 M@
November 07 2012, 05:12PM
Trash it!
0
trashes
+1
0
props

Call me old fashion.. but I prefer the eye test.

Great read Cam

Avatar
#8 mattyc
November 07 2012, 06:47PM
Trash it!
0
trashes
+1
0
props

Great article. While I respect and work quite a bit with models, I do think it's important to emphasize that this result doesn't confirm/validate/prove any truth in Silver's model, it only gives us more confidence in it's predictive ability.

Avatar
#9 NewAgeSys
November 08 2012, 12:53AM
Trash it!
0
trashes
+1
0
props

@Pizzy

The point is that this fella has an innate intuative ability and a superior system which produces above average results for him,and it doesnt matter if he is deeply educated in the subject he is analysing and assesing or not.His system can be overlayed onto any dynamic and produce superior results.At the end of the day it matters not if he is using a crystal ball--what matters is he is right more than everyone else consistantly.

Bottom line Sunshine is that he is right more than everyone else,Moma2 often feels that same way with posts about the NewAge Hockey System.

Results talk.Your background,pedigree,degree or Country Club membership mean nothing if you cant out perform someone else when the chips are down--and this means the 'bottom line" is all that matters in results based dynamics.

Oilers,are you paying attention to this stuff yet??Drop the guy a line maybe he can help out.

Comments are closed for this article.