Let’s talk about stats vs traditionalists

The advanced stats versus conventional hockey people fight is probably getting old for anyone watching the argument from the outside. Nevertheless, it rages on, flaring up whenever a new school person flagrantly denies conventional wisdom or an old school person balks at nerds analyzing the game with spreadsheets.

Many might be surprised to find out that the two schools of thought actually agree more than they disagree. The problem is, there are fundamental areas of friction between the two sides that may never be resolved. Let’s talk about these issues, from the view of a new school outsider with a background in psychology (i.e.; me).

Advertisement - Continue Reading Below

So sum up what you’re talking about here. Why can’t we get past this annoying sniping between the two sides?

The problem is there are intractable differences between traditional hockey thought and newer, evidence-based evaluation. Some things that are considered irreducible primary truths or axioms by traditionalists don’t seem to have basis. Or at least in the current environment of the NHL when it comes to, you know, things that help you win games. Culture can do this – inculcate and insulate beliefs that are widely held and wildly wrong all at once. That said, culture can also bury deep seated truths in long-held beliefs and rituals. The problem is, it’s difficult to separate the truth from myth, especially when some beliefs become sacrosanct or considered too “obvious” to be challenged.

So while there is probably a lot of overlap (coaches actually tend to make decisions that are largely in agreement with corsi, for example), some of the key friction points are almost irreconcilable between the two sides.

Advertisement - Continue Reading Below

So what are these “friction points”?

We’ll start with an area almost everyone struggles with: randomness. Traditional hockey guys (and almost everyone else) don’t think probabilistically and don’t understand randomness. Like, at all. The traditional mindset is very much “100% causal” in that what happens is considered inevitable. Everyone has full control over their results.

As a result, they are endlessly chasing results and tend to overweight a player’s previous accomplishments, even if they were mostly team based. They also seem to fall prey to the fundamental attribution error, a bias that supposes personal factors are the primary determining factors in all outcomes. Would the Canadiens consider Andrew Shaw “a winner who hates to lose” if he had played for Oilers rather than Hawks, for instance? Probably not, but the player didn’t really have control over what team he played for and his perception as a winner is almost wholly dependent on that variable.

How do you come to this conclusion?

While corsi gets the most play in the media, one of the biggest leaps forward in hockey analysis was actually PDO and the subsequent acknowledgement of the sway of randomness over results in the NHL. For years, analysts on the new school side of things have watched as narratives and stories about teams and players have been fabricated to explain the random swing of the percentages. Winners and leaders one year become losers and malcontents the next.

The culture of the league also bears explicit antipathy towards a luck/circumstances mindset in players. No one likes a guy who blames his coach, teammates, GM or luck for bad results. Similarly, no one wants to accept that success can be due to good fortune. Our brains prefer the story, which seems satisfactory and obvious in the aftermath.

Advertisement - Continue Reading Below

What’s next?

Traditionalists frequently mistake what they like with what is good (i.e.; what is useful) and that dissuades them from looking for actual relationships between winning and factor X. See: blocking shots, being “tough” and hitting and how they don’t actually tend to correlate to winning.

Again, this is a common psychological bias. It’s hard to know how much a certain action or metric actually influences winning, so instead of answering that question, the brain substitutes something easier – and seeing a guy who is willing to sacrifice his body to block a shot, fight an opponent, or body check opponents is observationally obvious, easy to recall, and superficially commendable.

On the other hand, stats analysts tend to treat each assumption as a theory that requires verification and testing. And here’s the thing: the stuff that turns out to be the most counter-intuitive is what tends to get the most attention and interest – because it’s controversial, noteworthy and points to a potential market inefficiency. 

Thus you get the conflict of “this is how the game has always been played” versus “but it’s not right.” You have guys who haven’t played at the highest levels telling ex-players, coaches and longtime fans what they have always believed as not only true but plainly obvious. 

Why do you keep talking about psychology?

Advertisement - Continue Reading Below

Because this is, at its base, a conflict about ways of thinking about the game and not actually the particular facts and squabbles that erupt. One of the problems with eschewing careful stats work and evidence-based analysis is that you can be prone to the narrative fallacy.

That makes people prone to reductive, “story”-based thinking when it comes to attributing success or failure to certain variables. So, for instance, Aaron Ward’s take that Carolina’s unique culture was what led them to win a Cup in 2005-06.

That’s an anecdote, not proof of the role culture has in winning.

No one mentions the teams with great togetherness or culture that don’t win anything (and they certainly exist, because only one team wins the Cup every year). Moves that are made to improve morale, leadership, culture etc. are rapidly forgotten if nothing notable happens after them. Also, a good culture can often trail winning (rather than predict it) because it’s a lot easier to have a good dressing room when the team enjoys success. This is true of “confidence” as well. Disentangling these issues is a lot more complicated than looking at the standings and assuming one team, coach or player simply “wanted it more”. 

So culture doesn’t matter? That’s ridiculous.

No, I wouldn’t say culture doesn’t matter. Managing a room of people to ensure everyone pulls in the same direction is a challenge in every walk of life, so it’s likely a factor in hockey as well. The problem is, culture and similar fuzzy concepts (like leadership and heart) are often wielded as fudge factors or deus ex machina devices that explain away certain outcomes after the fact when it comes to this stuff. 

Here’s the problem from a stats/evidence perspective: culture and such is almost always offered as post hoc rationalization and not as a useful, predictive factor in advance. 

To understand the role and effect of culture on winning, it would take careful, systemic study of what aspects of teamwork and leadership actually lead to improved outcomes. This is possible, but NHL teams and traditionalists haven’t done this (as far as I can tell) – they tell stories about what happened and assign roles to actors as heroes, villains etc. in the events afterwards. This is prone to misattribution, scapegoating and over- or underweighting of things because no one can really be sure of what matters and what doesn’t. 

In short: culture no doubt matters and it might matter a lot in certain ways. But we don’t really know what, how or why or how to use it to predict success.

Is there anything else to this feud? How about the jock vs nerd divide?

Yeah, there’s another aspect to things like toughness, hitting, size, tough to play against – i.e.; the various factors that still tend to be relatively overweighted in the league. They are part of a particularly masculine “might is right” psychology that underpins all competitions of this nature.

Sports are a proxy for war, sociologically speaking, so hockey men tend to eschew things that make them seem “lesser” – small, weak, vulnerable to physical intimidation – and value things that make them seem dominant – size, strength, aggression, toughness. This bias operates in society in general. Taller men are more often chosen as leaders and tend to make more money (and get more dates).

Being big, tough and mean subconsciously grades a guy as a warrior. Being smaller, less aggressive, more “childlike” marks a guy as someone vulnerable and in need of protection in our lizard brains. That’s why small guys have to prove they can play hockey and big guys have to prove they can’t in the NHL.

So only the stats guys are “right”?

No. I’ve mostly represented that side of the debate because that’s where I come from and best understand. It’s also notable that hockey coaches and decision makers have been making decisions in the absence of “big data” for a majority of the league’s existence, so they created methods and norms that probably best suited what was, or rather has been, available.

Next, know that the advanced stats side isn’t a single, monolithic group (there are on-going battles amongst new school guys going on all the time, but those don’t tend to extend beyond the community), so what is “right” is often in contention on that side of aisle, too. Remember, also, that new school guys don’t have perfect data or knowledge either.

GMs, coaches and players are acting on different information and often under different sets of incentives and pressures. For example, minimizing the role of “luck” in personal and team accomplishments in hockey is probably an adaptive response – believing you are in the arbiter of own your fate instills an internal locus of control, which helps motivate individuals to try hard, strive for me, opt for self-improvement, etc. 

The stats side may also have to come to accept that the ideal team on paper is entirely impossible to build for practical reasons that include the human, messy, intangible side of the equation. Personalities, demands, desires, egos and expectations can hopelessly complicate things in the ground but be completely imperceptible from the air. 

As mentioned, “culture” tends to convey embedded truths and experience, though the challenge is separating dogma and the credulity of blind convention from behaviour and beliefs that are actually useful. This is noteworthy because what was once true for a people, tribe, culture etc. may not necessarily apply anymore because things inexorably change and evolve. This will also apply to the stats side of things as “corsi” and such become part of conventional wisdom – it’s easy to stop evaluating your assumptions when they are widely established as true.

The next phase of evolution for both sides would be to identify what is useful from the other side of the divide. The team (or analysts) that can effectively view the forest and the trees will be able to sidestep common errors, identify market inefficiencies and integrate empiricism (test and prove, test and prove) throughout the organization, from scouts and GMs to the coaches and players. 

Maybe at that point we won’t see these fights anymore. Not that fans, coaches and analysts won’t always have different impressions and opinions about the game, but maybe we won’t have to talk about the fundamental ways in which we approach and think about it.

  • Corbs

    I think one of the biggest problems is that often the people talking about stats have never played the game. Also, it’s fairly easy to come up with a stat to back up any agenda. Doesn’t make it valid though.

    • IM80

      I think one of the biggest problems as well is the assumption that because you haven’t played you can’t formulate an educated opinion based on data…(granted, there are some dummies out there who come up with some far-fetched assumptions based on their “data”)

      Have you ever heard anyone suggest that a structural engineer can’t have an opinion because he/she has never built a building? No, because that would be ridiculous…

      • Shameless Plugger

        No, but I have built a building where I was told by an engineer that said plan would work when in reality there was no possible way it would.

        It Goes both ways.

        I don’t discount analytics. But for the most part I find them to be confusing. So I just skim over them. Doesn’t bother me if they’re applied to an article and doesn’t bother me if theyre not.

        • 100% agreed. There is merit to both sides, as well as drawbacks……it’s about finding a balance.

          I also agree some of the stats are completely ridiculous….and confusing, like you say. I would guess that folks that came up with those stats had never played the game.

          The funny thing is, the original fancy stat was invented by someone who did play the game….

        • Gravis82

          exactly. It goes both ways. I doubt that any analyst has ever said that his/her results were the be all end all of hockey. They are simply something new to be understood and to be incorporated into decision making. The level at which they lead to actual good decisions being made determines their influence and longevity.

      • Corbs

        I’m not saying those that haven’t played can’t form an opinion. I’m saying those that have played can likely form a more accurate opinion. Also, there are a lot of stats guys that build their opinion on the numbers without even watching a game and that is ridiculous.

        As far as building a building…I don’t have an opinion because I have no idea what that entails. And I sure as hell can’t look up some stats and pretend to be an expert on it either.

    • hockey1099

      Not only talking about stats but also gathering the data. If you don’t play the game you will never fully understand it. The biggest problem I have with these stats is that they are gathered by the same individuals who interpret them. Their Bias’ will filter into the stats they gather. I have no idea on the consistency of the data gathered. If I have 20 of these guys gather data on the exact same game how close will the stats they generate be? What’s the variance in data collection?

      I’m clearly in the old school camp and don’t see much usefulness in any current advanced stat. I enjoy the beauty of the game, It’s flow, its timing, and the vision the players have. The game live is so much better than on tv. Playing the game is so much better than watching it.

      Breaking the game down after must take out all the beauty out of it. And it’s useless. Let me tell you McDavid knows which side of the ice the weak defencemen is on. He knows who is slower. He knows which side attack.

      As a fan I don’t need stats to tell me mcDavid is elite and that klef is better than gryba. I can get all this from watching a game which sounds infinitely more fun than sitting in front of a screen replaying tape for hours trying to calculate stats.

  • Druds

    I think stats nerds problems usually start when they fall in love with their analysis and then scream and shout when coaches and GM’s ignore their brilliant player and/or trade them or mistreat them… i.e. Martin Marincin who was supposedly next to Bobby Orr according to some stats guys really is barely a third pairing guy.

    • Gravis82

      I believe the advanced stats guys said keep Marincin because he has the potential to be a solid NHL D man, with perhaps a likelihood of being fairly above average.

      I would say that so far, that is not incorrect. He is a solid NHL D man. We have needed some of those recently. Probably should have listened. Maybe would have not have needed to trade for Reinhart if we just kept on with Marty.

  • Gravis82

    Why are people so scared of alternative perspectives? That’s all this stuff is. A different view of the world for you to consider. How exactly is that a bad thing?

    • Reg Dunlop

      Everyone in hockey uses stats in one form or another. The problem lies in the general pretentious douchenozzlery exuded from every pore of the new breed of ‘advanced stats’ adherents. And, of course, the resistance to embrace new ideas on the part of veteran hockey observers. The soloution? A coal-miner’s glove match, of course.

    • Corbs

      Not every alternative view is worth being considered. Many examples of this throughout history. Just because someone has a “new” way to look at something doesn’t make it worth considering.

  • Rock11

    I think the issues are actually much more fundamental than the author or any of the comments have mentioned. Seems to me the old school guys are more interested in why something happened and the new school guys are more interested in why something WILL happen.

    • A Little Less Concerned

      I like how you put that. Although I don’t think that a”stats” guys thinks they CAN predict what “will” happen, it does seem like that’s the motivation of why they do what they do. Attempt to get to the point they can predict the production or value of a player.

      There are strengths on both sides of the coin and your comment kind of illustrates the bridge that needs to happen before analytics can really take off with the old school crowd.

      If I’m a GM and I want to take your theory serious, show me many many examples of how your theory would have (at least come close) to predicting previous outcomes. Then make some predictions and let me ride it out and see what happens.

      And if I’m an analytics guy, not only do I want more raw data than is available now, but I want to be convinced of “by the eye” decisions. I want to know why someone can make a decision because they saw him good.

      An analytics guy can say that so and so does real well in certain situations, but he/she can’t explain the true value and impact on the team as a whole of seeing a player take a clearing attempt to the teeth, rush back, get stitched up and get his arse back out there.

      They can’t quantify having your captain with two blown out shoulders, barely able to carry his stick after a game suck it up and put everything on the line during a playoff run.

      That’s what the old school guys want to see. If analytics could pull that off there would be no rift from the older generation towards the stats guys. But conversely the stats guys want to see the data where that matters. Then maybe they can come up with another fancy Stat that says “Smitty had 13 hearts and that equates to 6.2 more wins, and Smith had 11 hearts and that equates to 5.6 more wins and the team overall had 125.2 hearts so needs a small infusion to get to above average”

      There is room for both but until there is so much data that the analytics crowd “can’t ” be wrong, there will always be that rift.

  • Curcro

    PDO doesn’t at all equate to luck in my opinion. PDO operates on the assumption that your average NHL player is average and is going to have average results.

    However, when you dig deeper not all players are average.

    For instance in the discussion surrounding Adam Larsson the narrative is that Corey Schneider buoyed his PDO; which is a narrative that is more causal than luck.

    I don’t think there is a lot of luck; I think there are a lot of variables, and rather than measure those variables – the ones that are not easily measured get dumped into the “luck” bin.

    The truth is that not that there is a lot of luck; but that the “stats” guys are either a) Lazy or b) don’t have access to all the data.

  • Derzie

    Anyone who chooses a “side” is wrong. Understanding something is about information (all of it) and interpretation. Throwing perfectly useful information away because of a bias is careless. Calling ANYTHING that is unknown “luck” or “randomness” is a short-sighted interpretation. Watch the game, enjoy it at whatever level suits you. It’s a big world with lots of variety & almost none of us run NHL teams so don’t take your conclusions too seriously.

    • Rock11

      So you completely disregard luck as a factor? Smytty’s goal off his face is not luck? There is absolutely luck and randomness in the world and specifically in hockey. This is particularly true when discussing an individual player. If player X just steps on the ice for every goal that Dubnyk whiffed on and player Y had just made it to the bench then Y looks much better than X by traditional measures. PDO, and thus luck, normalizes that so we can measure them against each other in context. Ignoring that altogether is short-sighted and lazy in the same way as ascribing all outcomes to luck.

    • Dobbler

      I don’t think most stats guys call “the unkown” luck or randomness, I think they call “randomness” randomness. It’s a real thing. If you have a giant amount of data, you can find patterns, and there will be things that sit outside of those patterns. This is true of virtually any large data set describing anything in nature.

  • A-Mc

    I love stats.

    When i play a video game, the stats that you see are real. Every event and action is recorded and therefore can be turned into a stat. I love stats in video games because they are 100% accurate!

    I dont love stats from human beings, who are watching a performance and are interpreting it in a way that will eventually turn into a stat.

    I dont want shot metrics telling me which end of the rink the puck was in most. Shot metrics don’t tell me that definitively. Shot metrics tell me the puck was shot, and that’s it. I want REAL O-zone, Neutral Zone and D-Zone metrics, in minutes:seconds, that are indisputable; like in a video game.

    I want heart rate monitor stats, so that i can accurately see that the athletes on the ice were worn down physically due to: Physical play, Shift Length, Over playing the puck, Illness or possible substance abuse.

    I want puck sensors that accurately track Zone time down to the millisecond.

    I want shot metrics that also include puck velocity. Do hard shots score more than accurate shots?

    There are so many things that aren’t being measured right now that i find it really hard to put much weight in statistical analysis. Conversely, i also recognize the narrative angle as being inaccurate.

    For me, i take a little bit from both worlds and i sit with the hope that one day we get all the accurate info we could ever need or want.

    • I don’t understand the point of this comment.

      You know who wants these things even more than you? People who analyze statistics. It’s not available, but the second sportVU is applied to the NHL and the data becomes publicly available analysts will celebrate more than anybody.

  • madjam

    Nice thing about team sports is they don’t run true to form . That’s the allure of entertaining sports . If they did , then interest in them would be minimal and not worth going to , for lack of entertainment value . Minimum stats is all fans need to enjoy the sport .

  • YEGswede

    You could probably put me in the “analytics” bin of people. That being said, I think that the stats we are using at the moment are suspect at best when it comes to predicting future results. Yes, there are some (usually shot based) metrics that can give insight to why something happened in the past, but the correlation levels are still not super high . With the flowing team based nature of the game, we might never get the metrics that are so powerful that I would be comfortable dismissing someone who doesn’t think that analytics tells a better (more accurate) story than the traditionalist view.

  • I love hockey and consume everything about it. Including stats articles in the summer. It’s just information to go along with my “eye” wisdom. I find it odd that there are two camps on both sides that refuse to acknowledge the others point of view.

  • Seanaconda

    As for stats in the game go it’ll be interesting to see how chakya does with Arizona being the first full on stats guy promoted to GM. The Panthers seem to be going pretty full in on analytics as well.

  • Bills Bills

    I got about half way through this absolute crap of a post and couldn’t go any further. It is clear that who ever the person who wrote this article had clown for breakfast and think he’s smarter than many of the people on this site. I actually found your blog to be full of your ego and very deflated on facts.

  • RJ

    This article would have been really current five years ago. Lol

    For all the early love given to Corsi and Fenwick, the traditionalist view of getting to the tough areas of the ice seem to be borne out by shot percentages and the corollary of save percentages relating to high danger scoring chances.

    And likewise, while everyone can agree that there is always some randomness in the game (like that goal the Flames scored from behind the goal line on Talbot early in the season), there is often very little focus given to the role that coaching strategy can have on the game. If you know Ovechkin is very effective shooting from a particular spot, any coach can tell you that you need to take that spot away from him. Is his subsequent decrease in shot percentage attributable to mere randomness or the strategy of taking away his bread and butter play? (This was an actual debate a while back on Dellow’s old site and he didn’t have an answer.)

    And the stats community has no cohesion when it comes to managing certain stats. How do you calculate controlled zone exits/entries league wide? Every team has their own way of figuring it out. Just defensemen, or do you look at which forwards support a controlled exit best? For controlled zone entries, what counts as a controlled zone entry? btw, these were areas where Hall was brutal. He’d cheat during breakouts, or he’d turn it over at the blueline regularly. But the stats community here was largely silent. They just wanted to count Hall’s shots. He “pushed the river” so let’s overlook his flaws.