30
Photo Credit: Perry Nelson-USA TODAY Sports

What is best way to evaluate NHL players?

Statistical evaluation of players has always been the cause for great debates. It is impossible to find one evaluation metric which would appease the entire hockey world. Wayne Gretzky (2857) has 943 more career points than the second highest scorer, Jaromir Jagr (1914), yet some people still believe there were better players. Hockey players, management, coaches, media and fans like different players for various reasons. This will never change, but as we incorporate more analysis and statistical data into covering the game, is it possible we are getting closer to a more precise evaluation tool?

Over the weekend, Dom Luszczyszyn (an outstanding name to use in sports spelling bees) wrote an article for The Hockey News outlining NHL star power.

He had five tiers of players, ranging, in his words, from #1 “Year end award winners” to #5 “Very good players who are borderline stars in the right environment.” He finished with a total of 125 players split between 80 forwards, 37 D-men and eight goalies. Tier one players received five points, tier two four points, all the way down to one point for tier five players.

null

His calculations came from Game Score which consists of in short form:

“The stats I used are goals, primary assists, secondary assists, shots on goal, blocked shots, penalty differential, faceoffs, 5-on-5 corsi differential, 5-on-5 goal differential.

“The second step is figuring out how to weigh each component. To keep things simple,  I opted to weigh each stat by its frequency to goals and round up/down to keep the sig digs low.”

I believe it is great to have conversations about which stats are the best when trying to evaluate players.

The challenge we face as we try to incorporate metrics beyond goals and assists into the game is first off, how to accurately track their value, then, which ones are a true reflection of the individual. And finally: accurately tracking them in a game that moves so quickly and does not have the same starting point. Baseball works best for statistical analysis because every play starts at the exact same place: in the pitcher’s hand on the mound.

Hockey is the complete opposite. It can start on nine different faceoff dots, but more importantly it can be fluid for minutes at a time, with hundreds of variables occurring before the next “controlled” starting point.

For context, Luszczyszyn based his rankings on the past three years. I agree a three-year window can work better than one.

I applaud him for putting in the work to come up with it. Some rankings made a lot of sense, others had me look into a player, and I realized he has been quite good. And there were also some rankings which made me question the model.

I’m not sure using faceoffs is a great tool for ranking star players. It really only works for centres, and while faceoffs have value, many of the best players in the game don’t excel at them, yet they still manage to dominate play.

Also, separating primary and secondary assists can be a slippery slope. The primary assist on the scoresheet isn’t always the player who had the puck prior to the goal scorer. And sometimes the “secondary” assist is the best pass of the two, and without it starting the play, the guy who gave the puck to goal scorer wouldn’t have been able to make that play. I understand the premise of first and second assists, but I question the accuracy or difference in importance.

My bigger question, however, was after adding his formula…

Player Game Score = (0.75 * G) + (0.7 * A1) + (0.55 * A2) + (0.075 * SOG) + (0.05 * BLK) + (0.15 * PD) – (0.15 * PT) + (0.01 * FOW) – (0.01 * FOL) + (0.05 * CF) – (0.05 * CA) + (0.15 * GF) – (0.15* GA)

…there were some noticeably questionable rankings.

The first one for me was Evgeni Malkin. He would have been in the first tier. I realize he was in the second, and maybe because he’s only played 188 games the past three years, compared to 225+ for other superstars like Patrick Kane, Sidney Crosby, that was why. It was hard for me to see a ranking where Malkin not in the top-13 forwards in the NHL.

When we come up with a formula to rank players, if we see a few obvious errors, should we re-tweak it?

Posterior checking is what made Bill James such a revered statistical figure. He was always challenging findings and he was brilliant enough to either fix them or explain why they were incorrect.

It is easier said than done, and I don’t have the perfect formula for the above chart, but I think starting the conversation could possibly get us closer.

Here were the other rankings that had me questioning if this formula was incorporating the most important stats.

  1.  David Pastrnak was the seventh highest ranked player. He had a great season with 34 goals and 70 points. I like him as a player, but he had 53 points in his first two seasons in 97 games. He had a high P/60, but that should not be considered more valuable than actual points. He is 148th in points the past three years. I believe he will be very good, but this ranking seemingly had too much weight based on P/60, one season and his linemates (even Dom agreed with the latter). Boston had three of the top-seven forwards. I’d strongly argue they don’t have one of the NHL’s ten best forwards, nevermind three in the top-seven.
  2. I’m a big Taylor Hall fan, but he is not a better left winger than Jamie Benn. Benn has scored the 3rd most points in the NHL over the past three seasons. Is point production more valuable than shot attempts? Hall generates a lot of shots attempts, and he doesn’t play with Tyler Seguin, which helps Benn, but when we evaluate, what would be the best percentage to use to balance importance of goals to shot attempts? I understand the theory of shot attempts and the odds of more is better, and while some will say goals are so rare, it is hard to calculate, but the fact is in the game of hockey they matter significantly. If you are a top-three point-producer the past three seasons, I disagree with being the 27th ranked forward.
  3. Ryan Getzlaf was 34th among forwards. He has the 14th most points in the league the past three seasons. He has a very good goal differential. Dom told me Getzlaf’s low goal totals and his age hurt him under this formula. If so, how do we alter it? Should age be a negative predictor, until we actually see the drop off? We’ve seen many elite players stay great into their 30s.
  4. Drew Doughty and PK Subban are straight up better than Mark Giordano or Dougie Hamilton. The gap was minimal, but Giordano and Hamilton were rewarded for playing together. Hamilton and Giordano are very good, and as a pair, they played very well this past season, but they aren’t the fifth and sixth best defenders in the NHL. Can we accurately evaluate individual D-men without including their partner? I don’t think so,
  5. How is Phil Kessel (190 points) not among the top 80 forwards in the NHL, when he has produced the 23rd most points among forwards the past three seasons? Teammates Connor Sheary (63 points), Patrick Hornqvist (146) and Jake Guentzel (33 in 40 career games) are ranked ahead of Kessel. I understand part of these rankings are projections, but how do three teammates in Pittsburgh jump ahead of Kessel? Guentzel was excellent in his 40 regular season games and the playoffs, but Kessel also had 23 points in the playoffs to Guentzel’s 21. The calculation seems off if Kessel’s point totals mean so little.
  6. Similar question regarding Sean Monahan in Calgary. He has the 27th most points among forwards and the 17th most goals in the NHL the past three years. He didn’t crack the top-80, but his teammates Matthew Tkachuk(13-35-48 as a rookie), Michael Frolik (156th in points) and Mikael Backlund (138th in points) did. I like Backlund a lot and he is a very good two-way centre. He had 16 PP points to Monahan’s 17 last season. Backlund’s linemates were all plus players and had solid possession numbers. So does fewer goals and points equate to that much less in value to solid possession numbers? For me it would seem the equation leans too strong towards the latter. Would it help to balance out the value somewhat?
  7. Johnny Gaudreau was also ranked, as he should be, but he was actually ranked even with Tkachuk. Gaudreau has produced the 15th most points in the NHL the past three years, yet Tkachuk, after one 13 goal, 48-point season, was ranked the same. I’m sorry, but that makes little sense to me.
  8. Cam Atkinson was ninth in the NHL in goals this past season with 35. He has scored 84 goals during the past three seasons, tied for 18th most in the NHL. He is 60th in points. He was 21st among forwards in SOG this past season. His goal differential was three over the past three seasons, while he had a CF% of 48.8 over the past three seasons. It seems the latter two cost hurt him more than his goals and points did.
  9. Marc-Edouard Vlasic was the best defender at slowing down Connor McDavid this past season. His gap control was almost flawless. His role in SJ is to try and shut down McDavid and other top scorers. With Brent Burns in SJ, Vlasic doesn’t get much PP time. He had 20 EV points this year. Giordano had 22, Subban had 23. Vlasic had 27 EV points in 2016, 18th most among defeenders. Despite playing top players he has had very good goal differential the past three years, 31, and his possession numbers are pretty decent as well. There are so many young defenders ahead of him on the list who are not nearly as competent as him in his own end.
  10. Despite my lack of love for the position, are there really only eight goalies in the top 125 players in the NHL? Over the past three seasons, Cam Talbot has started 160 games, ninth most in the NHL, and of the goalies with 150+ starts, only Braden Holtby, Devan Dubnyk and Corey Crawford have a better SV% than Talbot’s .920sv%. Gibson and Anderson have a .921 SV%, but they have started 25 and 52 fewer games than Talbot. It is more difficult to maintain a high SV% the more you play and the more shots you face. Talbot has faced 1,700 more shots than Gibson the past three seasons and 600 more than Anderson. Talbot was a Vezina finalist and without question the second most important player on the Oilers last season.

Luszczyszyn wrote this below his ranking chart.

There are some surprises sprinkled throughout, but I do think this list works for the most part (fans from certain fanbases may disagree and to them I say yes, I do hate your favorite player and/or team). Most of the surprises are likely due to teammate effects which are difficult to suss out, making some players look better because of who they play with. Overall, it feels like the majority of players are ranked correctly, though I’m a little biased because it’s my model.

He recognized there were some errors due to teammate effect, but wouldn’t it have been better to adjust the rankings based on that, state why you did, and then post it?

His ranking is a good talking point. Name calling and insults accomplish nothing. His ranking has many accurate player projections, but I think we could take what he started with and tweak it to get a more accurate ranking.

I would consider making all assists equal, or at least until every NHL game sheet writes them down the order they occur, and then I could see the argument to have a slight difference, but if it drops Getzlaf that low then I’d alter it again.

I’d eliminate faceoffs, unless I was ranking only centres. And blocked shots, I’d only use for D-men. Admittedly, I dislike how much coaches ask players to block shots, so if someone wanted them out I wouldn’t argue.

To further the discussion we need to look at devising a formula that can be more accurate.

If you have any suggestions, feel free to post below.

Play Fantasy Golf with Draft Kings

  • Fantasy Golf Millionaire Contest:
  • First place wins $1,000,000
  • Over $3,000,000 in total prizes
  • More than 25,000 entries win money guaranteed
  • Only $33 to enter
  • Simply select a team of 6 golfers to play
  • Must enter by first tee time on Thursday morning

Recently by Jason Gregor:

  • Shredder

    Ovechkin is tier 2? That’s a surprise. Where’s Stamkos on this list? How would you rank Crosby after missing the first 66 games due to a concussion?…if hockey were a one stat league, we would have figured it out by now. The reality is in recent years the Stanley Cup has been won with a defense first team, all offense all the time team, grind it out team, etc. – there is no one way to do it. That’s what makes it so good, and what will drive stats nerds up the wall.

  • Nanook

    Makes me think the creators of this system were all sitting around a table passing the good stuff. Like that 70’s show. Very little of that chart makes much sense. I think they will need to share what they are on for me to agree with even half of what they are saying.

  • “To further the discussion we need to look at devising a formula that can be more accurate.”

    Watch the game while drinking bevies with your buddies. Hunt for hotties when its over. Who knows, it might catch on.

  • Averagewhitebread

    Larsson is a more valuable defence man to the oilers than Klefbomb. I think that’s the same reason Ryan Suter is a tier 5 even though he averages top 10 in minutes every year

      • Averagewhitebread

        I think Larsson and Klefbomb have fairly even value to us but I can see Larsson stepping up to the bigger minutes next year in Sekeras absence. That’s my two cents, I’m not the analysts

        I hope We can at least agree the oilers would be worse off without either of them.

        • toprightcorner

          This model is not about most valuable. Klefbom and Larsson are close to equal, Klefbom gets the nod because of offense. Klefbom is just tapping into his offensive ability and he will likely scor over 40 pts next year and will probably become the PP QB the Oilers have been looking for.

    • Samesame

      I’ve come around on the Adam – Taylor trade. Larsson is a solid defender with some quality intangibles. Good 2/3. But there’s no way he’s more important to the oilers than klefbom. Klefbom is the team’s best defender by a country mile and brings far more offensively as well as being comparable defensively

      Klefbom is the most important player to our overall success going forward Behind only Connor and talbot.

  • Jay (not J)

    I don’t get these efforts sometimes. Fuel for the kibitzing set, I guess.
    Hey, statistically speaking, who was the Worst Oiler of all time? Call it the Honorary (LOL) Flame Award.

  • ubermiguel

    It’s a fun experiment, I applaud Luszczyszyn for trying something new, but I think there is a fundamental flaw that can’t be fixed: what the hell is this “star power” he trying to measure?

    He makes several assumptions that undercut the validity of his measure: (1) “star power” is something; (2) “star power” can be measured; (3) “star power” is essential to winning; (4) “Game Score” is a valid and reliable; (5) the individual stats of the top five players drives team success; (6) each position is equally important to winning. And those are the issues just off the top of my head.

    I would ask him to rephrase his research question as “what are the individual player traits of the best players of each team that lead to team success”. Tackling that question might yield some valid and reliable results.

  • Rock11

    Very little to argue in the forward rankings IMO. Of course there are outliers(Malkin) but otherwise really solid. The Bruins guys look like they got a bump due to faceoffs and the Bruins style of play. while w all think the big names like Getzlaf and Ovechkin should be higher if you dig a little it makes some sense. Ovechkin has always been dinged for lack of 2 way play and I would imagine that shows up in his possession numbers and GF%. Plus his offense isnt quite what it used to be. Getzlaf on the other hand has had some serious scoring slumps in the regular season over the last few years. We all saw what he can do in a short series but any model that weighs seven games more heavily than 3 seasons worth of games is seriously flawed.
    Now those D rankings on the other hand. That is just hopelessly broken. D partner and usage vs elites and in the D zone look to have been completely ignored.

  • toprightcorner

    At lease he didn’t use +/- in the model.
    There are far better stats out there than Corsi that could be used, like Dangerous Fenwick. Corsi differential really doesn’t work well to compare players on different teams based on line mates and roster quality.

  • toprightcorner

    I think there are a lot more than a few outliers, at best I would say he would be around 60% but it is a start and hopefully he, or someone else, will improve it.

    Head scratchers

    Trochek and Arvidsson better than – Benn, Draisaitl, Backstrom, Toews, Getzlaf to name a few

    Nylander better than Toews, Getzlaf, Gaudreau, Laine and about 40 other players. How does he make it and Kessel not?

    Thachuk – apparently he is the best Flame forward – hahahaha

    Mantha the best Red Wing – Heeheeheehee

    Guentzel better than Couture,

    Werenski better than Kieth and Pietreangelo

    Trouba better than Carlson

    Slavin better than Webber

    Leddy better than McDonagh

    Barrie better than Vlasic (Vlasic not in the top 15!!!)

    Definitely more than a few problems. The biggest issue is an analytics guy not admitting their models can be improved upon and it sorta sounds like this guy thinks he is real close to the end product when he isn’t even within shouting distance

  • JimmyV1965

    Very confusing rankings. I Don’t think he’s accounting for injury. Barkov is in tier two, but I think
    his health is a huge question mark. What’s puzzling though is the only reason I can fathom Malkin being outside the top tier is injury. It appears that Malkin is penalized for injuries, yet Barkov isn’t. Guess it’s basically summer fodder for us hockey nuts.

  • hockey1099

    Summer is the worst. The stats guys sit there and crunch numbers and think they look smart. If I’m Calgary I trade Hamilton to Montreal for Weber right now based on the above evaluation Montreal should be extatic to make that trade.

  • 1979

    You nailed it here Gregor! This ranking shows how advanced stats guys can be so swayed by Corsi that they miss the big picture and evaluate incorrectly. I remember after the Larsson trade some trumpeting Severson as the better d man. I think this year showed that to be false and part of it was the first hand look at what Larsson brings to the table.

    Another thing I noticed in the list though that bodes well for Edmonton likely is that Boston has clearly done some good drafting in the past…

  • RedMan

    no stats are perfect, each analyses give a glimpse from another angle.
    Oiler fans, gotta quit being so sensitive to any snapshot that doesn’t rank your team first in the known universe.

  • Russky

    Besides regular stats that already used by NHL it is very difficult to find a better way to evaluate a single player in a team sport beyond Superstar players (McDavid, Crosby, Malkin, Ovi etc). As most of the regular NHL stats are going to calculate “Newly created” evaluations.
    To create a better player evaluation it wold be good to find a way to exclude “meaningless” points (ex: goals 5-6-7) in last 5-10 minutes of a 7-1 or 7-2 game. As i do not think opposition is playing at 100% during that time. Also first goal scored, game winning goal and game tying goals can not be ignored. Also evaluation of F and D should be separated and actually used in evaluation formula for one another. Goals and points in back-to-back games are also come little bit harder than goals with couple days of in-between.
    In order for math go be accurate, one has to include all the numbers, otherwise math is not going to be on your side.