The Value Of An NHL Draft Pick

Jonathan Willis
March 10 2011 06:58PM

 

 

Late last month, Jason Gregor looked back at the NHL draft from 1996 to 2005, in an attempt to put a value on NHL draft picks. Gregor’s far from the first to do so; after the jump I’ll look at his work and that of others to see what we can say about the NHL Draft.

Jason Gregor’s Take

Let’s start with Jason’s article. There are a lot of things to like about it; in particular the fact that the dataset is huge (10 years, ~2700 players) and the criteria is relatively straight-forward. That said, there are two things that I think could be improved: first, there is some subjectivity in the player ratings, and second Gregor breaks things down by round, meaning that we don’t know how much difference there is between a top-10 selection and a 10-20 selection. The latter is more of a problem than the former, as we will see later on.

It’s still quite a good article, and the key points I took from it were as follows:

  • A first round selection is far more valuable than any other selection – a player taken in that range has a slightly better than 60% chance of turning out to be a “decent” or better player; nearly three times the rate of the second round.
  • Second and third round picks are a cut above the rest of the draft, although the chances of landing a “decent” or better player are quite low: a little better than 1 in 5 for the second round, and slightly better than 1 in 7 for the third round.
  • There’s little difference between picks made from the fourth to ninth rounds of the draft; over those rounds players are roughly half as likely to develop into a “decent” or better player as a third round pick.

Scott Cullen’s Take

Over at TSN, numbers guy Scott Cullen did something similar to Gregor’s article in mid-2009. Like Gregor, he used a 10-year period (1995 to 2004) and subjective rankings. However, Cullen broke the draft numbers down into groups smaller than full rounds – groups of five for the first 60 picks, and then groups of 15 afterwards. A few months later, Cullen clarified things even further for the first round – using 30 drafts and breaking down the top thirty picks by each draft spot.

The key points I took from Cullen’s work:

  • There’s a huge gap between a top-three selection and a four through six selection, which is followed by another gap between the sixth spot and the rest of the draft.
  • Again, there’s a big drop between the first round and the second/third rounds, and another big drop from about the 100th pick on, at which point things flatten out – there’s very little difference between drafting 105th and 195th.
  • A player taken after the first round but in the first hundred picks has roughly a one-in-three to one-in-four shot at hitting the 100-game plateau in the NHL.

The Hockey Prospectus Take

Again from the summer of 2009, articles by two different writers at Hockey Prospectus. Richard Pollock considered both first round selections and late round selections, while Tom Awad considered the entire draft. Pollock uses both points and games played, while Awad uses Goals-Vs.-Threshold, a unified statistic that compares the value players of all positions generate above replacement-level players.

All the articles are good, but Awad’s method does the best job of comparing players across position objectively, and I like the fact that he weights his findings to reflect the fact that immediate production is better than later production. Plus, his fitted graph does the best job of showing how precipitously draft pick value drops off.

Key points:

  • The first few picks in the draft – especially those in the top five – are vastly superior to other picks, including picks later in the first round.
  • Once again, picks outside the first round but in the top-100 are roughly equivalent in value – there is some value in having a higher pick, but that value is not especially strong.
  • After the first 100 or so picks, there is almost no difference in the value of a draft pick.

Conclusions

Despite the different methods used and the various writers employing them, the findings from one article to another are extremely similar. I would argue the following, based on the data presented:

Get top-five picks. The top picks in the first round are very, very valuable. Teams moving up into this range cheaply are getting real value. Generally it costs an arm and a leg to acquire a marquee pick, but there are exceptions – for instance, Tampa Bay’s trade of the fourth overall pick in 2002 to Philadelphia for Ruslan Fedotenko.

After the top-five, move down in the first round. Teams trade up all the time to snag players that they like in the first round, but as a rule it is a very bad idea. Teams trading down don’t really lose much value, especially if they aren’t trading down very far.

Top-100 picks outside the first round are not especially good, and are virtually interchangeable. These picks are all about quantity over quality; good players are there to be found but the odds are stacked against any individual selection. Trading a mid-second round pick for two third round selections would seem to be a wise move the majority of the time.

Picks outside the top-100 are all but interchangeable. These picks are all about quantity. Every selection is a long-shot, teams play hunches and snag the favourites of regional scouts, and basically there is very little to separate the players involved except the biases of individual teams and who their scouts have seen.

Contending teams can trade mid- to late-round picks with minimal reluctance. Draft picks after the first round, and especially those outside the top-100, can be spent with almost no hesitation on actual players. For a contending team, acquiring depth or quality via draft picks is an extremely sound strategy.

 

74b7cedc5d8bfbe88cf071309e98d2c3
Jonathan Willis is a freelance writer. He currently works for Oilers Nation, the Edmonton Journal and Bleacher Report. He's co-written three books and worked for myriad websites, including Grantland, ESPN, The Score, and Hockey Prospectus. He was previously the founder and managing editor of Copper & Blue.
Avatar
#1 Krusher Ken
March 10 2011, 07:09PM
Trash it!
0
trashes
Cheers
0
cheers

Take Adam Larsson FIST and get out of the basement!

Avatar
#2 TonyDanzaPervo
March 10 2011, 07:09PM
Trash it!
0
trashes
Cheers
0
cheers

Zach Parise

Avatar
#3 Archaeologuy
March 10 2011, 07:10PM
Trash it!
0
trashes
Cheers
0
cheers

Gagner out for Season with severed Tendon

I told you all that Jones should have been traded. Now his skate cuts down the Oilers' most productive C. It's Cogliano and Horc the rest of the way as our 1 and 2 Centres.

Avatar
#4 James Scott
March 10 2011, 07:11PM
Trash it!
0
trashes
Cheers
0
cheers

Johnathan trading down isn't allways the best stratedgy as Zach Parise has shown us year after year. Draft stratedgy has more to do with the GMs confidence in his staff than anything. It depends if your scouts target the right players or just through darts at the wall. I trust Stu with LAs pick.

Avatar
#5 James Scott
March 10 2011, 07:13PM
Trash it!
0
trashes
Cheers
0
cheers

Throw.... not through

Avatar
#6 DonovanMD
March 10 2011, 07:13PM
Trash it!
0
trashes
Cheers
0
cheers

Good breakdown here. I'm sure some GM's have the same mindset when building their teams and their own little rules like this.

Avatar
#7 James Scott
March 10 2011, 07:14PM
Trash it!
0
trashes
Cheers
0
cheers

Thank god they took Ebs and didn't pull a Pouliot.

Avatar
#8 Next up, is Connor McJesus.
March 10 2011, 07:25PM
Trash it!
0
trashes
Cheers
0
cheers

So, who are the feathers in MacGregors cap besides Jordan Eberle and Jeff Petry, any of MacGregors kids going to help us out next season?

Avatar
#10 D
March 10 2011, 07:34PM
Trash it!
0
trashes
Cheers
0
cheers
Jonathan Willis wrote:

@ James Scott:

I think betting that your scouting staff is way smarter than the average is almost always a bad move. I say that despite Stu MacGregor's excellent record to date; over the long haul I have difficulty believing that the Oilers' scouting organization is either much better or much worse than average.

My suggested conclusions are based on the assumption that any one team can't believe their scouts are brighter than everyone else's.

Jon, I hope the Oilers adopt that philosophy. Rightly or wrongly, I feel that the Oilers management have, for the longest time, taken the attitude that they are smarter than everyone else.

Avatar
#11 Westcoastoil
March 10 2011, 08:01PM
Trash it!
0
trashes
Cheers
0
cheers
Jonathan Willis wrote:

@ James Scott:

I think betting that your scouting staff is way smarter than the average is almost always a bad move. I say that despite Stu MacGregor's excellent record to date; over the long haul I have difficulty believing that the Oilers' scouting organization is either much better or much worse than average.

My suggested conclusions are based on the assumption that any one team can't believe their scouts are brighter than everyone else's.

That's a HUGE assumption, akin to saying "I assume all teams are equal in player development.

There are teams that consistently draft better than others, and I expect that you can find widely varying degrees of $ spent on scouting. 12 very good scouts are going to do better than 6 average ones. It's also reasonable to assume that some scouts are better than others, just as some coaches are better than others. If the playing field on scouting was level then sure overtime it would all average out. Frankly I scouting is an areas where a team can gain a competitive advantage, particularly if it is combined with better player development (eg. not rushing prospects, vets to play with in the AHL, etc.).

This is especially true for picks outside the top 10 where a miss vs. a hit at pick #23 can make a big difference.

Avatar
#12 stevezie
March 10 2011, 08:06PM
Trash it!
0
trashes
Cheers
0
cheers

@Jonathan Willis

I cannot agree strongly enough. I was going out of my mind when they traded up for Riley Nash, and it had nothing to do with Riley Nash in particular. All Lowe did was trade away two birds in the bush for one bird in a different bush.

I once saw a special on Glen Sather talking about what a genius he was, and they asked him what makes a great GM. He said that if you can look at two full team playing hockey and figure out which two or three guys can really play, then you can run a hockey team. What a load of self-serving horse-feathers, this guy drafted Jason Bonsignore. Sather got lucky in the 80s and jinxed since, he didn't know anything more or less than anybody else.

The Parise argument doesn't completely hold because Kesler, Richards, Perry and Eriksson were all still available (so were Bergeron and Weber, but they would've been off the board.) Trading one pick for two is still a sound guideline.

Avatar
#13 John Chambers
March 10 2011, 08:08PM
Trash it!
0
trashes
Cheers
0
cheers

Interesting note Willis that the top 3 picks are usually bona fide, picks 4-6 are less bona fide, and so on ... especially considering that there are 4 names often floated around as the top of the 2011 draft class.

So my question to you, fair nation, is which of the 4 top draft prospects do you feel is the WORST selection of the bunch.

I'm not sold on anyone in particular, but I for one am least sold on Sean Couturier.

What do you think?

Avatar
#14 shau_co
March 10 2011, 08:13PM
Trash it!
0
trashes
Cheers
0
cheers

@Westcoastoil

I agree with Westcoastoil. If you look back at teams success in drafting in the first round, you will notice that some teams perform much better than others (e.g., San Jose, Boston, Philly, St. Louis).

Although it can be a combination of scouting and player development - not just scouting. However, if our measuring stick of success is number of games played it really comes down to the whole package that gets the asset into the NHL as a regular.

Avatar
#15 book¡e
March 10 2011, 08:13PM
Trash it!
0
trashes
Cheers
3
cheers

All of this aligns well with the natural curve (bell curve) that one would expect hockey expertise (skill, motivation, intelligence, etc.) would be distributed by. So the empirical work lines up well with the theory.

However, the other thing that the theory will tell you is that the quality of the top picks will be more chaotic from year to year than the quality of the later picks (basically small 'sample' size effects). So, it would be unwise to assume 'regular' drop off points at 3 and 6 because there will be significant variability in this from year to year.

If Hedman and Tavares were born a few months later, the 2010 draft would have shifted by a couple of picks to the better and the 2009 would have shifted a couple of picks to the worse. This has a huge impact at the top of the draft, but almost no impact at pick 100.

Avatar
#16 stevezie
March 10 2011, 08:30PM
Trash it!
0
trashes
Cheers
0
cheers

@John Chambers

I think, I don't know but I think that Landeskog has the lowest ceiling, but is the surest thing. I really couldn't call myself "sold" on anyone else.

Avatar
#20 Westcoastoil
March 10 2011, 09:48PM
Trash it!
0
trashes
Cheers
0
cheers
Jonathan Willis wrote:

@ Westcoastoil:

Barry Fraser - savant or moron?

Based on his early work with the Oilers, you could argue the former; based on his later work, the latter.

There's also little evidence to suggest that NHL teams can solve drafting issues with money. For example, the Buffalo Sabres have gone the video route, largely in an effort to control cost. It doesn't seem to have hurt them.

Leaving that aside, the principal problem of aligning reports from multiple scouts remains. As head scout, your Swedish scout raves about Larsson, your QMJHL scout raves about Couturier, your your OHL scout raves about Landeskog, and your WHL scout raves about Nugent-Hopkins. How do you weigh those opinions? I suspect that particular problem is a big part of the reason tournament play gets such weighting - it's the one time all the scouts see players from outside their regional bases.

Barry Fraser - savant who jacked a bunch out of the park then mailed it in b/c Cabo beats Moose Jaw in January.

Fair enough that $ doesn't solve all your problems, but just because they are doing it via video doesn't mean that are spending less time/energy on it by better qualified people.

If your theory is based on no one team's scouts being brighter than another, then why wouldn't a bunch of teams just share a scouting staff or use Central Scouting and then go down the list. If all the scouts are as smart as the next guy their input doesn't help you align the reports anyway.

If I have a Swedish scout pushing Larsson, a Q scout pushing Couturier, etc. then I can either: a) go by organizational need, b) go by a scout's track record or c) rotate the scouts by flying the Swede over to watch the Q for a stretch and vice versa so their final reports are fully informed (or use video a la Buf.). Maybe moreso, in the case of a top 6 pick where it clearly matters the most to get it right because that's where you find your impact players.

I get what you're saying as to where the true value is. However, i think an assumption that all scouting staffs are more or less equal sets up a flawed argument right off the hop.

Avatar
#22 Westcoastoil
March 10 2011, 10:03PM
Trash it!
0
trashes
Cheers
0
cheers
Jonathan Willis wrote:

@ Westcoastoil:

Fair enough; I'll re-phrase:

Mostteams can't be confident that their scouting group is significantly better than the average NHL scouting group. In some special cases, where the track record warrants it, individual teams might have basis to believe they can beat the NHL average.

I'd still argue that betting on an average staff is the smart move for the majority of teams.

I think that drafting (and development) has to be a top priority in an organization and question how successful a team can consistently be without it. I think it's one of the areas where you might have the most opportunity to gain a competitive advantage. Because of the cap, having underpaid impact players is a huge key to success. You can only get that through outstanding drafting or (sadly in Edm's case) years of bottom feeding. Putting an extra 2-3 million into scouting gets you way more bang for your buck than trying to plug a hole with a UFA and ending up with Ryder's $4M for 3 years(or whoever).

Avatar
#23 jimbones
March 10 2011, 10:14PM
Trash it!
0
trashes
Cheers
0
cheers

Wow great read. Very interesting stuff. I hope the oilers management does this type of research.

Avatar
#25 Butters
March 10 2011, 10:31PM
Trash it!
0
trashes
Cheers
0
cheers
John Chambers wrote:

Interesting note Willis that the top 3 picks are usually bona fide, picks 4-6 are less bona fide, and so on ... especially considering that there are 4 names often floated around as the top of the 2011 draft class.

So my question to you, fair nation, is which of the 4 top draft prospects do you feel is the WORST selection of the bunch.

I'm not sold on anyone in particular, but I for one am least sold on Sean Couturier.

What do you think?

I agree, Sean C worries me. He didn't look great at the prospects game and didn't make the WJC team. More to it than that for sure but my gut says the Oilers should pick someone else.

Mind you I thought Dustin Buff was going to be the next John Druce. Missed that one by a bit.

Avatar
#26 stevezie
March 10 2011, 11:31PM
Trash it!
0
trashes
Cheers
0
cheers

I guess if you didn't notice he was there he couldn't have been that great, but Sean C was on the WTC team.

Avatar
#27 James Scott
March 11 2011, 12:06AM
Trash it!
0
trashes
Cheers
0
cheers

@ Jonathan Willis

Trading down is a mistake unless you believe your guy will be available do to some overwhelming misconception about the player you want and you recieve a sure thing nhl player or prospect in return . Certain players are allways around later than they should be baecause of a generally accepted opinion Eberle his was size and skating, Parise size and Paajarvi finish Fowler toughness and shot. Thank goddness they didn't trade down and pass on those guys for a extra 2nd..... Opps, is 2 out of 3 bad. All kidding aside if the oilers had gotten a real player in that deal not a handful of magic beans it wouldn't have hurt so bad. Lowe and Prendergast had a bad 1st rnd record, as did Fraser after the first few years(The top picks are all well known prospects now days 1sts should not be off the board selections). Their 1sts reeked of arrogance like they had the whole world beat and everyone was missing something especialy Fraser(Doan and Miknov for example). The Lower the pick, the less chance of success period if you only look at one scouting staff(judging Team A vs Team B it is impossible to get data that means anything because a good staff will in all probability do better the higher they pick), Edm is going to have a better draft across the board when they finish 30th vs 15th, rnd 1,2,3,4,5 these are breaks that have nothing to do with talent(a title for the player selected). Talent in any year goes from high to low, the variations in success come from scouting bias or quality, post draft development(handling) / injury / luck(everyone can get lucky, on the Oilchange pilot it showed how lucky Fraser was when it showed the oilers 79 draft list), the first two can be controlled the last two can't be foreseen. IMO the best and worst staffs are probaly both getting guys that are way higher on thier list in later rounds. Being able to assess which players can overcome what ever warts in his game has caused their slide, (Hamilton it was injury, pitlick played for a defensive team hurt his offense, Martindale effort/maturity, Davidson late bloomer, Blaine late surge in the season, was it a anomoly) is the differance? I don't know if these guys are available 5,10 or 15 picks later, another team with a talented staff might have grabed them. Trading picks for picks doesn't make sense in most cases, it only works for the best scouting staffs as you have to have a specific target and you can't be wrong or you are most likely going to get burned and the optics are bad. Lastly IMO not taking the BPA is the downfall of most teams.

Avatar
#28 Westcoastoil
March 11 2011, 12:12AM
Trash it!
0
trashes
Cheers
0
cheers
Jonathan Willis wrote:

@ Westcoastoil:

I believe in both drafting and development, but I think people forget how important support UFA's are. Detroit is the team I watch closest for management trends, and one of the things they've consistently done is grabbed value UFA's for depth roles - guys like Mikael Samuelsson and Patrick Eaves.

It's one of the reasons they're so consistently good.

I agree Detroit is a good team to watch for mgmt, trends. By consistently drafting their impact players: Lidstrom, Datsyuk, Zetterberg, Franzen, Kronwall, Howard, Ericsson it then gives them increased flexibility when it comes time to plug a UFA in because the depth guys are more interchangeable.

As one of the few teams who've stayed on top without a top 5 pick, I'd argue they've made their hay by having a scouting staff better than most teams.

Good debate JW

Avatar
#29 Alex Hemsky
March 11 2011, 08:13AM
Trash it!
0
trashes
Cheers
1
cheers

Great article.

Are there any similar studies out there that broke it down further and looked at the type of players picked in terms of position, league, etc?

Someone at Lowetide posted a list of defenseman drafted in the top 5 over the last few years. The number of busts was astounding. It really makes me question the wisdom of drafting Larsson.

Avatar
#30 OB1 Team Yakopov - F.S.T.N.F
March 11 2011, 08:24AM
Trash it!
0
trashes
Cheers
0
cheers

Excellent!

I'm glad this is getting more and more play.

I've basically come to the conclusion that I'd likely be trading every pick past 15-20. And for sure every pick past the 2nd round. (assuming the return is decent)

Avatar
#31 Next up, is Connor McJesus.
March 11 2011, 09:01AM
Trash it!
0
trashes
Cheers
0
cheers

Of the four kids, Couturier,Larsson,Landeskog and Hopkins, is it a stretch to say only two of these kids make the jump to the NHL this coming year?

Avatar
#32 Dave Lumley
March 11 2011, 09:13AM
Trash it!
0
trashes
Cheers
1
cheers

Has anyone compared Oiler draft track record to central scouting? I often thought the Oilers would have been better off firing the the scouting staff, using Central Scouting and then focusing on player development.

Avatar
#33 Westcoastoil
March 11 2011, 09:21AM
Trash it!
0
trashes
Cheers
0
cheers

I haven't seen any of them play, but based on what I've RNH makes me a bit nervous - is he another Sam Gagner? Great numbers, but how much is PP and will he have the size/speed to be a first line player. I don't know, but I'd be tempted to chicken out and take the best "sure thing" over most potential.

Based on Tambi's intermission interview a couple games, as tough as it is to swallow i don't think they'd mind one bit if they drafted a player that wasn't ready and rode it out for one more lottery pick.

Avatar
#34 fuck off
March 11 2011, 10:01AM
Trash it!
0
trashes
Cheers
1
cheers
John Chambers wrote:

Interesting note Willis that the top 3 picks are usually bona fide, picks 4-6 are less bona fide, and so on ... especially considering that there are 4 names often floated around as the top of the 2011 draft class.

So my question to you, fair nation, is which of the 4 top draft prospects do you feel is the WORST selection of the bunch.

I'm not sold on anyone in particular, but I for one am least sold on Sean Couturier.

What do you think?

Say it with me,

"Don't Take, R-N-H!!"

Avatar
#35 Next up, is Connor McJesus.
March 11 2011, 10:07AM
Trash it!
0
trashes
Cheers
0
cheers

@Westcoastoil

There must be a certain amount of pressure to step into the lineup right away if he's the first overall pick. Not sure how many kids have been selected first and then went back to junior, if any.

I remember the days when 6 or 8 kids drafted made that step their first year, now we're down to less than half that.

Avatar
#36 Westcoastoil
March 11 2011, 10:59AM
Trash it!
0
trashes
Cheers
0
cheers

@Next up, is Connor McJesus.

I hear what you're saying, but which of the 4 ready to make the jump. Just the 2 Swedes? If Couturier wasn't able to stand out in the WJC, how's he going to look lining up against an NHL vet. In Stu we trust:)

Avatar
#37 John Chambers
March 11 2011, 11:35AM
Trash it!
0
trashes
Cheers
0
cheers
fuck off wrote:

Say it with me,

"Don't Take, R-N-H!!"

Alright. Thanks for your thoughts. You too, Zissou, and Butters.

Too small a sample size, but I figured the Nation would be torn as to who should be the odd man out ... except for Larsson. I doubt many people have Larsson as their absolute least favorite lottery pick.

Avatar
#38 Robert V
March 11 2011, 11:58AM
Trash it!
0
trashes
Cheers
0
cheers

Great write-up Jonathan.

For more, there's a great write-up by Marc Foster: http://www.puckprospectus.com/article.php?type=2&articleid=849

and Iain Fyffe: http://www.puckprospectus.com/article.php?type=2&articleid=807

Unfortunately you need a subscription to read them. :(

However, I personally guarantee that the data and graphs within are worth the price of admission. They're both very careful and detailed studies of draft choices, similar to Awad's.

Avatar
#39 book¡e
March 11 2011, 11:59AM
Trash it!
0
trashes
Cheers
0
cheers

Re:Thinking that you (or your scouts) are better than others around you.

Check out this article by David Brooks in the NYT (it relates to biased self assessment)

http://www.nytimes.com/2011/03/11/opinion/11brooks.html?_r=1&hp

Avatar
#40 Monaertchi
March 11 2011, 03:50PM
Trash it!
0
trashes
Cheers
0
cheers

I've heard (don't remember where or from who) that most teams don't do any better, and some do worse, than Central Scouting.

So why pay for your own scouting department when Central Scouting is already doing as good a job for you?

Perhaps you've heard this same rumour. If so, where is it from and does it have any merit?

Avatar
#41 Todd
March 11 2011, 03:57PM
Trash it!
0
trashes
Cheers
0
cheers

Don't worry lamers you will taket he best hard working third liner in the draft. NICE move on trevor Kidd too. Who would some guy named Martin Brodeur, when you have your usual West boys.

Avatar
#45 Hands McDangles
March 11 2011, 07:28PM
Trash it!
0
trashes
Cheers
0
cheers

@Jonathan Willis

Willis, I have to say that is one of the most interesting articles I have ever read on ON. I know this must have taken a lot of time and research so thanks a lot. You earned your writing credit on this one.

Comments are closed for this article.