The Value of Training Camp (and small samples)

9 years ago

The Edmonton Oilers’ training camp this past fall was really something to see. Several slots were open and there were multiple candidates competing for each job, and as a result we saw some really tight races and some brilliant performances from the contenders.

In hindsight, how much stock should the Oilers have put in that training camp assessment period?

Framing The Issue

If we look at the opening night roster, there were a bunch of players who were locks for roster spots before anyone entered camp. My list looks something like this:

Forwards (9): Taylor Hall, Nail Yakupov, Jordan Eberle, Teddy Purcell, Matt Hendricks, Boyd Gordon, David Perron, Benoit Pouliot, Ryan Nugent-Hopkins
Defence (5): Jeff Petry, Mark Fayne, Justin Schultz, Andrew Ference, Nikita Nikitin
Goalies (2): Ben Scrivens, Viktor Fasth

There were 2-to-3 jobs open on defence, and 4-to-5 jobs open up front; seven slots available in all. How did things play out?

Winners and Losers

The centre position was the obvious hole coming into training camp. Edmonton opted to keep three pivots – two for jobs in the top-twelve, one as a healthy scratch with the thought that if injury hit the team would have a little more versatility up front with a spare centre than with a spare winger. Leon Draisaitl and Mark Arcobello won the first two jobs and Will Acton the third, while the trio of Anton Lander, Bogdan Yakimov and Jujhar Khaira were all demoted. Here’s how the Oilers’ decision-makers put it at the time:

Craig MacTavish on Draisaitl: “The thing that I liked about Leon, one of the things I liked about Leon, is even when he wasn’t productive offensively he wasn’t particularly unproductive defensively. He wasn’t hurting us with his decision-making defensively, he wasn’t cheating.”
Dallas Eakins on whether Acton playing centre won him the spare forward job: “We deemed right now that we need somebody that’s versatile. Will Acton, he can play centre, he can play the wing, he’s arguably our best penalty killer. Right now he would be on the outside of our lineup barring any injury. The other two [Tyler Pitlick and Steve Pinizzotto], they’re wingers, and if we needed to make a quick change in the middle then we were in trouble.”
MacTavish on Lander: “I think Anton had a much better camp than he had last year, but I think there were guys that outplayed him. We’re not closing the book on Anton, but he’s got to back and wait for his next opportunity.”
MacTavish on Yakimov and Khaira: “Bogdan was close; we entertained thoughts of keeping him. Jujhar was very close, we entertained thoughts of keeping him. We need these guys to develop and get here quickly. Because of the performance level of those two, we’re less likely to add somebody that would be filler in the centre ice position.”

It’s interesting to look at those comments with the benefit of hindsight. MacTavish’s comment on Draisaitl could well be extended to his entire 37-game stint in Edmonton; even when he didn’t produce (which was most of the time) he wasn’t particularly bad defensively. It’s not clear that the Oilers made the wrong decisions on Acton/Lander, as Lander did manage to clear waivers and his additional stint in the minors doesn’t seem to have hurt him.

The really interesting comments are on Yakimov and Khaira. The performance of those two players in training camp evidently convinced the Oilers that they had depth at centre, with Yakimov even getting an early season call-up. That assessment was a mistake; Yakimov ranks ninth among Barons forwards in scoring this year and he’s scored more than twice as much as Khaira. In hindsight, it is abundantly clear that both players needed AHL seasoning.

With the decision to keep a spare centre and eight defencemen (more on that shortly) there was only one job open on the wings. Jesse Joensuu, Steve Pinizzotto and Tyler Pitlick were all excellent in camp, but Joensuu won the job.

MacTavish on Pinizzotto and Joensuu: “I think there are going to be times that we want the toughness. Steve Pinizzotto would have filled that bill for us but the decision was based on the way that Jesse Joensuu played; he was maybe our most effective forward.”
MacTavish on Pitlick and Pinizzotto: “I think that we can debate the effectiveness of Tyler Pitlick. I think Tyler is at a position right now that we wished he would have been at a year or a year and a half ago. He’s turned himself into a professional. It ended up with Tyler as well as with Pinizzotto was two difficult decisions, but at the end of the day for me it got down to a positional bias, and I thought the 13th forward here would be best served by being a centreman. [Waivers] was a risk I was willing to take.”

All three wingers eventually got a shot. Joensuu was effective as part of a very good Hendricks/Gordon line but eventually fell out of favour for reasons that were never really explained. Pinizzotto was effective in the same role for the most part but his penchant for bad penalties probably cost him an NHL job. Pitlick was effective in the same role but got hurt. Training camp performance didn’t lie here; all three guys ended up being competent end-of-roster options on a tough zonestarts line.

The defensive battle was one of the most controversial, and saw two players with significant advantages lose their spots. Brad Hunt won a job out of camp because he’s the only defenceman in the organization who can shoot the puck with speed and accuracy from the point on the power play. Darnell Nurse earned a cup of coffee, in large part because the team couldn’t bring him back if they made the decision to cut him. Finally, Oscar Klefbom unseated a guy with a one-way contract (Keith Aulie) and a player who had been very good in a top-four role the year before (Martin Marincin).

MacTavish on Hunt: “Brad Hunt is a guy that we all like for his ability to provide offence… Hunt definitely does some things and can help us in an area of need.”
MacTavish on Nurse: “This isn’t permanent. This is to start the year. There are always going to be injuries, there is always going to be performance issues. It’s not that we can’t unwind; the only decision we really couldn’t unwind would be Darnell, sending him back. We couldn’t unwind that decision.”
MacTavish on Klefbom and Marincin: “I thought Oscar gives us a bit of a different element. I think he had a better camp than Marty. Marty started slowly in camp… I thought Marty played better the last little bit of training camp, as he had to.”
Eakins on Marincin: “An individual does not have a Corsi; his Corsi is a whole bunch of players that are with him. I thought Marty played very well last year. I thought he came in in a difficult circumstance and matched up well with Petry. We thought about that, but the makeup of our D, how we want it right now, we’re looking in that spot of more battle, more heaviness, and Marty was one of the guys that came out of the gate slowly.”
MacTavish on Aulie: “Keith was a guy that started off slowly at training camp and continued to get better, as you might expect from the amount that he played last year… My assessment of Keith was that he wasn’t going to be in our lineup on opening night, so let’s keep him playing, keep him in the rhythm. He did, as I said, play that much last year. You run the risk of losing him off waivers, but it was an acceptable risk, obviously, at least I thought it was, but keep him playing and see what happens down the road.”

With Hunt, the primary question was how to weight his strengths (offence!) vs his weaknesses (playing five-on-five hockey in the NHL). The Oilers decided they needed the scoring; in hindsight it’s clear that while Hunt can be a power play point man he’s not good enough in other areas. If the Oilers had dressed him as a seventh defenceman it might have worked, though that would have put the forward group shorthanded.

Nurse and Aulie are both defensible decisions in hindsight; the top prospect got his cameo and the guy who cleared waivers would manage the feat again later in the year. Klefbom ended up getting a brief minor-league cameo which didn’t hurt him any; he’s NHL-ready now.

Marincin was an odd decision at the time and an odd decision in hindsight, particularly given Eakins’ line about how great the Marincin/Petry pairing was. Just three games into the season, Petry was healthy-scratched in favour of Nurse and Hunt while Marincin was down in the minors (the Oilers lost that game 6-1); in a single stroke the team’s best shutdown tandem from 2013-14 was out of the lineup in favour of two guys who both ended up on the outside this season.

Preponderance of Evidence

When I look back at training camp, I’m reminded what a mixed bag it is. Sometimes, a standout performance means something, as it did in the cases of the three wingers. Sometimes it means almost nothing and can even lead a team astray, as it did in the cases of Yakimov and Khaira. And sometimes a team can talk itself into a specialist rather than grabbing the best player, as in the cases of Hunt/Marincin and Acton/Lander.

We talked about how to weigh training camp performance back in September, and I think the Bayesian process stands up well in hindsight. From that piece:

It’s all about weighting our information correctly. The first thing is to have a firm prior – in other words, not to forget the miles and miles of road that led us to where we are now. Every one of these players has a track record coming in, and when deciding whether or not they’re ready it’s extremely important to keep that track record in mind. That’s why NHL training camps aren’t really about the guys at the top of the roster or the prospects at the bottom – they’re about all the guys in the middle, the Joensuus and Pitlicks and Pinizottos and Pakarinens, the players close enough together coming in that the training camp results can push one in front of another.

Training camp does matter; it just matters a lot less than the seasons of data in the rearview mirror. If a player who shouldn’t be close to NHL-ready – say a first-year pro like Yakimov or Khaira – has a stellar camp, that matters a lot less than his performance in past years. Training camp can’t be treated as a blank slate where all players come in at zero and earn spots on merit; it’s instead a weighted competition between a handful of players who are close enough that a great or a poor performance can tip the scales.

Why are we talking about this now? We’re at the point of the year where AHL’ers get major-league cameos and the team’s summer plans get drawn up. The same rules that apply to training camp apply here. Performance over a short span of time needs to be handled carefully and decisions made based on the preponderance of evidence rather than a dozen good or bad contests.

RECENTLY BY JONATHAN WILLIS

The Value of Training Camp (and small samples)

Framing The Issue

Winners and Losers

Preponderance of Evidence

RECENTLY BY JONATHAN WILLIS

Check out these posts...