The Toronto Maple Leafs’ Great Experiment

“There hasn’t been a higher profile test case of hockey analytics condemnation of a team yet.” –

Training camp for the Toronto Maple Leafs starts this morning, and for the first time in his nearly two-year long run as head coach of the team, Randy Carlyle is putting his own training camp together with a roster that’s 58-strong. 59 once Cody Franson gets himself under contract.

Advertisement - Continue Reading Below

New general manager Dave Nonis had one modus operandi this offseason, and that was to get the players under contract that Carlyle wanted to play with. Out are speedy and skilled players Mikhail Grabovski, Matt Frattin and Clarke MacArthur. In are bruising tough guys David Bolland and David Clarkson. The Leafs made the playoffs last season with Grabovski and MacArthur in a reduced role—no matter how good they can be or were in the past for the team, it was proven that they weren’t important to the team winning.

Carlyle, and Nonis, took a gamble this summer, going all in on the players they believed were the most important on the Leafs in the team’s short playoff run. To Nonis’ credit, he was able to, for the most part, lock up all the players he wanted and I have a hard time believing the training camp roster released last night diverges wildly from what the Leafs wanted back in May. Where I disagree with Nonis isn’t the way he assembled his roster, but I disagree fundamentally with his philosophy, because him and Carlyle are going into the season rolling the dice on a style of play that may not have been as effective as it was on the outside last season.

As a team, the Toronto Maple Leafs shot 11.0% last season (subtract the number in the 5v5 Sh% number from 1000 and divide by 1000). No team since the start of the era has a larger single-season shooting percentage. The 2010 Washington Capitals tied them, but in the 2011 season, the Capitals dropped to 8.1%. Very few teams have been able to consistently repeat shooting percentage seasons outside one standard deviation from the mean, or approximately 9.2%. Of the 19 teams that shot over 9.2% in an 82-game season, just 4 repeated the feat in the next 82-game season.

Here’s the thing. I, or any other analyst, can crunch the numbers however I want and talk about PDO in these grand, hypothetical terms. I expected the Leafs’ PDO, the addition of shooting and save percentage (it doesn’t stand for anything, but just pretend it stands for ‘percentage-driven output’) to fall after February 20 last year and force the Leafs out of the playoffs, citing that five of the seven teams with percentages that high during the first 17 games of the season crashed, falling by an average of about seven points.

Advertisement - Continue Reading Below

The Leafs stunk as a data point. Their PDO fell from 1.039 to 1.029 according to behindthenet’s numbers, but the drop wasn’t enough to force them out of the playoffs. Despite their win percentage dropping, the Leafs made it to overtime enough to actually increase their points per 48 games played from February 17 to the end of the season. I was wrong.

There is a chance I could be wrong about all this. The general complaint about “shot quality” among the analytics crowd is that there’s no tangible proof a coach can affect his team’s on-ice numbers. Generally, the more skilled player, the higher his shooting percentage will be. Star players in the NHL can do two things: they can shoot for a high percentage on their lonesome, and they can generate a lot of shots to take. What the Leafs are doing is in the absence of star players that can generate a lot of shots, they are banking on the idea of “quality possessions” that result in a higher percentage of shots going in. This is meant to be in response to criticism that no playoff team has been as heavily out-shot as the Leafs since the 2002 Montreal Canadiens, and that there were different areas this offseason in which the team ought to have improved.

Tally up the career shooting percentages of the 19 skaters projected to make the Maple Leafs this season, from Dave Bolland to James van Riemsdyk. At even strength in their careers, they’ve scored 579 goals on 6365 shots, or 9.1%. That’s a little higher than NHL average, but had the Leafs’ shot percentage been 9.1% last year instead of 11%, they’d have lost about 14 goals at even strength last season, and depending on where those goals were scored and the game situations they were in, that probably would have been enough to boot them out of the playoffs.

I’m open to the idea that I’m wrong about this. I don’t think so, generally. Fenwick numbers have been shown to be better predictive of wins than wins themselves. Corsi Tied numbers are some of the best things we have for predicting the future of hockey teams, and while the Leafs’ roster underwent a significant turnaround in the offseason, the Leafs cast aside two players—Grabovski and MacArthur—that can positively influence the team’s shot counts and got just one back in Clarkson. While Bolland works better than Grabovski stylistically in Randy Carlyle’s dream system, Carlyle is still gambling that his system is any better than blowing it all up and playing a fast-paced offensive style. The team’s coup-de-grâce in the offseason was to go out and pay a massive premium for Jonathan Bernier, a goaltender with half the career starts of James Reimer, the goaltender that brought the Leafs to the playoffs last season. The theory here is that Reimer’s success in two of his three years as a Leaf has nothing to do with Reimer and instead thanks to the systems devised by Carlyle.

It’s all a big experiment, and there’s likely been no higher profile test subject in the application of hockey analytics. I’m as interested as ever in what the Leafs do or don’t accomplish this season and a good batch of people’s acceptance of new hockey analysis is probably going to depend on whether the Leafs compete for the Atlantic Division crown or not. It’s going to be divisive, entertaining, and every small winning or losing streak is likely to be accomplished by a thousand “i told you so”s in unison.

Basically, check your snake oil at the door. The Maple Leafs will be just one team out of 210 in the “Behind the Net era” at the conclusion of the 2013-2014 campaign, but until then they’re a test case. It’s going to be a hell of a season.

Advertisement - Continue Reading Below

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  • MaxPower417

    Actually there is some evidence that Reimers save percentage is inflated by RC system. The Leafs allow a crap tonne more low percentage shots and a couple less high percentage shots than average. This still results is more goals against but a higher save percentage. I don’t have my numbers on me, but I think this difference from the average could possibly account for between a .002 to .004 difference in Reimers save percentage which isn’t insignificant.

    • James Reimer, 2011 – 35 games started, .933 EV SV%
      James Reimer, 2013 – 31 games started, .924 EV SV%

      The only time Reimer has played below average in his career is that period of time where it turned out he was concussed. He was exceptional in Ron Wilson’s system, probably because he is an exceptional young goalitender.

      • Problem us if the Leafs are to be playoff bound they will have to win 35+ games… which means Reimer is reliant on another goalie being successful. Exceptional players who can’t stay healthy can’t be weighed on to save the team.

      • MaxPower417

        Actually, interestingly enough, the Leafs in 2011 with Reimer in net,pretty much the same percentage of 5v5 shots from 30+ feet out then under RC (58.58% vs. 59.46%) and league average during this years was also almost the exact same (53.3% vs. 53.84%. So Reimer has had this statistical advantage during his entire tenure with the Leafs.

        Don’t get me wrong, I’m a big Reimer fan, I don’t think he’s a below average goalie by any means and thought the Bernier trade (and subsequent contract)was…odd at best. B

        ut as much as I believe in Reimer, I also realize that a lot of that belief stems from fandom and he still needs a full season to really cement himself as a upper tier number 1. Although, with Bernier now in this mix, I’m worried he won’t be given that chance.

  • MaxPower417

    “The team’s coup-de-grâce in the offseason was to go out and pay a massive premium for Jonathan Bernier”

    I don’t think the Leafs paid a “massive premium” at all. This isn’t saying much, but Bernier was probably Nonis’ best aquisition of the summer. Scrivens was ok at times but if Reimer goes down, him playing 50+ games would scary the bejeezus out of me.

  • Cam – Shouldn’t you weight overall shooting percentage by the number of shots per individual rather than just taking a simple average?

    For example., if Kessel and JVR take significantly more shots than Kulemin and McClement, and they have higher career shooting percentages, then the expected team shooting % would be higher, no?

    • There’s no perfect way to do it because Kessel and JVR will take more shots than McClement, but we don’t know how much.

      They were weighted by totals in the last six years. Bolland has taken 372 shots and Kessel has taken 1114. That’s all I really had to go on as far as weighting goes.

  • I’m hopeful that the defense will be improved. Assuming Franson gets signed, the fact that he, Gardiner and Ranger will be playing a larger role as opposed to Koska, Holzer and Komisarek should mean an improvement in Fenwick. At least that’s my hope.

    • Plus there’s no way that they were as bad as last year’s shot total indicated. Would have to get much worse to be a 44% team again. I haven’t run the numbers yet, but they don’t correlate perfectly year-to-year and certainly regress towards 50%, even if there’s a lot of talent that goes into those numbers.

      If the team stays healthy, there should be a marked improvement in those numbers. David Clarkson can do a lot of things, and take shots is one of them. Defence, centre, and depth are the issue, but they’re excellent at wing.

  • Bob Canuck


    Terrific piece.

    With reference to your February 25 post, have you looked at the correlation between blocked shots, expressed as a percentage of shot attempts against, and wins? Intuitively, the lack of correlation between total shots blocked and wins makes sense. My hunch is that there will be a higher, but not significant, correlation between shot blocking rates and wins than there is between total shots blocked and wins.

    Keep up the good work. I always look forward to reading your posts.


      • Yeah I don’t mean you in partcular… I just think that alot of people are getting excited because they’ve found a way to keep saying ‘Leafs suck’ when they had a good season. If it was any other team no one would really care.
        But you are right let the games be played and we will see whats what!

  • Badger M

    Cool piece Cam.

    Player to watch this season is Gardiner IMO. I loved what I saw in the playoffs, and I believe he can definitely contribute to improved puck possession numbers if given enough minutes.

  • Badger M

    I’m drawn to the advance statistics that have become so important in rating NHL players and teams but, as a long-time basketball coach, I keep asking the age-old question, ” This guy has great numbers but can he play?”
    It just seems to me that there are qualities that some players have that aren’t necessarily found statistically. At the top of this list, I would put the ability to play well when the game is on the line. I’d really like to see a way to measure that but I guess the best way might be to track the players that the coach puts out there when it is do -or – die time.
    One final thought on the upcoming Leafs season – if both goalies stay healthy and play well, the Leafs will be just fine, advanced statistics or not.
    PS There are three kinds of lies – lies, damn lies and statistics.

    • Badger M

      I think the general thinking in pro sports analytics is that “clutch” is descriptive, not predictive. When you trim the game down to just the last couple of minutes when the score is close or OT, there is going to be a lot of noise in the data, and in sports where there’s more data (i.e. the other three), it doesn’t appear to exist as a player skill–just randomness.

      As for player qualities that don’t show up on the scoresheet, in every sport, the quality has to show up on the scoresheet somehow–otherwise, it’s not useful. If it doesn’t lead to extra goals, or extra points (for yourself or for teammates), why does it matter?