Zone Start Adjusting PK and PP Performance

Given the importance of 5v5 shot metrics in predicting future outcomes, it’s easy to forget about the importance of special teams when it comes to driving results. Over the last decade, approximately 20% of the game has been spent on special teams, accounting for almost 25% of the total goals scored. Despite contributing to nearly a quarter of goal differentials, this area of the game tends to receive very little attention from hockey nerds like myself.

This isn’t to say that it’s been completely neglected. There’s been some excellent research proving the value of shot metrics in predicting future goal scoring on the power play, which is similar to the trend we’ve observed with 5v5 play. The consensus within the analytics community seems to be that shot suppression (CA/60) is the best predictive measure of future goal prevention on the PK. Predicting future goal scoring on the power play is more of a mixed bag. Shot generation metrics (CF/60 and FF/60) tend to have the most predictive value within-season, while goal metrics (GF/60) actually have more predictive power from season to season at the team level.

One area we haven’t dipped our feet into yet is zone start adjusting special teams metrics. While there’s been some awesome work by Prashanth Iyer and Tyler Dellow highlighting the importance of 4v5 zone starts, it hasn’t made its way into public databases yet. We have great 5v5 ZS adjustments on websites like Corsica and Puckalytics based on some of the terrific research in the field, but unfortunately similar adjustments aren’t available for the penalty kill or power play. This got me thinking: why don’t we have ZS adjustments for 4v5 or 5v4 play? After hours of digging, I wasn’t able to find a good answer to that question. The frustration that ensued has motivated me to start this project. My goal here is to make an introductory attempt at creating zone start adjustments for 4v5 & 5v4 shot metrics.

Before we get into the numbers side of things, let’s think about the dynamics of a power play from the perspective of the defending team. As a penalty killer, the initial faceoff is in your defensive zone. Statistically, there is a stronger chance of the team with the man advantage winning the faceoff (~54%), so you’re already at a disadvantage before play’s even started. Even if you win the 4v5 faceoff, there’s still a decent chance that the opposition will hound you on the forecheck and regain possession before you’re able to clear the zone. For example, Washington’s power play accomplished this on 18% of lost faceoffs last year.

To make a long story short, if you’re starting your 4v5 shift in the defensive zone, you’re much more likely to face an opposing power play that’s set up in formation. This is extremely important to remember because once in formation, a power play unit is all but guaranteed to generate a shot attempt. It’s pretty hard to suppress shots when most of your time killing penalties looks like this:

To contrast this, let’s think about starting your 4v5 shift ‘on the fly’ (OTF). If your PK unit gets the opportunity to change on the fly, it probably means that they’ve iced the puck, or at the very least cleared the zone. This allows the players hopping over the boards to set up in a neutral zone trap and defend opposing zone entries at the blue line. So if you’re starting your 4v5 shift on the fly, it typically looks like this:

The contrast between these two images paints a pretty good picture of why context matters when we’re looking at players’ 4v5 shot suppression. Players given a 4v5 DZ start are much more likely to face a power play that’s set up in formation than penalty killers with an OTF start. Based on Arik Parnass’ work, we know that elite players will successfully gain the zone and establish formation (or a rush shot) about 40% of the time on the power play, with most players falling in the low 30% range. So to quickly summarize: penalty killers with a DZ start are facing an in-formation power play about half of the time, while those with an OTF start are only doing so about one-third of the time. That’s a pretty sizeable difference.

When it comes to the impact of zone starts on the power play, we can simply reverse this logic. Players given more OZ starts are put in a position to succeed, while those given OTF starts face more of an uphill battle. Arik Parnass’ research has proven that a team’s ability to get in formation is currently the most effective way to predict future success on the power play, so it’s important to remember that players on the 1st unit PP have a much better opportunity to accomplish this with an OZ start than 2nd units do with an OTF start. This is going to result in 1st unit PPs generating more shots and chances, independent of ability.

After breaking it down logically, we know that zone starts clearly matter on special teams. The question now becomes: how much do they matter?

This is where things get a bit nerdy. I’ve used Corsica’s data from 2007-2016, looking at players with a minimum of 500 minutes at either 4v5 or 5v5. This gives us a sample of 190 Forwards & 191 Defensemen on the penalty kill, as well as 297 Forwards & 128 Defensemen on the power play. If you’re wondering why those PP numbers look a bit weird, keep in mind that 4-forward PP units are becoming more common these days.

Using Corsica’s awesome data, I was able to quantify how many shift starts per 60 players had in each zone (and on the fly) at both 4v5 & 5v4. I used these numbers to determine the r² between the players’ zone starts and shot metrics. This r² number refers to how much of the variance in a player’s results that can be explained by their zone start usage. For example, 49% of forwards’ 4v5 shot suppression (CA/60) can be explained by the number of shifts they start On The Fly per 60 minutes (OTF/60). Here’s a list of the r² numbers:

The take-home point from these numbers is similar to our qualitative analysis earlier: zone starts really matter on special teams. The data indicates that 4v5 DZ & OTF starts impact players’ shot metrics, but don’t have as significant of an impact on goal metrics. This is similar to the trend we’ve seen in 5v5 play, since on-ice save percentages fluctuate like crazy.

When we look at the power play, OZ & OTF starts have a very strong impact on both shot metrics and goal metrics. It makes sense that these would impact shot generation, like we discussed earlier. When it comes to the strong correlation with goals though, my theory is that coaches are successfully identifying their players with the best shooting talent (impact on Sh%) and giving them more 5v4 OZ starts by placing them on the 1st unit PP. This isn’t to say that there aren’t other factors in play; it’s simply my best guess at explaining the relationship.

One tricky area for me when looking at the data was 4v5 OZ starts & 5v4 DZ starts. On the one hand, they seem to explain a decent amount of the variance in players’ results. On the other hand, they account for such a tiny proportion of players’ shifts on special teams, so how much can we really trust them? To show you what I’m talking about, here’s a quick breakdown of how often players start their shifts in each zone:

As you can see, 4v5 OZ starts only account for 3% of your PK shifts, while DZ starts & OTF starts account for over 85% of the shifts. These numbers are similar when it comes to 5v4 play, with OZ starts taking the place of DZ starts (since that’s where most faceoffs will occur on the PP). Since they account for such a small amount of players’ ice time (only 3% or 4%), I’m going to be excluding these particular zone starts in my analysis. Although the correlations indicate that they may have some value, personally I think the sample size is an issue. With special teams analysis, we’re already dealing with very small samples, so I’m skeptical of putting significant stock in a 3% slice of that small sample. NZ starts will also be excluded since they explain very little of the variance in players’ special teams results. This leaves us with DZ & OTF starts for 4v5 play, as well as OZ & OTF starts for 5v4 play.

Although I made ZS adjustments for every metric (Corsi, Fenwick, Shots, Expected Goals, and Goals), to make this explanation easier we’ll use the example of shot suppression (CA/60) for Forwards on the PK. After doing some regression analysis, I was able to determine players’ “expected” CA/60 based on their zone starts. I then took the league average 4v5 CA/60 (which is 86.13 over the past 9 seasons) and subtracted it by the player’s expected CA/60. This tells us how much more difficult or easy that player’s zone starts were relative to league average. Finally, I added that amount to the player’s Observed CA/60 to get the final ZS adjusted CA/60.

To help explain how this works, let’s look at a couple players on extreme ends of the spectrum. Boyd Gordon would be expected to allow 10.4 more shot attempts per 60 based on his heavy DZ usage, while Blake Wheeler would be expected to allow 13.2 fewer shot attempts per 60 because of his heavy OTF usage. I simply took these numbers (-10.4 in Gordon’s case, +13.2 for Wheeler) and added them to their observed CA/60 (the number you’d see on a site like Corsica or Puckalytics) to determine their final ZS adjusted CA/60. So we subtract 10.4 from Gordon’s observed CA/60 of 101.5 to reach his ZS adjusted CA/60 of 91.1, while we add 13.2 to Wheeler’s observed CA/60 of 74.7 to reach his ZS adjusted CA/60 of 87.9. Looking up these players’ shot metrics on Corsica, we’d see 101.5 vs 74.7 and probably conclude that Gordon’s much worse than Wheeler on the PK. After adjusting for zone starts though, we can see that they’re much closer in 4v5 shot suppression when you account for their extreme ZS usage.

This process was repeated for all 4v5 metrics (CA/60, FA/60, SA/60, xGA/60, GA/60) and 5v4 metrics (CF/60, FF/60, SF/60, xGF/60, GF/60).

Here’s the link to the final ZS adjustments. For transparency, I’m also providing the formulas I used in my analysis, which can be found here if you’re interested. For 4v5 play, I’d recommend sorting players by ZS adjusted CA/60 and xGA/60, since they’re the best predictors of future goals against on the penalty kill. When looking at 5v4 play, I personally prefer xGF/60, but I’ve heard several good arguments for using CF/60 or FF/60.

Thanks to the wonderful Sean Tierney (who you should definitely follow on twitter @ChartingHockey), I also have some lovely charts to show you. We agreed that the best way to visualize the data was by comparing players’ “Expected” Corsi (based on their ZS usage) to their “Observed” Corsi (the number you would find on Corsica or Puckalytics). What we end up with is a great way of visualizing which players are outperforming their usage and which ones probably shouldn’t be on the ice. If a player is closer to the top right, it means that they’re playing very well. The closer they move to the bottom left…not so good. I recommend checking out the interactive charts here, where you can run your mouse over the scatter plot to see which player is which, but here’s a quick preview:

Again, if you click on this link it will take you to the interactive tableaus. Dashboard 1 is PK Defenders, Dashboard 2 is PK Forwards, Dashboard 3 is PP Defenders, and Dashboard 4 is PP Forwards. If you would like to look at players on a specific team, just click on the logo of the team want to see.

I’ve found that the best zone start analysis looks at individual shifts using Play-By-Play data. Unfortunately, my coding skills aren’t quite at Chris Bosh levels, so I had to rely on aggregated data for this project (Corsica + Excel = the extent of my abilities). In the future, I think digging into 4v5 & 5v4 Play-By-Play data will provide a more detailed look into the impact of zone starts on special teams.

Another important thing to consider is that there are plenty of contextual factors that impact players’ results on special teams besides zone starts, most notably a player’s Quality of Teammates (QoT) and Quality of Competition (QoC). For example, it’s a lot easier killing penalties alongside Patrice Bergeron than it is with Tanner Glass. Similarly, playing against Ovechkin’s unit is much different than those tough shifts against PP wizard John Mitchell. It would be awesome if we could weight these QoT & QoC factors the way that DTM About Heart has with his 5v5 XPM metric.

Even if we developed a 4v5 & 5v4 XPM metric though, it’s important to remember how much of an impact coaching and systems can have on special teams results. Just take a look at how drastically the Leafs’ 4v5 & 5v4 shot metrics changed in their first season under Mike Babcock:

They put up those excellent shot metrics despite icing power play units with the likes of Brad Boyes, Peter Holland, Shawn Matthias, even Joffrey Lupul’s corpse – I think it’s fair to say that coaching can have a huge impact on special teams.

At the end of the day though, I think these ZS adjustments provide us with a better evaluation of shot metrics on special teams. They’re definitely not perfect, but they’re much better than relying on the unadjusted data that we currently have. Personally, I think the biggest advantage of using these adjustments is to better identify PK performance, since most of us can already identify effective PP performers pretty easily (sorting by 5v4 Points per 60 over the last 4 years gives you a pretty good look at the league’s best players on the power play). I’m looking forward to how the analytics community can build on this work. It’s only a matter of time before we have ZS adjustments for special teams in public databases like Corsica and Puckalytics. I’m glad I could do my part to help accelerate that process and get the conversation started.