Morgan Rielly, Jake Gardiner, and the Importance of Quality of Competition

The topic of how significantly Morgan Rielly is affected by quality of competition has been making the rounds recently.  Rielly’s possession statistics this year have not been especially strong, as he’s currently just barely breaking even in terms of shot attempt ratio, 4th best on the team after Jake Gardiner, Connor Carrick, and Martin Marincin.  While Morgan continues to generate a lot of offence, his defensive results look poor – the only Leafs defenceman who allows a worse rate of shot attempts relative to the rest of the team is Roman Polak.

The question of how Rielly is being affected by the difficulty of the opponents he faces was recently addressed in this piece by Tyler Dellow, who he argued that Rielly is playing well compared to other defencemen given similarly tough minutes.  “In terms of possession,” Dellow says,  “Rielly looks fairly good when contrasted with guys who are fed into the wood chipper like he is.”  The contrary position was taken by David Johnson, who wrote on his site that, “In no way does quality of opponent change the debate in any significant way. Yeah, extreme players will see a shift in their statistics but if you have terrible statistics to start with they at best will be really bad once you make any QoC adjustment.”  Johnson’s argument is that the gaps in competition between players are actually fairly small, so any adjustments we’d need to make to account for those differences should be small.

That argument lines up fairly well with most statistically-oriented bloggers.  For example, back in 2012, Eric Tulsky argued that, “competition metrics provide valuable insight into what a coach thinks of a player and how he tries to use them, but in practice they do not show differences large enough to have significant impact on the player’s results.”  Tulsky’s work showed that most players spend roughly the same frequency of playing difficult or easy competition.

While Rielly’s shot attempt statistics don’t look great, Jake Gardiner’s are consistently excellent.  Gardiner currently leads the Leafs in preventing shot attempts against, with his -3.42 Corsi against per 60 minutes ranking 23rd best in the entire NHL.  If these statistics are to be taken at face value, Gardiner is an elite defenceman, one of the best defensively in the entire league, while Rielly’s defensive play is quite poor.

But should we take those results at face value?  Tulsky’s work suggests that quality of competition shouldn’t make a big difference to Corsi (at least as quality of competition is currently measured), and Johnson agrees.  While I understand what Tulsky’s work shows in terms of the spread of competition, it seems intuitively wrong to me.  On the other hand, Gardiner and Rielly are separated by less than a minute in ice time per game this season, with Gardiner playing a fairly heft 21:26 a night.  Surely Gardiner can’t actually be that sheltered playing so many minutes, right?

To look at Rielly’s competition, Dellow and Johnson tried to pick out “star” players on each team and see how often Rielly played each of those players.  I decided to take a similar-ish approach, but I wanted to be a bit more objective about it.  I put together lists of the top centres on each of the 29 other teams by two metrics: relative Corsi and goals scored per 60 minutes.  Both of these stats are only from 5v5 play.  My thinking is that on most lines the centre is the best player, so this is a reasonable proxy for who is the most difficult to play.

I then took a look at how much ice time Jake Gardiner and Morgan Rielly played against each of those opposing centres over the past two seasons (ie. since Mike Babcock took over as head coach).  That data was pulled from David Johnson’s site.  He only lists ice time for players that another player has faced for at least 10 minutes each season, so for any player who played under 10 minutes against Gardiner or Rielly, I’ve simply listed the result as “<10”.  Conveniently for my purposes here, Gardiner and Rielly have played a virtually identical amount of ice time at 5v5 over the past two seasons, so there’s no need to adjust these numbers to make a fair comparison.  Let’s start by looking at how frequently Gardiner and Rielly played against the top centres in the league by relative Corsi over the past two seasons:

One thing you’ll likely notice is that it doesn’t look like this is really capturing the hardest centre to play against for some teams.  I think Andrew Cogliano’s a good player, but I think we would all agree that he’s an easier match-up than Ryan Getzlaf.  Is Radek Faksa really harder to play against than Tyler Seguin?  I doubt it.  But I think we’ve at least got a rough approximation of some of the best centres in the league.

Nevertheless, let’s take a look at the top centre on each team based on goal scoring rate so we can do a bit of a comparison:

I think this second list likely comes closer to what most people would think of as the best centres in the league, even though there are still some notable omissions (like Kuznetsov over Backstrom).  At any rate, between the two lists we’ve capture most of the top centres in the league.

One thing that’s easy to see when looking at these numbers is that there are almost none against whom Gardiner played significantly more minutes than Rielly.  There are a few, but they’re mostly the 2nd or 3rd best centre on their team, which is what we’d expect if Rielly was being hard-matched against top competition.

And it’s pretty easy to see that there are a number of cases in which Rielly is clearly being hard-matched against top competition while Gardiner is kept away.  The players who Rielly played a lot of minutes against while Gardiner barely played them (<10 minutes) are Nathan Mackinnon, Connor McDavid, Sidney Crosby, and Joe Thornton, four of the best players in the NHL.  We also see pretty huge splits against other difficult competition, like Steven Stamkos, Ryan Getzlaf, Patrice Bergeron, and John Tavares.  I think it’s pretty safe to say that, in terms of the top centres in the league, Rielly is facing significantly more difficult competition than Jake Gardiner, and that intuitively it seems like such a big gap in competition should affect the numbers each player puts up.

And yet if you look at their Corsi quality of competition, the gap looks small.  Gardiner’s Corsi QoC is 49.9% and Rielly’s is 50.6%.  It’s not nothing, but it’s certainly much smaller than the gap in Corsi Rel between the two players, which is about 3%.  So if it looks in the above charts like Rielly is being given way tougher minutes, why isn’t it showing up in the tables above?

One reason is that QoC is based on all five opponents on the ice, so even if you play against a top centre, the overall QoC can be brought down if some of the other players on the ice aren’t as good.  It’s also true that the ice time listed here is only a fraction of the total played by each player.  If you add up all the players above who Rielly played against far more than Gardiner did, you’re only accounting for about 20% of his total time on ice.  Maybe Rielly plays tougher minutes in some situations, but the overall spread remains small because such a huge portion of each player’s ice time is played against everyone else.  And yet Rielly clearly does play tougher competition than Gardiner in a reasonably large number of situations.  20% is still a sizeable portion of your his ice time.

I also have a theory about why the spread in QoC seems smaller than it intuitively ought to.  I don’t think there’s any way to prove it, but it sounds to me as though it ought to be correct.  My theory is this:

The spread of QoC around the league looks smaller than it really is because good players play so much of their ice time against other good players, while coaches try to match their worse players against lesser competition.  Now I know what the counter-argument to that is: “We’ve looked at the spread of QoC and everyone plays roughly the same level of competition.”

But I don’t think that’s quite true.  Here’s a simplified way to think of it:

Let’s say two players have a “true” Corsi ratio of 55%.  If those players play head-to-head, the expected outcome for both of them is 50%.  Now imagine two players whose true CF% is 45%.  If those players face each other, their expected outcome is also 50%, even though their actual skill level is far lower than the first pair of players.  If those kinds of situations happen often enough over the course of a hockey season, the differences will slowly add up.  Yes, every player does play some spread of competition, so in the long run you’re still able to separate out the good players from the bad ones, but the gaps will be smaller than the actual spread of skill because of that compressing effect.

To put it in more specific terms, think of it this way: This season there’s about a 3% gap in Corsi between Gardiner and Rielly.  So if you spend a lot of minutes playing against Rielly, your QoC will be lower than if you played against Gardiner.  Now, if we believe that QoC doesn’t have much impact on players, that’s fine.  But if Rielly really is being pulled down by the level of competition he faces, then the QoC of everyone who plays against Rielly is just a little bit under-estimated.  And if that kind of effect is also happening for Rielly’s opponents, then we can start to see how, when you start to add up those small impacts against many opponents, there could be a systematic compression of QoC relative to the real skill gaps across the league.

I’ll leave you with this: Previous research has shown that the spread of QoC across the league is small.  And to some degree that’s probably true of Rielly and Gardiner.  They do play a fairly even level of competition across a sizeable chunk of their ice time.  But it’s also clearly true that Rielly plays a large number of minutes against especially difficult competition that Gardiner is not subjected to.  And I find it very difficult to believe that that gap, which could be something like 20-25% of Rielly’s minutes, doesn’t have any meaningful impact on the shot results for those two players.  I don’t think it explains the entire gap (other factors like quality of teammates and differences in skill certainly also count), but I think it’s fair to say that the gap in ability between Gardiner and Rielly is smaller than you would think just by looking at their possession metrics.