Photo Credit: Dan Hamilton / USA TODAY Sports

Morgan Rielly, Jake Gardiner, and the Importance of Quality of Competition

The topic of how significantly Morgan Rielly is affected by quality of competition has been making the rounds recently.  Rielly’s possession statistics this year have not been especially strong, as he’s currently just barely breaking even in terms of shot attempt ratio, 4th best on the team after Jake Gardiner, Connor Carrick, and Martin Marincin.  While Morgan continues to generate a lot of offence, his defensive results look poor – the only Leafs defenceman who allows a worse rate of shot attempts relative to the rest of the team is Roman Polak.

The question of how Rielly is being affected by the difficulty of the opponents he faces was recently addressed in this piece by Tyler Dellow, who he argued that Rielly is playing well compared to other defencemen given similarly tough minutes.  “In terms of possession,” Dellow says,  “Rielly looks fairly good when contrasted with guys who are fed into the wood chipper like he is.”  The contrary position was taken by David Johnson, who wrote on his site that, “In no way does quality of opponent change the debate in any significant way. Yeah, extreme players will see a shift in their statistics but if you have terrible statistics to start with they at best will be really bad once you make any QoC adjustment.”  Johnson’s argument is that the gaps in competition between players are actually fairly small, so any adjustments we’d need to make to account for those differences should be small.

That argument lines up fairly well with most statistically-oriented bloggers.  For example, back in 2012, Eric Tulsky argued that, “competition metrics provide valuable insight into what a coach thinks of a player and how he tries to use them, but in practice they do not show differences large enough to have significant impact on the player’s results.”  Tulsky’s work showed that most players spend roughly the same frequency of playing difficult or easy competition.

While Rielly’s shot attempt statistics don’t look great, Jake Gardiner’s are consistently excellent.  Gardiner currently leads the Leafs in preventing shot attempts against, with his -3.42 Corsi against per 60 minutes ranking 23rd best in the entire NHL.  If these statistics are to be taken at face value, Gardiner is an elite defenceman, one of the best defensively in the entire league, while Rielly’s defensive play is quite poor.

But should we take those results at face value?  Tulsky’s work suggests that quality of competition shouldn’t make a big difference to Corsi (at least as quality of competition is currently measured), and Johnson agrees.  While I understand what Tulsky’s work shows in terms of the spread of competition, it seems intuitively wrong to me.  On the other hand, Gardiner and Rielly are separated by less than a minute in ice time per game this season, with Gardiner playing a fairly heft 21:26 a night.  Surely Gardiner can’t actually be that sheltered playing so many minutes, right?

To look at Rielly’s competition, Dellow and Johnson tried to pick out “star” players on each team and see how often Rielly played each of those players.  I decided to take a similar-ish approach, but I wanted to be a bit more objective about it.  I put together lists of the top centres on each of the 29 other teams by two metrics: relative Corsi and goals scored per 60 minutes.  Both of these stats are only from 5v5 play.  My thinking is that on most lines the centre is the best player, so this is a reasonable proxy for who is the most difficult to play.

I then took a look at how much ice time Jake Gardiner and Morgan Rielly played against each of those opposing centres over the past two seasons (ie. since Mike Babcock took over as head coach).  That data was pulled from David Johnson’s site.  He only lists ice time for players that another player has faced for at least 10 minutes each season, so for any player who played under 10 minutes against Gardiner or Rielly, I’ve simply listed the result as “<10”.  Conveniently for my purposes here, Gardiner and Rielly have played a virtually identical amount of ice time at 5v5 over the past two seasons, so there’s no need to adjust these numbers to make a fair comparison.  Let’s start by looking at how frequently Gardiner and Rielly played against the top centres in the league by relative Corsi over the past two seasons:

Player Team CF% Rel Rielly TOI Gardiner TOI
Cogliano ANA 3.5% 14:29 18:07
Rieder ARI 2.3% 26:21 12:48
Bergeron BOS 8.3% 72:38 26:54
Reinhart BUF 3.6% 35:25 39:44
J.Staal CAR 4.4% 45:23 20:08
Backlund CGY 5.6% 16:02 17:58
Toews CHI 1.3% 22:26 22:41
Gagner CBJ 3.1% 18:41 20:17
MacKinnon COL 3.7% 40:39 <10
Faksa DAL 2.2% <10:00 27:46
Nielsen DET 2.2% 12:24 37:54
McDavid EDM 4.4% 32:26 <10
Trocheck FLA 5.4% 25:52 34:30
Shore LAK 2.9% <10:00 11:54
E. Staal MIN 4.3% 40:28 20:32
Galchenyuk MTL 1.0% 30:34 31:28
Johansen NSH 4.4% 21:44 10:59
Josefson NJD 2.4% 11:32 11:01
Tavares NYI 5.4% 74:42 16:06
Stepan NYR 3.0% 21:05 38:32
Smith OTT 4.1% 30:14 40:54
Couturier PHI 4.1% 14:03 31:40
Crosby PIT 3.6% 52:56 <10
Hertl SJS 5.9% 12:59 <10
Lehtera STL 1.2% 21:34 22:29
Namestnikov TBL 3.3% 34:14 19:41
H. Sedin VAN 2.6% 40:46 <10
Perreault WPG 6.1% 19:30 23:14
Eller WSH 3.0% 23:33 20:50

One thing you’ll likely notice is that it doesn’t look like this is really capturing the hardest centre to play against for some teams.  I think Andrew Cogliano’s a good player, but I think we would all agree that he’s an easier match-up than Ryan Getzlaf.  Is Radek Faksa really harder to play against than Tyler Seguin?  I doubt it.  But I think we’ve at least got a rough approximation of some of the best centres in the league.

Nevertheless, let’s take a look at the top centre on each team based on goal scoring rate so we can do a bit of a comparison:

Player Team GF/60 Rielly TOI Gardiner TOI
Getzlaf ANA 2.34 39:47 10:43
Domi ARI 2.50 <10 <10
Bergeron BOS 2.63 72:38 26:54
Eichel BUF 2.15 31:40 30:27
Rask CAR 2.50 22:01 31:08
Monahan CGY 2.76 14:01 13:22
Anisimov CHI 2.73 22:01 <10
Dubinsky CBJ 2.55 27:19 28:01
MacKinnon COL 2.54 40:39 <10
Seguin DAL 2.88 29:46 17:28
Zetterberg DET 2.45 53:08 28:54
McDavid EDM 3.41 32:26 <10
Barkov FLA 3.13 78:08 22:22
Carter LAK 2.51 12:48 30:19
Koivu MIN 2.67 33:34 15:37
Galchenyuk MTL 2.82 30:34 31:28
Ribeiro NSH 2.80 15:41 23:21
Henrique NJD 2.42 35:30 17:20
Tavares NYI 2.77 74:42 16:06
Zibanejad NYR 2.87 33:30 22:04
Brassard OTT 2.71 56:28 19:31
Couturier PHI 2.57 14:03 31:40
Crosby PIT 3.18 52:56 <10
Thornton SJS 3.02 40:07 <10
Stastny STL 2.57 31:01 11:15
Stamkos TBL 2.73 52:20 12:24
Horvat VAN 2.27 10:45 20:29
Scheifele WPG 3.31 26:42 22:56
Kuznetsov WSH 3.25 22:21 25:13

I think this second list likely comes closer to what most people would think of as the best centres in the league, even though there are still some notable omissions (like Kuznetsov over Backstrom).  At any rate, between the two lists we’ve capture most of the top centres in the league.

One thing that’s easy to see when looking at these numbers is that there are almost none against whom Gardiner played significantly more minutes than Rielly.  There are a few, but they’re mostly the 2nd or 3rd best centre on their team, which is what we’d expect if Rielly was being hard-matched against top competition.

And it’s pretty easy to see that there are a number of cases in which Rielly is clearly being hard-matched against top competition while Gardiner is kept away.  The players who Rielly played a lot of minutes against while Gardiner barely played them (<10 minutes) are Nathan Mackinnon, Connor McDavid, Sidney Crosby, and Joe Thornton, four of the best players in the NHL.  We also see pretty huge splits against other difficult competition, like Steven Stamkos, Ryan Getzlaf, Patrice Bergeron, and John Tavares.  I think it’s pretty safe to say that, in terms of the top centres in the league, Rielly is facing significantly more difficult competition than Jake Gardiner, and that intuitively it seems like such a big gap in competition should affect the numbers each player puts up.

And yet if you look at their Corsi quality of competition, the gap looks small.  Gardiner’s Corsi QoC is 49.9% and Rielly’s is 50.6%.  It’s not nothing, but it’s certainly much smaller than the gap in Corsi Rel between the two players, which is about 3%.  So if it looks in the above charts like Rielly is being given way tougher minutes, why isn’t it showing up in the tables above?

One reason is that QoC is based on all five opponents on the ice, so even if you play against a top centre, the overall QoC can be brought down if some of the other players on the ice aren’t as good.  It’s also true that the ice time listed here is only a fraction of the total played by each player.  If you add up all the players above who Rielly played against far more than Gardiner did, you’re only accounting for about 20% of his total time on ice.  Maybe Rielly plays tougher minutes in some situations, but the overall spread remains small because such a huge portion of each player’s ice time is played against everyone else.  And yet Rielly clearly does play tougher competition than Gardiner in a reasonably large number of situations.  20% is still a sizeable portion of your his ice time.

I also have a theory about why the spread in QoC seems smaller than it intuitively ought to.  I don’t think there’s any way to prove it, but it sounds to me as though it ought to be correct.  My theory is this:

The spread of QoC around the league looks smaller than it really is because good players play so much of their ice time against other good players, while coaches try to match their worse players against lesser competition.  Now I know what the counter-argument to that is: “We’ve looked at the spread of QoC and everyone plays roughly the same level of competition.”

But I don’t think that’s quite true.  Here’s a simplified way to think of it:

Let’s say two players have a “true” Corsi ratio of 55%.  If those players play head-to-head, the expected outcome for both of them is 50%.  Now imagine two players whose true CF% is 45%.  If those players face each other, their expected outcome is also 50%, even though their actual skill level is far lower than the first pair of players.  If those kinds of situations happen often enough over the course of a hockey season, the differences will slowly add up.  Yes, every player does play some spread of competition, so in the long run you’re still able to separate out the good players from the bad ones, but the gaps will be smaller than the actual spread of skill because of that compressing effect.

To put it in more specific terms, think of it this way: This season there’s about a 3% gap in Corsi between Gardiner and Rielly.  So if you spend a lot of minutes playing against Rielly, your QoC will be lower than if you played against Gardiner.  Now, if we believe that QoC doesn’t have much impact on players, that’s fine.  But if Rielly really is being pulled down by the level of competition he faces, then the QoC of everyone who plays against Rielly is just a little bit under-estimated.  And if that kind of effect is also happening for Rielly’s opponents, then we can start to see how, when you start to add up those small impacts against many opponents, there could be a systematic compression of QoC relative to the real skill gaps across the league.

I’ll leave you with this: Previous research has shown that the spread of QoC across the league is small.  And to some degree that’s probably true of Rielly and Gardiner.  They do play a fairly even level of competition across a sizeable chunk of their ice time.  But it’s also clearly true that Rielly plays a large number of minutes against especially difficult competition that Gardiner is not subjected to.  And I find it very difficult to believe that that gap, which could be something like 20-25% of Rielly’s minutes, doesn’t have any meaningful impact on the shot results for those two players.  I don’t think it explains the entire gap (other factors like quality of teammates and differences in skill certainly also count), but I think it’s fair to say that the gap in ability between Gardiner and Rielly is smaller than you would think just by looking at their possession metrics.

  • BarelyComments

    Great article! I know this is relatively unquantifiable, but I wonder if for Rielly there is a somewhat psychological element to his matchups. Ever since the creation of the Gardiner-Zaitsev pair (used for matchups), Rielly seems to be back to his old self, carrying the puck out of the zone, leading the rush, showing some patience and looking for options when he has the puck, as apposed to rimming around the boards or chipping it off the glass as soon as he touches the puck in the d-zone. I can’t help but think that his state of mind – knowing he was going up against Crosby or McDavid or whoever – was too defence orientated which may have hurt his game. Obviously coaching should be able to help him overcome that but I wonder if it was playing a role. Additionally does anyone have the passions numbers for the defence since the switch to the Gardiner-Zaisev pairing (I believe after the Florida blowout)?

    • Petersversion

      This is a good article. The problem, for me, with the over reliance on hockey analytics, for me, is that they’re still relatively new and evolving. It’s such a fluid, fast, team game. Even in baseball, where individual performance is far more easily isolated, these metrics are still being refined and loopholes still exist (particularly in pitching and defence.) I think extreme usage must surely present a breakdown of sorts in this system. I mean, how can playing against All-Stars all the time NOT be harder than playing against 2nd and 3rd liners? It seems absurd for anyone to suggest that it doesn’t.

      Eventually, baseball found a pretty good place in player evaluation between analytics and traditional scouting.

  • MartinPolak

    The work on fan sites like this one are hands down much better quality then other free or pay for content places. I can’t say it enough. First, Jeffler’s masterful hit piece on TSN’s (POS/Button) garbage Kapenen analysis which showed the stupidity of TSN. And today, this piece that takes down at the cluelessness of another media talking head figure Athletic (Dellow). When I first read the Athletic piece, I thought Athletic’s take was solid on Rielly given him facing extremely difficult Star Quality of Competition. But after reading this piece today, I’m with the author here, these writers should find another job and skip looking at numbers all together because it shows they don’t understand either that well. And I don’t get why the criticism is so understated here because for a subscription I pay for, I expect better quality and hold the publication to a much higher standard.