James Reimer, Jonathan Bernier, and the hot-hand

This paragraph surfaced in a story written last week by Sean Fitz-Gerald at the National Post, with Randy Carlyle speaking about this goaltenders James Reimer and Jonathan Bernier and how he plans to use the two for a tandem. I’m not going to bicker about Carlyle in this post because 99 per cent of hockey coaches fall into a trap of thinking that streaky play is predictive of the future.

A lot has been written about the “hot-hand” effect as it pertains to basketball and to baseball. Streakiness is generally accepted in sports and coaches have an unfortunate tendency to lean towards players who have produced better results in the recent days. In one unfortunate decision in the fall, with his team down by a run in the ninth inning of Game 3 of the American League Division Series matchup against the Baltimore Orioles, New York Yankees manager Joe Girardi pinch-hit Raul Ibanez for Alex Rodriguez, one of the best hitters of all-time, but had been slumping. The move was instantly panned on Twitter—until Ibanez cracked the game-tying home run off of Orioles pitcher Jim Johnson.

The move wasn’t a good one. The Yankees needed base runners, and Ibanez’ on-base percentage of .308 was miles behind Rodriguez’ .353 that season. Had the move not paid off, Girardi would have told reporters after the game that he wanted to wake Rodriguez up, or something of the like. He certainly wouldn’t have said that he expected Raul Ibanez, a career journeman, to hit the game-tying home run in the 9th and the game-winning home run his next at bat in the 13th inning, which also happened. The national media would have thought he was insane.

I bring up the Rodriguez-Ibanez story not because it proves my point about the fallibility of managers, but because it does exactly the opposite. Without knowing Girardi’s thought process when he switched Rodriguez for Ibanez, we have no idea of knowing whether his hunch paid off. It is very likely that Girardi and the Yankees got lucky—almost assuredly—but there’s a way of rationalizing those explanations after the fact as to why it was a good move. ‘Rodriguez wasn’t playing well‘, ‘you’ve got to do what your gut tells you‘, that Ibanez is a ‘great low-ball hitter and we’ve got a shorter porch in right field‘.

Once an event happens, it’s easy to look back and try and find an explanation for why. Our minds are attuned to finding patterns and sequences. It’s just why, if you’re like the average person, you’re more likely to accept that the words VOTE and NOTE rhyme with each other at first glance than you are to accept that REIGN and PANE rhyme with each other.

Apply that to hockey (I wrote up a post last December about how then-Canada coach Steve Spott ought to deploy his goaltenders). A goalie has a good night? “He’s been playing great for us.” A goalie has an off night? “He’s been struggling a bit in practice.” Sports are presented to the public as a theatrical production, but the scripts are written at the conclusion of the act, not at the beginning.

…let’s talk about James Reimer, or his performances as they relate to the opening quote of this post. While it’s easy and accepted for Carlyle to suggest that he’ll go with the winning goalie in a tandem system, you’re not giving yourself a better chance at victory than if you flip a coin.

I went through James Reimer’s game log and isolated all the games he started in, and then took away starts where there were five or more days break between them to account for streakiness or feel. That left me with 98 starts (I included the postseason) and I broke them up into how Reimer did in his 49 starts following a win versus the 49 starts he had following a loss.

The results aren’t too surprising, if you’ve broken this up before:

Why does James Reimer have a higher save percentage, win percentage (OTL not counted in this percentage), and quality start rate after a loss than a win? It’s not like this is a new thing. I’ve done this sort of analysis for 20 or so goalies in my spare time and none of them have shown to have better statistics after a win than a loss. In fact, Reimer seems to do dramatically better in his 49 games coming off a loss in the last five days than a win.

But wins and losses aren’t the best way of determining game-to-game goaltender performance. At Hockey Prospectus, Robert Vollman created a metric called “quality starts”, that was later touched on in his Hockey Abstract book. For our purposes, a “quality start” is a game where a goaltender has a save percentage of .913 or higher, or has a goals against average of 2.00 or below with at least an .885 save percentage.

The converse is “blow ups”, created initially by Thomas Drance at Canucks Army, which is a game where a goalie has an .849 or below save percentage, or allows five goals without his team having surrendered 40 shots on goal. Those two statistics are labelled in the graphs above and below as QS% (rate of games with a quality start) and BU% (rate of games where the goalie was blown-up).

Here’s how Reimer does in the 49 games following a quality start:

It’s not as distinct, but Reimer does have a slightly higher save percentage, slightly lower save percentage, and lower blown-up rate in games coming after a non-quality start. The Leafs also enjoy a slightly higher win percentage.

So what’s that to tell you? I’m sure there’s a reasonable explanation for why goalies do slightly better following a loss than a win. It could be anything from fatigue to team adjustments being more common after losses, but I’m not going to subscribe to either theory until I see evidence.

Just to show how independent these are, here is how Reimer does in games following a start on an odd-numbered date as opposed to an even-numbered date:

I actually had to double-check to make sure this was true, but that’s how it goes. There’s no possible way that the arbitrary date on the calendar where a game was scheduled could have a tangible effect on the way Reimer plays, but over 98 starts, Reimer puts up better save percentage, goals against, win percentage and blown-up numbers. Quality starts are the same.

What’s the lesson from this? It isn’t to say that Reimer will perform better in the future in games coming off an even-numbered date, a loss, and a non-quality start (the Leafs are 11-4-1 in such games and Reimer has a .937 save percentage), it’s just to say that perhaps creating a competition in goal isn’t the most efficient way to deploy goaltenders over an 82-game season.

If I had to wager, Reimer will have a slightly higher save percentage than Bernier over the next three seasons, but betting on goaltenders, coming off a win or a loss, doesn’t seem to be a very fruitful endeavour.

My suggestion for Carlyle would be to draft up a schedule for starts two or three weeks in advance and stick to it, because it’s easy to get carried away when you think you’ve come up with a system that could just be determined by randomness.