The perils of evaluating a hockey player based on a small sample
By Cam Charron
10 years ago
I don’t want to write any “I told you so” columns. Those can wait until March. I am cool with watching sportswriters twist in the wind without actually twisting in the wind.
From an October column, written when the Toronto Maple Leafs were 5-1 (and published after the Leafs went to 6-1) titled with the harbinger headline “Where are the critics of Nonis’ off-season moves now?” the Toronto Sun’s resident straw man assured Leafs Nation that Toronto’s controversial offseason moves made the team better:
“[The online statistical geniuses] yelled that Nonis doesn’t know what he’s doing and that tone continued throughout the summer after the deal for David Bolland, the buyout of Mikhail Grabovski, the signings of David Clarkson and Tyler Bozak and the awkward contract negotiations of Nazem Kadri and Cody Franson.”
“[They] make their hockey assessments via charts and graphs,” whined Simmons. “[they] all but murdered Dave Nonis when he traded for Jonathan Bernier.”
It’s an interesting contrast when presented with Simmons’ column from January 11. Since the 6-1 start, Toronto have gone 15-19-5 and, after Saturday’s action, dipped to 12th place in the Eastern Conference. Over the last few days, Carolina, Washington, the New York Rangers, Ottawa and New Jersey have all climbed above the Maple Leafs in the standings, and Columbus could theoretically do it Monday. They’re a point back with a game in hand.
“Randy Carlyle was a success last season and thus far a failure this season. He is the same coach: His team clearly isn’t the same.”
Well… damn it all.
Not to rub salt in any wounds, but one of my major gripes with sportswriters is how quickly the majority leap on small sample sizes. Simmons and Brian Burke feuded, and you have to think Simmons loved every second of the Leafs having winning success a season ago after he got let go. I’m fine debating Simmons any day about why the Leafs had success last year: anybody who read this site will know that I believed that the Leafs winning was not due to toughness or grit or fighting or whatever it was, but a ridiculously high shooting percentage and an above-average save percentage, giving the Leafs the highest single-season PDO.
PDO, explained here, isn’t all that complicated. It’s incredibly volatile in a small sample, which is why I’m wary counting out the Leafs now. Goaltending and shooting are wicked unpredictable, and the Leafs could go anywhere over the next 30 games or so. If their goaltending and shooting are league average, however, they’ll continue to lose games, and that’s mostly because during the offseason, the Leafs prized the wrong talents. This was my assertion, and I’ll continue to stick to it.
I mention small samples. Remember, Simmons wrote this column when Jonathan Bernier had a .974 save percentage and a 0.84 goals against average. Since, Bernier has a .919 save percentage and a 2.83 goals against average. Again, not bad, but we’re no longer talking about a deal that will “surpass” the Roberto Luongo trade made by Nonis in Vancouver as a “piece of hockey thievery”, and it especially won’t considering Ben Scrivens is third among qualified goalies in save percentage (.931) and fourth in goals against average (1.97).
Rather than drop “I told you so” rants after a small sample of games, what I’m going to do is continue to disagree with last season’s narrative fallacies (it’s easier to prove something doesn’t exist than prove it exists, in the case of the Leafs toughness, all you need is a stretch like the Leafs since the start of the Leafs season after Simmons published his original column) and try to ignore the aspects of the game we can’t predict. It’s tough to build a team focused on intangibles because those talents are quite literally intangible, and you can’t tell if current success is going to lead to future success.
There once was a time when Colton Orr served a purpose on the Leafs. Not anymore. As James Mirtle of the Globe and Mail pointed out on Twitter the other night opposing teams are just not engaging Orr and the Leafs in as many fisticuffs as they were last year. The Leafs will be better served having a fourth line that actually can play 10 effective minutes every night than what we are seeing right now.
I think a lot of sports media are too concerned with “what has happened” rather than “what will conceivably happen”. Seasons are long, and we have to take into account adjustments, regressions, injuries, or any unpredictable events that shake up the makeup of a team. Rookies play their way on. Veterans play their way off. They’re traded. Opposing coaches adapt to a powerplay strategy. The goalie develops a bad case of stomach flu and stops 80% of pucks for a two-week period…
That can be traced to what the Leafs were apparently doing a year ago that just doesn’t work this year. Teams have learned to play against the Leafs. They don’t have to engage them physically, and the Leafs’ best players aren’t necessarily physical. That’s not a bad thing, but the Leafs are trying to win with an unsustainable style. Maybe there are situations where a fight or a hit can rally a team to win. All I know is that the data tells me you can’t count on fights, hits, grit, or even talent, alone.
Hockey is a weird game, in that luck dominates a small sample and can lead people to curious conclusions. I don’t pretend to know the answers, but I do know what the answers aren’t. You can’t focus on one small aspect of the game and expect that this is the indicator a team is primed for success or failure, without doing the necessary legwork to see if a trend that presents itself in October will last in March.
The most important goal in hockey is “the next one”, which is what any team is playing for, whether the team is 0-0 or 6-1. Shifting your understanding of the game to tell me what “will” happen and “why” is probably what’s going to get me to pay attention again to mainstream sportswriters who love to fabricate a compelling story based on characters they love or hate and project them on the page.
Recent articles from Cam Charron
