You never played the game

The last few years have seen NHL teams make many analytics hires. Tyler Dellow, Eric Tulsky, and Darryl Metcalf to name a few. A couple days ago, the Montreal Canadiens went backwards and made an analytics firing, letting go of their recently hired analyst Matt Pfeffer

Pfeffer released a statement regarding what happened, so I won’t make any speculation further than that.

Regarding the incident, NHL television analyst and former professional goaltender Corey Hirsch had this to say:

Here we get to see a sharp distinction between the thought process of players-turned-analysts and analysts who were more… organically grown. In the latter, I’m referring to those who are more statistics-inspired hockey people. Surely, you’ve heard the argument “you’ve never played the game, you don’t know what you’re talking about” or something of the like. Maybe you’ve even made that argument. Well, Matt Pfeffer never played the game, and yet the Canadiens, at one time, believed he did know what he was talking about.

Nevertheless, I want to point out that Hirsch clearly has a different point of view than the analytical people would. I’m not here to make fun of him for that, or call him ignorant. I think what he’s saying makes sense.

It’s wrong, but it makes sense.

PERSPECTIVE

Now, I’ve never played the game. I mean, I played houseleague a bit and a ton of road hockey. But, I’ve never been to “the show”. Hirsch has. He played hundreds of games of pro hockey, including stints with several NHL teams. He knows what the league is like from a player’s perspective. You can see him in the image above robbing Mario Lemieux. Mario Lemieux!

To his points, his Kris Draper analogy is entirely correct. But unfortunately, I see it as irrelevant. Hockey players often see management in pigeonholed roles. A scout scouts. A manager manages. A data analyst provides data for analysis. A coach coaches. And, a player plays. They see it this way because the last point is the only thing that matters to them. Play the game.

But what about those of us who never played the game? Or, those who did and have shifted away from that perspective? Things are seen differently. A manager does more than just manage. A scout does more than just scout. And a data analyst, in kind, has to do more than just look at data.

In reality, the front office staff all have skills relating to managing a hockey team, including the data analysts. Matt Pfeffer was hired because he has some of those skills. His primary skills are as a data analyst, but if you threw him in the manager’s seat, he’d have an idea of what to do.

But players don’t see that. They don’t care about that, nor should they. And this is exactly where Hirsch’s take comes from. It’s easier for a player to look at the front office as homogenous entities that have entirely separate functions. But the real and far more complicated picture is that everyone participates in all areas of managing the team, in different capacities. This is what Hirsch, and many other player-turned-analysts fail to see.

A good scout watches players, tracks events the player was involved in, analyzes that data (albeit usually quantitatively), and uses the conclusions to present an opinion on a player. A good manager takes all the sources of data he has, analyzes them, inputs his own experience and opinions on players, and uses those to make a decision. A good data analyst researches the nitty gritty statistics to the bare bones, combines this with scouting of players, and presents the conclusion they come to in a format that contributes to building a better hockey team. A good coach utilizes direction from the management and analytics staff, scouts the opposing team to prepare a game plan, and takes all that into the bench to manage the game from there.

CONCLUSION

This doesn’t have to be a never-ending problem in the media. Networks can make a change and shift away from people whose only contribution is their experience playing the game. I’m not even suggesting to not hire former players. I’m suggesting to hire the right ones. The ones who can get out of the mindset they had as a player and actually analyze. Mike Johnson, Ray Ferraro, Kevin Weekes, and Darren Pang to an extent, are all great examples of former players in the media doing it right.

I’m not saying Hirsch is necessarily doing it wrong. But he seems to still be carrying the perspectives he had as a player. Unfortunately, in his new job, I think that perspective can hinder as much as it can help.

The takeaway point here is to always keep your mind open to new perspectives. It’s the best way to learn and stay ahead of the curve. 

  • Hockey Hoser

    Hirsch, the fire in your cave is about to burn out. Better go take care of it.

    Pteffer did his job and provided an alternate opinion based on metrics. So his contract is not being renewed.

    We don’t know for a fact that it’s because of the report he submitted about Subban. Could’ve been due to many reasons that occurred during the time of his contract that we are unaware of. Media making mountains out of mole hills as per usual.

  • Jeremy Ian

    This whole thing is a little absurd, and unfortunately does not reflect well on the place of using quantitative data in decision making. Not for Corey Hirsch’s reasons necessarily. But because Matt Pfeffer behaved unprofessionally by coming out, after his contract was up, and disclosing his side of the story when he knows full well (or should know) that others in the room making the decision can’t and should not air their reasons.

    So, what gets recycled in the press is a one-sided account, which has itself become the story, which only adds to the misunderstanding and false dichotomization between quantitative and qualitative evaluations.

  • DJ_44

    Perhaps Hirsch’s comments touch a sore point amongst the analytics crowd. My interpretation of the entire thread: his comments were more general about the nature professional employment.

    I know nothing of situation or the individuals involved. But Hirsch’s point about an individuals role is not wrong; it is spot on.

    If the analysts role in the Habs organization is to present information to management (clearly, concisely), then that is what they should do: tables, charts, information sources, confidence intervals, summary points.

    The person who is responsible for making the decision will take the information, along with a lot of other information that the analyst may not be privy to, assign appropriate weight to each, and make the decision.

    I would be concerned, if presented with a report where I wanted the numbers, and it included “passionate” unsolicited opinion. We all know that statistics can be skewed to fit one’s opinion or argument. Mixing in opinion will taint the analysis (or give that appearance).

    …..there’s a Hollywood screenplay in this somewhere………Tom Cruise as the under-appreciated analyst…. help me help you……help ME help YOU

  • jasken

    I think you are being overly sensitive about Hirsch’s comments and so misinterpreting them.

    His key comment is “everyone on a team has a role.” As such the Pfeffer’s of this world have a right and responsibility to fulfill their role and conclude that PK is more valuable than Weber. However, not every decision can be or should be based 100% on analytics. For instance, a guy could be a better player through the analytics lens, but regularly shows up late for practices, setting a bad example for other player’s and undermining the coach’s authority. How does this type of “data” factor in to the decision to move a player?

    I think the point Hirsch is making is that the analyst can say that A is better than B, but saying that they should not swap A for B is going too far, because the decision is a little more complex than just who has better fancy numbers.

  • Heschultzhescores

    “I think the point Hirsch is making is that the analyst can say that A is better than B, but saying that they should not swap A for B is going too far, because the decision is a little more complex than just who has better fancy numbers.”

    This.

  • jasken

    Hirsh should have just said I will tell you what go get a Network Adminstrator we will make him the new programmer because he can do his job better than the programmer does his and we will make the programmer the NA and everything will work right. It’s not their roles but based on the NA’s history he will be better in that role look how good he is being NA. Nice logic

  • jasken

    This is a timely article as it demonstrates why Polak and Martin are ideal for the leafs whereas analytic nerds hate on those players. By fancy stats Polak and Martin are puck possession black holes which is the level of their analysts.

    That said, Dubas and Charron detailed analysis for the leafs also includes context like character, grit, locker room fit to complement the simple raw fancy stats. This is the key insight that is missing from more 1 dimensional analysis that jeffler and dragalike have shown.

    • Gary Empey

      Polak and Martin aren’t “ideal” for the leafs. They are OK deals though.

      Polak isn’t the worst 3rd pair defenceman, and considering the number of RHD available in free agency and how much they wanted his contract is fine, especially at 1 year. Polak is just fine for the leafs right now.

      Martin is most definitely not an analytics black hole, there are actually metrics that look on him favourably. Contrary to popular belief “fancy stats” don’t all say the same thing. Other than the top elite players, and the below replacement level players it’s very much up to how you interpret the data, and different analysts put more value on certain areas. The biggest knock on Martin is that he’s a bottom 6 forward no matter what you say, and while you do need those to be good, giving them term generally works out poorly. His contract is market value, and again, good, but not ‘ideal. The analytics arguments I’ve seen are that what’s happening is fine, maybe even good, but not perfect.

      If ‘locker room fit’ means more to your team than playing ability I doubt the team will actually make it that far. I don’t want to say the ‘intangibles’ don’t exist, because they very much do, I’m just saying they should come second to player’s abilities.

    • Gary Empey

      Re- “why Polak and Martin”

      After the Leafs traded Phaneuf, Polak, and Winnik they were likely the lightest team in the league. With the players coming up from the Marlies they were about to get even lighter. Opposing coaches can easily figure out how to play a really light team.

  • jasken

    “I think the point Hirsch is making is that the analyst can say that A is better than B, but saying that they should not swap A for B is going too far, because the decision is a little more complex than just who has better fancy numbers.”

    Exactly this. Perfect.

    I agree with Pfeffer stating that he feels Subban is a better player, based on the underlying numbers.

    But I don’t agree with Pfeffer making a “passionate” plea to management to not make the trade. Doing so means he thinks the numbers are ALL that matters, which is being just as closed-minded as the traditionalists are being when they scoff at analytics.

    • Gary Empey

      Re- “Pfeffer making a “passionate” plea to management to not make the trade.”

      Is simply wrong. It never happened. It was reported in the media but is was incorrect. Hirsch likely made his comments after reading the erroneous initial report from a Quebec radio station. It has since been corrected.

      Pfeffer made two analytical reports and send them up to management for review. That’s it.

      His contract ran out and Montreal didn’t renew it.

  • Gary Empey

    Analysts in general make the statement analytic’s is only one of many tools people should use to evaluate players.

    Then they go on to infer analytic’s is the most important tool and should trump all other information about a player. Anyone who doesn’t agree must be ignorant.

    It has far too many unexplained anomalies that make the whole skill questionable.

    Eg. Analytically speaking Corrado is the best Leaf defenceman. Does any Leafs fan here actually believe that?

    Eg. The Leafs were a good possession team last year.

    They were also 30th overall. Considered the worst team in the league. What value can I really place on possession?

    I am not completely against analytic’s. It is still in it’s infancy. When the data completely contradicts the eye test, there must be a major flaw, that needs to be accounted for, and corrected.

  • BorealNinja

    Good read!

    Analytics are a useful tool that is worth the proper investment, just like any other aspect of research and develpment. However that doesn’t mean that it is to be used in a “all the eggs in one basket” approach.

    I feel bad to the guy from Montreal, to me it sounds like a passionate individual who was confident and trying to do the job they pay him for. I know I can get worked up too trying to make a point at my job, especially in situations where you think your opinion is being dismissed without due consideration. So when the “I told you so” moment comes I hope he is smiling to himself.

  • Hoping you guys will promote this fine piece again this week in light of the analytics vs played the game kerfuffle on July 18th, specifically this comment, “I’m waiting to meet an analytics person who’s had to go into the corner and get a puck against Shea Weber. Let me know if anyone finds one.”