Goaltenders. The single most impactful individual position in hockey. Masked, brave, batshit crazy, and for the most part, completely unreliable. Developmental trajectories and predictive metrics that have been curated and adjusted by analyticians over the years to judge NHL skaters, cannot be applied as reliably to goalies.

Goaltenders live on the margins. Think about the difference between a netminder with a .930 save percentage, versus one with a .900. A three-percent delta in efficiency that separates a future hall-of-famer, from a garden-variety backup. By default, small variations in performance can create huge disparities in the metrics.

It’s part of the reason why eight of the top-10 WAR players in the NHL last season were goaltenders. It’s the position with the most direct impact on wins and losses, paired with the slimmest margin for error. Naturally, this has made the position somewhat immune to predictive analysis.

Being the stubborn person that I am, I wanted to dig deeper. Perhaps there is a better way to properly predict outcomes for these masked psychopaths. With that objective in mind, I spent an unsavory amount of time looking at long-term goaltender trajectories. Then I applied a myriad of different circumstantial factors in search of anything that represented a trend.

Predictably, once the initial layer of chaos was peeled back, I found several more layers of unmitigated randomness. That being said, glimmers of clarity revealed themselves in the process. It inspired my position that, while goaltenders are inherently difficult to predict year-over-year, many of us are probably doing it the wrong way.

Let’s dive in.

Sample Parameters

Before we get into the deep contextual analysis, let’s establish the parameters of the sample, and the study as a whole. As previously stated, my interest here is long-term predictive ability. With that in mind, the timeline for the study consisted of four seasons (2019-20 through 2022-23).

As for goaltender sample parameters, I set the minimum at 18 games played per season. In doing that, we’ve ensured that every goalie in the sample has played in at least a primary backup capacity (or greater) in each of the last four seasons.

The part that amazed me was how few goalies met the criteria (which I felt was extremely reasonable). In total, only 29 NHL netminders have played 18 games or more in each of the last four years. It speaks to the position’s parity and turnover rate leaguewide. Even more stunning was the fact that only 11 of them played for the same team all four years.

Baseline Data

Before I started compiling the data for each player in the sample, I wanted to establish some generalized averages for a comparative baseline. Primarily, I was interested in the average degree to which GSAx/60 rates fluctuate (whether it be positively or negatively) year-over-year for the average NHL goalie. As a reminder, GSAx is an acronym for “goals-saved above expected”.

The result nearly turned my brain into soup. Over the last four seasons, the average NHL goalie (who met the sample parameters) experienced a .361 GSAx/60 delta from one season to the next. Since the goalies in the sample averaged 2,258 minutes per season, it means that they fluctuated by 13-14 goals allowed per year (again, that could mean more or fewer) on average. This equates to a goal about every three games or so.

The most consistent goalie in the sample, John Gibson, experienced a four-year average variance of .051 GSAx/60 (which equates to about a three-goal average disparity per season). That’s pretty incredible. Sadly, this player was a major outlier. On the opposite end of the spectrum is Carter Hart (another major outlier), who averaged a year-over-year variance of .812 GSAx/60 (which equates to about a 34-goal average disparity per season).

Both are staggering compared to the .361 average I mentioned above. The variances in general were vast, even among the rest of the sample. Only six of the goalies on the list fell within a 10% threshold of that average (whether it be less or more). Simply put, these guys are all over the place, which is about what I expected.

You might be saying – hey wait a minute, John Gibson has lackluster GSAx numbers over the last four years. How is he the most consistent goalie? Well, remember, “consistent” and “good” aren’t always synonymous (more on that later). Consistency is consistency, whether it be consistently good, bad, or indifferent.

With a framework established for baseline trends for the average goalie, it’s time to observe their respective environments. Primarily, is there a correlation between poor defense and poor GSAx rate? Perhaps defensive consistency plays a role? Could year-over-year individual goaltender consistency correlate with projection success? Maybe there’s a way to amalgamate all of it to better prognosticate future success.

Or maybe it’s all an act of futility and goalie analysis is >50% luck. Only one way to find out…

GSAx Rate Versus Team Defense

Let’s start with the basics. For years, fans have asserted that GSAx fails to adequately adjust for poor defensive play. Nobody to my knowledge, however, has taken the time to try and prove or disprove said narrative. Naturally, it was the first thing I wanted to test against the sample.

The average NHL team from 2019-2023 produced an overall xGA rate of 2.96 in all situations. In the chart below, I mapped out whether good GSAx goalies tended to play behind good defenses, and vice-versa.

As you can see, the sample is pretty well-distributed. While there is a subtle correlation between the GSAx rate and team defense (i.e. the sample concentration in the top-left and bottom-right of the chart), the GSAx metric seems to do a pretty good job of adjusting for defensive play.

That being said, good goalies are spread all over the place, playing behind a wide spectrum of defensive competence. Poor goaltending does gravitate a bit more toward the sub-par defensive side of the spectrum, but again, the distribution is subtle and certainly not a significant indictment of the metric.

Since the most entry-level comparison didn’t yield anything noteworthy, I had to apply additional context. Consistency was an idea that came to mind. Perhaps goaltenders who experience more volatile ranges of shot-quality-against rates per year, tend to fare poorly (and vice-versa).

So, my next step in the research was to measure year-over-year individual GSAx variance with year-over-year xGA variance. Essentially, do the goalies with the highest average GSAx variances play behind comparably volatile defenses? This is especially interesting when observing goalies who have changed teams at some point in the sample (which, as a reminder, ranges from 2019 to 2023).

GSAx Consistency versus xGA Consistency

People talk all the time about goalies getting into a “groove”. Maybe that’s a real thing. Said groove could be tough to maintain if the team in front of a given netminder fails to produce consistent results, right? Well, let’s see.

Now remember, this chart measures raw variance. It does not distinguish between the negative and positive. It’s simply a full-spectrum, year-over-year volatility metric for both goaltenders and the defenses they’ve played with over the past four seasons. As previously mentioned, the average goalie in the sample experienced an average GSAx/60 delta of .361. The average NHL team that these goalies played with, fluctuated by .269 in terms of xGA/60 delta, year-over-year (all situations).

I’ll be honest, this wasn’t the result I expected. The more consistent goalies in the league played in front of a wide range of defensive consistencies. That’s not groundbreaking. What surprised me is that Philipp Grubauer was the only goalie in the sample whose inconsistent year-over-year nature correlated with a similarly inconsistent defensive environment (to a noteworthy extent at least).

A vast majority of the goalies who played in volatile defensive environments (on a year-over-year basis, not to be mistaken with game-by-game) seemed unaffected by them in terms of GSAx consistency. Around half of the goalies with high GSAx variances played behind some of the most consistent xGA teams in the sample (see the top-left quadrant of the above chart).

Yet another nod to the formula that goes into GSAx metrics, and its proficiency in accounting for team defense.

Year-Over-Year GSAx Deltas versus Long-Term GSAx Average

Intentionally, I went about this in sort of a reverse order from my norm. Typically, I’d have looked at goalie performance in a vacuum before applying the contextual factors I described above. There’s a reason I did it that way.

I wanted to give readers the ability to refer back to the contextual charts when going through this section. That way, if you see a goalie who experiences year-over-year GSAx volatility below, this format gives you the ability to go back and search for additional context, if you so choose. Maybe I’m nuts, but I felt like the digestibility of the content would be easier that way.

Anyway, let’s do that thing where we start broad and narrow our scope as we go. This first chart is simple. We’ve already talked about the degree to which the average goalie fluctuates in terms of GSAx delta on an annual basis. Let’s now see how that individual consistency (or lack thereof) correlates with long-term GSAx success.

Candidly, this is my favorite chart from the study. It’s the first one that gave me enough information to establish a new facet in how I analyze goalies moving forward (which I’ll expand upon later).

We’ll start with the top-left quadrant of the chart. These are the goalies with below-average year-over-year GSAx deviations (i.e. more consistency), who also post good long-term results. There is an equal sample of goalies who are just as consistent, but they’re consistently bad (bottom-left quadrant).

So, consistency isn’t a trait that leans positive or negative regarding sample distribution. Year-over-year inconsistency, however, nearly exclusively correlates with poor long-term results. So, while you have goalies who vary wildly each year in terms of GSAx, they’re usually that way because of a positive outlier season that they couldn’t replicate.

Beware of Outlier Seasons

Look at the goalies in the bottom-right portion of the chart. Notice anything about them?

No? Let’s see if I can help – what do Philipp Grubauer, Joonas Korpisalo, Jack Campbell, and Matt Murray all have in common? Do they all sound like guys who got paid a lot of money and term at some point and their team ended up deeply regretting it?

How did this happen? Well, it seems like general managers tend to bank on an outlier season as a given netminder’s “new normal” so to speak. Previously struggling or inconsistent goalies can get paid a lot of money after one remarkable season, especially youngish goalies who can trick GMs into believing that they’ve finally hit their prime.

This rarely works out. At best you hope that they can replicate it 1-2 times across the length of their contract and pray the rest of your team is built to capitalize on it. Signing a goalie with a volatile history means being okay with a bad season for every good one they give you, and GMs probably don’t see it that way, as evidenced by their willingness to give them term.

In best case scenario, you end up with a player on the “chaotic average” section of the chart (Fleury, Andersen, Samsonov, Markstrom) who is volatile but at least can produce a random GSAx bender.

Simply put, not all consistent goalies are good, but the best goalies are almost always consistent. I’ll stop just short of calling it a requirement, but if you’re planning on signing a goalie long-term, the trend above seems to speak for itself.

Age Versus Consistency

We’ve reached the “grasping at straws” portion of the study. Since it was tough to glean anything from the baseline data other than “good goalies are consistent goalies”, I thought perhaps there might be some sort of age correlation as it pertained to consistency.

The sample contained a healthy portion of both young and old netminders, so if there was some sort of correlation, the data distribution would likely provide a nice visual representation of it. Sadly, no such trend existed.

Sure, the youngest and oldest goalies in the sample happened to be the two most inconsistent in terms of year-over-year reliability, but other than that, the results were all over the place. So much for that.

The High-Danger Factor

In the interest of complete transparency, I’ll preface the following by admitting that I have no recollection of where and when I first read this theory. A few years ago (maybe more), I recall stumbling upon an article that asserted that goalies with strong high-danger save percentages tend to have more favorable long-term success.

I don’t remember the author or publication, but it’s something that stuck in my mind. I wanted to test it against my 2019-2023 sample. So, I took the goalies with the best and most consistent track records and compared their HDSV% to the goalies with the worst and least consistent track records to see if there was a correlation.

Predictably, all of the best goalies had above-average high-danger save percentages (save for Jake Allen, who was just below average). What I didn’t anticipate was players like Murray, Hart, and Korpisalo putting up strong high-danger numbers amid rather abysmal GSAx marks overall.

We’ve gleaned nothing novel here. At best, you would use this tool to gauge the extent to which a goaltender is defensively-reliant. Even then, you take a player like Allen who has played well despite poor team defensive metrics, yet he isn’t great with high-danger chances, so I suppose he just gets peppered with quantity versus quality, historically speaking? He’s a peculiar one in this case.

Again, I wouldn’t put this metric in the core analytical process in goalie evaluation, but it is a peripheral consideration, especially if a given GM knows that his team lets up a substantial rate of high-danger chances against. Naturally, a goalie with a historical propensity to make high-danger saves at an impressive rate might fare better in that environment.

Join us for Part Two as we unpack our findings and decide how to use what we’ve learned in an attempt to improve our chances of correctly prognosticating future performance:

Part Two Link Here

Advanced Metrics courtesy of Evolving Hockey and Natural Stat Trick

Photo Credit: Debora Robinson/NHLI via Getty Images