Umpire Calls: Is the Impact Upon the Toronto Blue Jays Overstated?

Early in the 2022 MB season, there is much discussion concerning the number of incorrect umpire calls of balls and strikes. Fans of the Toronto Blue Jays have been very loud on social media with claims that the Jays are losing games because of umpires. Is the claim valid? How impactful are missed ball-strike calls? Are incorrect calls distributed evenly across all teams?

Featured Images Courtesy Of DaveMe Images. Prints Available For Purchase


The most recent umpire to attract the ire of Blue Jays fans was Jeff Nelson. His April 16 performance as the home plate umpire was one to remember. is a website that measures the home plate umpire’s accuracy, consistency and impact on both teams. After the April 16 game with Oakland, where the Blue Jays lost 7-5, Umpire Scorecards tweeted this image. The notable Nelson data points from Umpire Scorecards are as follows:

  • 18 of 56 called strikes were actual balls
  • 4 of 104 called balls were true strikes
  • The MLB average accuracy for umpires is 94%, but Nelson’s accuracy mark was 86%
  • The net impact of the incorrect calls was 1.41 runs in Oakland’s favour


Nelson’s performance on April 16 was terrible. Social media was the place to gauge the anger towards MLB umpires in general and specifically Nelson. As a result, there were pleas for automated ball-strike calls (namely, robot umpires). Let’s dig into the Umpire Scorecards data and see how impactful incorrect ball-strike calls were in 2021. The 2022 season is too small a sample size to reach any conclusions.


In addition to examining how Umpire Scorecards estimate the impact on runs from erroneous ball-strike calls, I will use two other approaches to assess the team-run effects for American League East teams during the 2021 season. Those different approaches are as follows:

After the three methods and results are outlined, I will translate the estimated run impact into team wins.

Umpire Scorecards Method (“USC”)

According to Statcast, in 2021, 353,538 of the 709,851 pitches thrown by MLB pitchers were called a strike or a ball. Of the 116,618 called strikes, 17,746 (15.0%) were outside the strike zone. 236,920 pitches were called balls, and 11,615 (4.9%) were in the strike zone. The number of incorrect calls is notable, particularly pitches deemed to be strikes but were balls.


The folks at USC have developed a methodology to estimate the impact upon runs from these missed ball-strike calls. A full explanation can be found here. Here is a brief description of the USC approach:

  • For every team in every game, the missed ball-strike calls data is collected for the pitchers and batters.
  • USC uses the Run Expectancy Matrix to estimate the impact of an incorrect call.
  • For example, with the bases loaded and two outs, an incorrectly called strike in a 3-2 count ends the inning. However, if the umpire had made the correct call, the batting team’s Run Expectancy would have increased by 1.74. The one run that would have scored from the walk with the bases loaded plus the 0.74 expected runs with the bases loaded and a 0-0 count for the next batter.
  • USC tracks all games and repeats the process for each missed call in the context of the count and game state (runners on base and the number of outs).

Table 1 shows the result of missed ball-strike calls for Toronto, New York, Boston and Tampa for the 2021 season.


The Toronto Blue Jays’ net run impact from missed ball-strike calls in 2021 was -7.02, which ranked 24th in MLB. Tampa was worst among the 2021 American League East playoff contenders, with an -8.02 net run effect and a 25th MLB ranking. The Yankees were middle-of-the-pack with their 0.22 net-run loss (15th in the MLB). Boston was the best of the American League East bunch. Their favourable net runs were 5.61, which was the seventh-highest in MLB.


Table 1 also illustrates that the Blue Jays benefited least/were negatively affected more in most games. In other words, in almost 52% of their 2021 games, Toronto was helped less/hurt more by the impact of missed ball-strike calls than their opponent.


Table 2 shows the distribution of net unfavourable and net favourable by wins and losses. There are a few items to point out. First, having your opponent benefit more from incorrect calls in a game does not necessarily mean that the opposing team will win the game. In 2021, Toronto won 42 games in which they were the relative beneficiary of incorrect calls and had 31 victories when their opponent was the net beneficiary from missed ball-strike calls.


The 2021 Tampa Rays demonstrate that having almost 57% of your games whose opponent came out better in the missed call metric is not an impossible challenge. 58 of the 100 Tampa victories occurred when the missed ball-strike calls favoured their opponent.


Toronto did not have a win or a loss in which the score difference was less than the run impact of missed calls. The Yankees, Red Sox and Rays each had a loss where the run effect of missed ball-strike calls exceeded the score difference.


By the method employed by USC, the net effect upon runs from missed ball-strike calls is not as much as some fans believe.

wOBA Method

The wOBA approach is similar to USC’s, but instead of using the Run Expectancy Matrix, I used the wOBA of each team by count. However, this method does not consider the game state (number of outs and runners on base).


To understand the calculations, consider when Toronto was at bat with a 0-0 count, and a strike was incorrectly called. The Blue Jays’ woBA in the incorrect 0-1 count was 0.401, but the count should have been 1-0, with an 0.422 wOBA. The difference of 0.021 (0.422 – 0.401) was multiplied by the net number of incorrect calls (88) in the 0-0 count to determine the run effect (1.85 runs). Please note that a wrong ball call (the pitch should have been deemed a strike) is netted against incorrect strike calls.


A similar approach is used concerning the effect of incorrect ball-strike calls on pitching results. The wOBA against for each team is used in each count.


Table 3 shows the net impact upon runs (batting less pitching) from incorrect calls. As the data illustrates, the net effect of the team is similar to the results produced by USC.

Run-Strike Equivalency

The run-strike equivalency approach relies upon the conversion rate developed by Baseball Savant for its Catcher Framing metric. Baseball Savant determined that the run conversion rate for strikes gained via catcher framing is 12.5 runs per 100 strikes gained.


Table 4 shows the incorrect calls for Baltimore, Boston, New York, Tampa and Toronto when batting and pitching. The right side of the table shows the run effect of wrong calls using the 12.5 runs per 100 strikes conversion rate.


The table also demonstrates that the impact of incorrect ball-strike calls is negligible and similar to the results produced by USC and the wOBA method.

Deficiencies of the USC, wOBA and Run-Strike Equivalency methods

There are a few deficiencies to note concerning the three approaches

  • All three rely upon averages, which could affect the results
  • None of the approaches consider leverage (high, medium or low)
  • The wOBA and Run-Strike Equivalency methods do not factor in the game state (bases occupied and the number of outs)


However, in my opinion, the USC results are reasonable when one considers that the wOBA and Run-Strike Equivalency methods produce similar results to those of USC.

What About Wins?

FanGraphs annually updates seasonal constants, including runs per win. In 2021, every 9.973 runs can be converted into one additional victory. Table 5 shows the conversion of runs into wins for the three methods.


My conclusion is that incorrect ball-strike calls did not affect the 2021 standings. I suppose one could look at Boston’s 0.7 win reduction in Table 5 and Toronto’s 0.7 win increase and claim a one-game difference between the two clubs in 2021. However, I believe that the numbers are not so precise that one can reasonably sustain that argument.

Why Robot Umpires?

If missed ball-strike calls have a minimal effect on game results, why have automated balls-strike calls? The first reason is that if MLB has the technology to get more ball-strike calls correct, then why not use it? Adoption of robot umpires would be consistent with the utilization of video review. If MLB can get the right call via technology, then do it.


The second reason to adopt robot umpires is MLB, and its teams have partnered with gambling/betting companies. I do not doubt that these gambling entities will pressure MLB to get calls correct if the technology exists. Even if the true impact of incorrect ball-strike calls is negligible, the perception that umpire ball-strike missed calls impact game results (wins, losses, the margin of victories, strikeout totals, etc.) is not in the best financial interests of MLB and its betting partners.


The third reason is that correct ball-strike calls should, all things being equal, lead to more runs. The USC, wOBA and Run-Strike Equivalency methods support that conclusion. According to USC’s 2021 summary, the average American League team would score 22 more runs if there were no missed ball-strike calls.


Table 6 reflects 2021 American League batting data, the run value of each batting event and an estimate of the increase in events, assuming that each team scores 22 more runs because there would be no incorrect ball-strike calls. That works out to be an additional league-wide 231 hits, corresponding to a four-point increase in batting average. According to Jayson Stark, in his article for The Athletic, What would happen if baseball killed the shift?, Bill James estimated that the mean batting average would rise by four points if the shift were banned. The identical estimated impact from the use of robot umpires.


Furthermore, the 231-hit increase may be understated. Suppose that 2021 batters reacted to the 15% error rate on strikes called by expanding their hitting zone. By enlarging the hitting zone, the average slugging percentage would be reduced.


Consider the 2021 Toronto Blue Jays and the attack zones. Table 7 shows that when the average Blue Jays batter swings at pitches further away from the heart zone, the lower the slugging percentage on batted balls. The same is generally true concerning batting average, but the average in the outer shadow zone is higher than that of the inner Shadow Zone.  Therefore, to the extent that robot umpires will reduce the tendency of hitters to expand their hitting zone, more hits should occur.

The last word

Home plate umpires miss many ball-strike calls. Not only does it enrage fans but also players such as Kyle Schwarber. However, the impact on runs and game results is negligible. For the Toronto Blue Jays, incorrect ball-strike calls are unlikely the reason they missed the 2021 playoffs by one game. Yet, there are reasons to use robot umpires in the future. The reduction of missed ball-strike calls should lead to more hits and runs. Hopefully, the automated ball-strike calls technology can be developed so that MLB can feasibly implement robot umpires.





*Featured Image Courtesy Of DaveMe Images. Prints Available For Purchase.






Bob Ritchie

Bob was a St. Louis Cardinals fan until the Blue Jays arrived on the baseball scene, although he still has a soft spot for the Cards. Similar to straddling the Greenwich Meridian, as depicted in the avatar, Bob applies sabermetrics when applicable, but his heart tells him that Lou Brock belongs in the Hall of Fame.