Is there a relationship between passing volume and efficiency?

Arguing on the internet

A common argument on the internet (e.g. Twitter, where I spent too much time) is that the efficiency of players like Prescott and Wilson in their rookie seasons (and subsequent seasons, for Wilson) was not impressive because they were not asked to throw the ball as much. Once they are asked to throw more often, the argument goes, we can expect their efficiency to fall off. Here is one of many, many examples:

Do QBs really look good because they throw less?

We have evidence that in the NBA, players on a given team distribute shots among themselves in a manner that the marginal value of the next shot is roughly equal among players. This results in good players generally taking more shots than bad players in order to maximize the scoring of the team. Further increasing a given player’s volume is typically associated with a decrease in efficiency. This sort of logic is probably what people are leaning on when they assume that a QB’s volume increasing should cause a drop in efficiency. However, this doesn’t really make sense because each team only has one QB on the field at a given time: in the NBA, defenses can focus their attention on one player who shoots a lot, but in the NFL, there’s only one QB. If a team is going to pass, the defense knows who is going to be throwing the pass (trick plays aside). Another argument is that teams facing an opponent who passes a lot can focus their attention on the passing game rather than the rushing game, but is this really correlated with meaningful changes in efficiency? Let’s take a look.

Boring background stuff

  • I took every quarterback season in which the QB started at least 9 games from the 2002 season through the 2016 season. This results in 436 distinct seasons played by QBs.
  • I measure efficiency using yards per attempt, which has the benefits of both being very simple and doing about as well at out-of-sample predictions of team success as other metrics. Results are similar using passer rating or other efficiency stats. The mean Y/A season in the sample is 7.11.

Findings

Here is a basic scatterplot of the data. Again, each point represents a season played by a QB.

That looks like a giant blob with no real relationship, but already we can see that it is certainly not the case that the simple act of passing at low volume is sufficient to carry a QB to strong efficiency stats.

Next, here are all of the seasons played by QBs with at least 10 seasons in my data. I’ve added player labels to give a sense of what some individual seasons look like:

Again, it is not the case that the seasons with the fewest attempts are the seasons with the best efficiency.

For the remainder, I’m going to be looking at regression output. If this doesn’t sound interesting, skip to the end.

The first regression is a simple regression of a player’s yards per attempt on the average number of passes thrown per game and year effects (to account for efficiency rising over time):

The coefficient of .014 tells us that every 10 additional passes a player throws is correlated with an increase of .14 yards per attempt. In this simple look at the data, as with the scatterplot above, there is no evidence that increased volume is associated with a decrease in efficiency.

Perhaps the small, positive relationship between passes per game and yards per attempt found above is due to more experienced QBs throwing more and being more efficient. When I add controls for quarterback experience as well, the coefficient shrinks to essentially zero (-.004). Finally, it could be the case that a true negative relationship between volume and efficiency is masked by good quarterbacks being the ones who throw a lot and who are more efficient.

For the last exercise, I take the set of players who have played at least 10 seasons (Ben, Palmer, Brees, the Mannings, Rivers, Brady) and run a player fixed effects regression, which essentially compares each player to himself in his high volume versus low volume seasons. The point estimate becomes -0.029, which is small and not statistically significant from zero (the standard errors for the regression coefficient grow larger here because of the limited number of players under consideration). The way I would describe this result is that among the set of quarterbacks who have been in the league a long time, they have beenĀ slightly less efficient in the seasons in which they threw more.

Summary

In a sample of QB seasons in the past 15 years, there is a small, positive association between volume and efficiency. This may be driven in part by skilled quarterbacks throwing more (on average) and being more efficient (on average). When looking at QBs with long careers, there is little evidence that they were meaningfully less efficient in the years they were asked to pass a lot relative to the years they were not. Comparing Brees in 2004 vs Brees in 2013 is a nice illustration — his passer rating was nearly identical despite a massive difference in attempts/game.

To sum up, looking at the last 15 seasons in the NFL reveals no statistical relationship between a QB’s volume and efficiency.

  • Adam

    Excellent article, Ben. I’ve always believed that a high volume of attempts depletes efficiency, but the data you’ve presented takes the air out of that argument! That said, even if pass attempts in a single game have no correlation with efficiency, I do think it’s harder to sustain greatness over a full season compared to a partial season. This is due to regression toward the mean, not because passing more often makes passing harder. As a QBs’ number of attempts climbs throughout a season, his efficiency will gradually regress toward his true talent level. Regression obviously has a stronger effect on 600 attempts than 300, which is why I believe volume still matters at the season level.

    • Ben B

      These are results over a full season. For example, on the graph above, the dot for Peyton Manning 2004 means that he threw for 9.2 yards per attempt and 31.1 passes per game (or, over the entire season, 497 passes).

      • Adam

        Do you think the correlation would look any different at the game level? It sure seems like QB games with 50+ attempts are less efficient on average than those with 30 attempts.

        • Doesn’t there seem to be a ton of competing factors at play? Like some QBs throw a lot in a game because they are trailing, and they are trailing because they are playing inefficiently. So volume is causing ineffective play; ineffective play is creating the need for more volume.

          On the other hand, some QBs play poorly out of the gate and either get pulled or cause the staff to change the play splits. Those two extreme examples could cancel each other out. I haven’t looked at the data that granularity yet; I’m just throwing stuff out there.

          Then we have the efficient guys. I’ve seen Brady play efficiently and have crazy volume that was required based either on poor rushing or opponent weakness. It’s not just him, but he does seem to be the king of winning when the team rushes ten times.

          I’ve also seen insanely efficient performances that were so good they basically made the need for further volume obsolete that day. Brees versus the Pats in 2009 comes to mind.

          Related: if we were to accept that low volume lends itself to higher efficiency, we must also accept the inverse, that low volume also lends itself to poor efficiency.

          Put more simply, low volume lends itself to extremes because stabilization hasn’t happened yet.

          • Adam

            “Related: if we were to accept that low volume lends itself to higher efficiency, we must also accept the inverse, that low volume also lends itself to poor efficiency.”

            I think this is the key. We see some wacky Y/A numbers (high and low) in 20 attempt games, but almost never from 50 attempt games. This goes along with my original comment about high attempt seasons regressing toward the mean. To use Brady as an example, the vast majority of his high attempt games produced a Y/A between 6 and 8. If he was ineffective, Bill wouldn’t let him throw so much; if he was extremely effective, his team would have a big lead and call runs to kill the clock.

            Basically I agree that several factors pull the correlation in different directions, and the net result is no correlation.

        • Ben B

          Yep, like Bryan said, I would expect game scripts to play a bigger role when looking at the game level. We know that QBs are less efficient when trailing, at least as measured by passer rating, because they have to take more risks (you can see this in things like the splits page on PFR), so if QBs throw more when trailing we would expect them to be less efficient in those games.

  • These are interesting findings, Ben. I hope you’ll be patient if there are a few points I’m not sure I understood.

    My concern relates to confusion between correlation and causation. I don’t doubt the correlations you’ve identified, but I’m skeptical on the causation front. I struggle to imagine a compelling explanation for why throwing more passes might increase efficiency, but there are very obvious reasons that efficient passers would throw more often.

    It seems problematic to compare Alex Smith to Drew Brees, or even ’04 Brees to ’13 Brees, though the latter comparison is obviously more telling than the former. Again, perhaps there’s something I’m missing or misunderstanding, but I just don’t think we have enough data for the kind of conclusions you’ve drawn. Our usable samples are too small and there are way too many variables.

    When you compare ’04 Brees to ’13 Brees, you’re comparing Brees’ first half-decent season to his prime. You’re comparing him playing for the San Diego Chargers to the New Orleans Saints. His coaches are different, his blockers are different, his receivers are different, his home stadium is different. The rules have changed: defenseless receiver, Tom Brady rule, etc. It’s apples to oranges; we simply can’t draw a confident conclusion from that comparison.

    Even shorter time differences can be problematic. Take Tom Brady. From 2002-06, he played on a team with dominant defense and lacking Pro Bowl receivers; he averaged 530 attempts per season. In ’07, he got Randy Moss and Wes Welker and started passing more, with great success. I don’t think that demonstrates anything meaningful about the correlation between attempts and efficiency. Shifting rosters, injuries, the efficiency of the run game, and other variables make this very difficult to measure. I applaud your ambition, but it seems to me that your counter-intuitive findings reflect these difficulties rather than implying the conclusion you’ve drawn.

    Unless there’s something I’ve missed, I remain confident that efficient passing causes more attempts, rather than the reverse.

    • Ben B

      Thanks for reading and posting.

      Yes, these are correlations, and yes, sample size will always be an issue with studying the NFL due to only having 16 games per season. However, my preference is to look at the data and see what it tells us rather than ignoring it altogether.

      People seem sure that it is easier to be efficient when passing at low volume (hypothesis). If that is the case, then we can look at QB seasons (data) and see whether there is any support for this hypothesis that low volume lends itself to being efficient (testable prediction). When we take the hypothesis to the data, we see that this prediction does not hold. As you noted, there are many reasons why this might be happening, but it is far from obvious *in the data* that there is any link.

      • I agree with your final clause: “it is far from obvious *in the data* that there is any link” between higher attempts and lower efficiency. I disagree, though, with any conclusion implying that absence of evidence is evidence of absence.

        Take a simple example. Teams throw more often almost every season. Attempts have risen steadily from 2002-16: 36.1, 34.3, 34.3, 34.5, 34.3, 35.4, 34.3, 35.4,35.9, 36.3, 37.0, 38.0, 37.3, 38.1, 37.9. From 2002-09, teams consistently averaged 34-35.5 att/gm; since 2010, it’s never been lower than 35.9, and since 2012, it’s never been lower than 37.

        Efficiency has risen correspondingly: 6.3, 6.2, 6.6, 6.3, 6.4, 6.4, 6.5, 6.6, 6.6, 6.8, 6.7, 6.7, 6.8, 6.8, 6.8. This could be interpreted as supporting your conclusion: teams are passing more often, so they’re passing more efficiently. A more plausible explanation, though, would cite rule changes that favor passing, the growth of pro-style college offenses, and other environmental factors.

        There are so many variables that could distort your data set. I don’t think we can or should draw any conclusion whatsoever from the findings here. There just isn’t enough evidence here to refute a link between higher attempts and lower efficiency.

        • Adam

          I agree with your conclusion, but can’t figure out which measure of efficiency you’re referencing. The cited numbers are too low for Y/A but too high for NY/A.

          • Argh, I think I used net yards divided by attempts (rather than att + sacks). Thanks for catching that.

            The NY/A figures are: 5.9, 5.8, 6.1, 5.9, 6.0, 6.0, 6.2, 6.2, 6.2, 6.3, 6.2, 6.2, 6.4, 6.4, 6.4. Regular Y/A follows the same general pattern.

          • Adam

            Cool, makes sense!

            In past articles you’ve opined that rate stats are worthless without accounting for volume. Assuming this is true, how steep of a volume adjustment do you think is appropriate? As an example, if QB A averages 7 Y/A and QB B averages 8 Y/A, how much of a volume advantage would QB A need to have for you to consider the two performances equal?

          • That’s a tough question. Off the top of my head, I’d say that in today’s game you’d need at least 100 more attempts to make up for a one-yard difference in net average. A yard per play is a lot.

            By QB-TSP, a player with 3600 yards on 450 attempts (8.0 avg) would outscore a player with 4200 yards on 600 attempts (7.0 avg), 1575-1500. I’d say that more or less jives with my sense of things, though subjectively I might be inclined to even it out a little. Go much below 450 and you’re flirting with 2013 Nick Foles territory.

        • Arin Franz

          Besides having a star quarterback, you could also throw a lot because your team stinks (the last decade of Jaguars football) or scheme (I’m looking at you, Mike Martz). I think it would be interesting to extend this by integrating Chase Stuart’s work with Game Scripts and Identity. Using these to control for game state and scheme could provide some additional insight.