Football Analytics Pros and Cons

[Thinking Skins]

So I was in the Haskins thread talking about QBR and I've been reading on it a lot. So I thought I'd share some thoughts on some of the different analytics. 

 

First there's the passer rating formula. Basically this was the only metric for a long time and its still used by many to grade passers. But what this boils down to is a look at four main factors, completion percentage, yards, TDs and Interceptions. The good thing is that those are the four things that people discuss the most when discussing a QB so its somewhat simple in that light. But with scores going up to 158.3, what's a good result? For a while it was above 70, then above 80, now I think its above 90 and that's ever changing, especially as teams favor short passes as an extension of the running game more. 

 

Total QBR. This was brought cooked by ESPN as a competitor to the passer rating. I love Brian Burke's answer where he laughs about it not being perfect but basically says look at what was before it. Which I understand and kinda agree with. Total QBR takes a lot more into play, which according to Burke is also things like how much time is left in the game (garbage stats). But the reports on this (including Burke's own response) say that its basically a look at all the plays of the QB (throws, runs, sacks, fumbles, interceptions, etc) and looks at two main things: The expected points before the play and the expected points after the play and the question is how much did the single play change the EPA. Total up a QBs plays over a game and you can calculate it for that game. 

https://www.quora.com/Is-QBR-a-good-statistic-to-measure-a-quarterback

 

Yards per attempt (and adjusted yards per attempt). These two are simpler than the ones above and say we can determine a lot by how good a QB is by simply looking at the passing yards gained and dividing it by the number of total passes (or looking at the net yards passing). 

 

Air yards: In judging a QB, one of the key things is to separate the play of the QB from that of the WR. Of the analytics above, only Total QBR claims to attempt to do this, but it is not clear as to how it is done because they make no mention of using air yards in their calculations. But air yards is just how it sounds, it is the number of yards a QB threw the ball to the player in question. 

https://www.quora.com/What-does-the-NFL-statistic-Air-Yards-mean-What-is-its-significance

 

So I wanted to first post these analytics. I know in some of the Stadium posts we grade QBs using these analytics (and possibly others), I wanted to create a separate space to discuss them. If you know of any more, or if you feel I have improperly described one or if you'd like to add some clarity to the description or talk about some of the pros and cons of these analytics, that's what I'm here for. 

 

In some later posts I want to talk about my thoughts on some of these and why none is the universal 'best'. 

 

 

[Spaceman Spiff]

7 minutes ago, Thinking Skins said:

So I was in the Haskins thread talking about QBR and I've been reading on it a lot. So I thought I'd share some thoughts on some of the different analytics. 

 

First there's the passer rating formula. Basically this was the only metric for a long time and its still used by many to grade passers. But what this boils down to is a look at four main factors, completion percentage, yards, TDs and Interceptions. The good thing is that those are the four things that people discuss the most when discussing a QB so its somewhat simple in that light. But with scores going up to 158.3, what's a good result? For a while it was above 70, then above 80, now I think its above 90 and that's ever changing, especially as teams favor short passes as an extension of the running game more. 

 

 

 

 

I think comparing your QB to whatever is essentially "league average" is what's important when judging stats.  If you're looking at a passer rating and your QB is 90, that might not be all that impressive but if he league average is 75, then your guy is better than most.  

 

In regards to QBR, I kinda get it but I think ESPN was trying to throw their hat into the analytics ring and make themselves appear to be credible in an area outside of highlights, talking heads and documentaries.  They just wanted to be able to add to the conversation and since baseball analytics are oversaturated, they decided to focus on football.  

 

I'm not sure if QBR still has this problem but I do remember a game where Tim Tebow came on in relief and posted a better QBR than another QB (I think it was Aaron Rodgers) who threw for something like 400 yards and multiple touchdowns.  Maybe they've adjusted it since then.

 

I'd agree that there's no one stat that's perfect to grade quarterbacks, but I'm old school and I just like the good old quarterback rating and YPA.  

 

[Fergasun]

Football Outsiders...they use success rate, and have adjustments for opponent strength. It's not perfect, but I have trusted their methodology. 

 

Haskins and Allen have been poor (Haskins worse) per their metrics. 

[Spaceman Spiff]

Also, big fan of completion percentage.  Maybe not from a game-to-game situation as a QB can complete 55% one week and then 65% the next...but over the course of multiple seasons, it's a telling statistic, IMO.

[Thinking Skins]

1 hour ago, Spaceman Spiff said:

I think comparing your QB to whatever is essentially "league average" is what's important when judging stats.  If you're looking at a passer rating and your QB is 90, that might not be all that impressive but if he league average is 75, then your guy is better than most.  

 

That's great but the thing becomes the whatabouts. Like for example the problem with passer rating historically was that when you had a guy like Steve Young or Randall Cunningham, they may not show up as well in something like that but they were impacting the game in other ways that the average fan could see but the league nerds weren't into it yet because they were still convinced you couldn't win with a running QB. 

 

And then we have the emergence of things like the WCO and these short passing games and so we are seeing completion percentages above 60 and 75 percent when back in the 90s and early 2000s 60 was considered top of the league. So the question becomes is your guy Brunelling it and just throwing dinks and dimes. Now, if that works then great but there's an argument to be made to say that if you can't make all the throws you can't be the best. 

1 hour ago, Fergasun said:

Football Outsiders...they use success rate, and have adjustments for opponent strength. It's not perfect, but I have trusted their methodology. 

 

Haskins and Allen have been poor (Haskins worse) per their metrics. 

Cool, Have you seen a writeup of their work? I don't subscribe to them and visit the site only occasionally when I'm linked to it from here. 

 

I'm trying not to make this a Haskins / Allen thread (hence why I put this in tailgate). 

[Thinking Skins]

One thing I wanted to say about QBR, and I told a friend I may try to replicate it. I saw someone post on reddit that they were able to pretty much replicate it by doing EPA after - EPA before after each play and dividing that by the number of total plays. 

 

But here's my problem with EPA based metrics. Lets say we have two QBs. QB1 and QB2. 

QB1 drops back and throws a pass deep and its picked off and he's tackled inside the 5. 

QB2 drops back and throws a pass across the middle and its picked off and its returned for a TD.

How does QBR grade these two plays? If I'm using ProFootballReference's EPA (you can see it on a play by play in the boxscore), at returned TD is -7 EPA (7 for the other team). And pinning them inside their own 5 is much metter EPA (they're unlikely to score on the next play from inside their own 5). 

 

But in my opinion both these two plays should be graded similarly. Maybe there's some difference for factors like "catchable ball" or "right read" or even that the deep ball was lower risk because he knew an int would likely pin them (kinda the Hail Mary pics that we don't mind). But the int returned for a TD isn't the QBs fault. Maybe we can put up an argument that it is, but just like there's a movement that YAC should count different than air yards, the yards after the interception should probably be different from the interception itself. 

[Spaceman Spiff]

22 minutes ago, Thinking Skins said:

That's great but the thing becomes the whatabouts. Like for example the problem with passer rating historically was that when you had a guy like Steve Young or Randall Cunningham, they may not show up as well in something like that but they were impacting the game in other ways that the average fan could see but the league nerds weren't into it yet because they were still convinced you couldn't win with a running QB. 

 

And then we have the emergence of things like the WCO and these short passing games and so we are seeing completion percentages above 60 and 75 percent when back in the 90s and early 2000s 60 was considered top of the league. So the question becomes is your guy Brunelling it and just throwing dinks and dimes. Now, if that works then great but there's an argument to be made to say that if you can't make all the throws you can't be the best. 

Cool, Have you seen a writeup of their work? I don't subscribe to them and visit the site only occasionally when I'm linked to it from here. 

 

 

There's always gonna be whatabouts.  But even if you're ranking QB by QBR, you should still look at what is league average performance and go from there.  But I think if you want to evaluate QBs these days without QBR you know which guys are runners and which guys are statues in the pocket and adjust accordingly by taking the running into account.  

 

Odd that you picked Steve Young, he was the all time leader in rating when he retired, pretty sure he's still top 10.  

 

 

[Thinking Skins]

23 minutes ago, Spaceman Spiff said:

Odd that you picked Steve Young, he was the all time leader in rating when he retired, pretty sure he's still top 10.  

 

I picked Steve Young because I remember when he was a young pup (he's older than me) but he was being criticized for being undisciplined and so freelance. And so even though he had good things like a good passer rating, he was always criticized for running and it was never included in the discussions about how great he was, except as an add on. Like yeah he has this great rating and oh yeah he once ran for 400 yards in a season. 

 

But I bring up these things because we have metrics that are including a lot of things but there are a lot of things that are not being included in these metrics that maybe should.

 

Like what about QBs with a quick delivery? That has an opportunity to help an OL and beat a blitz. But its also possible for a QB to rush their reads and throw the checkdown too soon.

What about moving the pocket. That's something that Russell Wilson has done very well since 2012 and kinda has put him at the top of these running QBs because he doesn't look to run but to create an open lane.

What about looking off defenders? 

What about throwing certain type of passers (jump balls, or fade routes)? 

 

What about beating different type of defenses? 

 

There are a lot of things, and especially when we look at certain QBs, certain areas where they may be expert and certain areas where they may be remedial. One thing is to just look at these different question areas one by one but as analytics evolve  the question becomes will they evolve to include some of these more advanced things?

[China]

2 hours ago, Spaceman Spiff said:

Also, big fan of completion percentage.  Maybe not from a game-to-game situation as a QB can complete 55% one week and then 65% the next...but over the course of multiple seasons, it's a telling statistic, IMO.

 

Fun fact:  In the 1991 Superbowl season Mark Rypien had a completion percentage of 59.1%,  The league average has been higher that that every year since 2002.

[Kosher Ham]

Percentage is also situational. 

 

Rather throw it away than take the sack or potential turnover. 

 

Skews the numbers when you have a pocket passer and/ or poor offensive line. 

[Thinking Skins]

Something i was thinking about last night: How much does EPA change over time? Like we've seen completion percentage steadily increasing over time, does that or the simple fact that athletes are becoming bigger stronger and faster change the EPA on a given down and distance and field position? 

 

Taking it way back but before some of the offensive gurus came out and revolutionized the game (from a coaching standpoint) there were some things that were unthinkable, so I could imagine that the EPA would be very low for scoring from certain field positions. Fast forward to the running QB and you have another element to consider. Fast forward to some of these running QBs who have a deep ball or accuracy (and particularly the ones who run to throw) and you've probably got a changed EPA. 

 

So would a Mahomes or a Favre change the EPA the way I'd think an Agent Zero or a Curry changed the likelihood for a person to shoot the ball from all over the court? 

[ClaytoAli]

I thought you meant Analytics for this type of pros and cons

 

[London Kev]

On 10/21/2020 at 12:04 PM, Thinking Skins said:

Something i was thinking about last night: How much does EPA change over time? Like we've seen completion percentage steadily increasing over time, does that or the simple fact that athletes are becoming bigger stronger and faster change the EPA on a given down and distance and field position? 

 

Taking it way back but before some of the offensive gurus came out and revolutionized the game (from a coaching standpoint) there were some things that were unthinkable, so I could imagine that the EPA would be very low for scoring from certain field positions. Fast forward to the running QB and you have another element to consider. Fast forward to some of these running QBs who have a deep ball or accuracy (and particularly the ones who run to throw) and you've probably got a changed EPA. 

 

So would a Mahomes or a Favre change the EPA the way I'd think an Agent Zero or a Curry changed the likelihood for a person to shoot the ball from all over the court? 

 

I can't really add anything constructive to the conversation except for a few musings.

 

I think that rule changes have had a big effect on QB stats, which in turn has enabled the offensive gurus to do their thing.

Also, and I don't know if there are stats that account for this, but some QBs appear to have an inherent ability to escape the pocket when it looks like they are virtually in the grasp. Players like Lamar Jackson, Murray, Wentz, Prescott, etc. They seem to have an extra sense in the pocket and are just so slippery.

[Redskins Reparations]

I am sure someone someone has created a next-gen quarterback algorithm that provides statistics along with situational analytics that cross references at what point of the game the player statistics were achieved.  Did the player produce the best when there was a blowout and the opposing team was playing a soft zone or did the player perform and a higher efficiency level in tight game?

[Thinking Skins]

13 minutes ago, Redskins Reparations said:

I am sure someone someone has created a next-gen quarterback algorithm that provides statistics along with situational analytics that cross references at what point of the game the player statistics were achieved.  Did the player produce the best when there was a blowout and the opposing team was playing a soft zone or did the player perform and a higher efficiency level in tight game?

The situational QBR is what I'd vote for in this realm. For instance if you go to pro football reference and view a QB's page you can see the advanced passing stats and situational stuff. But right now it doesn't give QBR (I guess because that's a protected formula) but if it can be estimaged by Expected Points, then we could look at something similar to situational QBR for different QBs in different situations. 

 

But even these stats don't keep track of things like ball placement or scrambled left/right, or made a guy miss, etc. 

 

One of the things I wanted to do was look into analyzing things like a twitter feed for a game (normally each game has a given hashtag) and trying to correlate each tweet to a given play. That forms a more informational play by play than just the pfr or other sites, and can give more context to the play like "shoulda been intercepted" or "that was a beautiful pass".