There has been a lot of discussion across many threads about the 4th down decision Staley made late in the Vikings game. Here is a video of the play: jump to 14:10.

The Chargers were unsuccessful on the 4th down attempt, but won the game, anyway, which makes this a safe play to talk about here.

Analytics Models

So there are a number of organizations that will tweet out their assessment of 4th down decisions, but Ben Baldwin actually created this 4th down calculator. Here is a snapshot of what it showed prior to this play:

We can obviously ignore the FGA option, which is correctly shown with a success percentage of 0%.

That leaves two choices. The model says the win probability is 89% if the Chargers go for it and 80% if they punt. But the model lacks context, so let's talk about that.

Model Limitations

Baldwin himself said this about this model:

Win probability numbers are not handed down by computer overlords on stone tablets. A human has to gather data, figure out what it means, make a bunch of judgment calls along the way and then write a piece of software that makes a final recommendation. Coaches then need to make further judgment calls based on weather, injuries and other factors that computers can’t know about ahead of time. The unhelpful way to use this decision tool is to dunk on a coach who made a decision that the model says is "wrong" by something like 1 percentage point — there are too many model choices that can swing things by small amounts to treat this as an exact science. However, when the model makes a strong recommendation, we should probably listen.

Baldwin's model uses data from nflfastR, which is based on play by play data going back to 1999. This means the model:

• Assumes league average offense, defense, and special teams from that entire period
• Has a blended dataset for all weather, venues, opponents, teammates, coaches
• May not fully reflect recent trends in the league if/when they have divererged from the entire 24 year dataset
• Does not reflect specific injuries, coaching, players, or game performance

Also, the calculator referenced above has some fine print that I didn't capture in my snapshot. Here it is:

1. "These are ESTIMATES; please use accordingly."
2. "On 4th and 1, the model cannot know whether it's a long 1 or a short 1. Shorter distance would favor going."
3. "Do not use this in overtime."
4. "Use with EXTREME CAUTION in the final minute of a game as the model is not good with end of game clock scenarios."

Regarding #2 here, this 2019 article says the following:

According to the Wall Street Journal, teams going for it on fourth-and-inches convert 82 percent of the time over the past two seasons. Long fourth-and-1s are converted just 55 percent of the time.

On the videos I have seen, it appears the Chargers needed about 2 feet, so about 2/3 of a yard. Based on this data, this implies a generic conversion rate of perhaps 65% or so.

Regarding #4, it was not within the final minute of the game, but it was close. If the model should be used with EXTREME CAUTION in the final minute, I am left to wonder if it should be used with some caution (not EXTREME CAUTION!) in the next to final minute...

Chargers-Vikings Game Scenario

Okay, let's turn to things that could have influenced the decision by Staley.

The model assumes average offense and defense. But:

• The Vikings have a great offense, particularly a great passing offense.
• The Chargers have a bad defense, which frequently gives up explosive passing plays. The defense was also missing Derwin James for the rest of the game at this point.

This suggests that the model underestimates the Vikings' win probability in both scenarios where they get the ball (Chargers punt, Chargers go for it and fail). If true, this changes the math and could change the model's recommendation, or at least the margin of the recommendation.

The model said the Chargers had an 89% win probability by going for it. That is based on the model expecting the Chargers had a 73% chance of converting the 4th down.

According to PFR, in 37 games under Staley before this game, the Chargers had 41 4th and 1 plays. Decisions/outcomes:

• Punt - 11
• Attempt FG - 2
• Go for it - 28
  • 19 conversions (~68%), including one by defensive penalty
  • 9 failures to convert

Obviously, there are differences between those plays and the play in question -- different teammates, OC, opponents, venues, weather, game situations, etc.

That success percentage is closer than I expected to Baldwin's model's assumption of a 73% success rate. I would intuitively adjust that number down somewhat given:

1. Neither Ekeler nor Mike Williams were available for the play in question. In particular, Ekeler had 6 of those previous conversions.
2. The Chargers had struggled to run the ball to that point in the game. In particular, Kelley had 10 carries for 12 yards up to that point, with 4 of the carries (40%) going for 0 or negative yards. He had one "and 1" carry on a 3rd and 1 earlier in the game, and gained 0 yards. This data does not seem to support a 73% chance of converting by running Kelley.

I get that my #2 here is a rub for many fans, who think the team should have gone for it, but didn't like the play call. Here is a further breakdown that supports the notion that the play call was not ideal. In the 28 times the Chargers had gone for it, the plays broke down like this:

• 8 passes by Herbert that resulted in 5 completions, 2 TDs, and 3 additional first downs
• 8 rushing attempts by Herbert that resulted in 7 first downs -- can't tell from the play data if all were sneaks or if any were other types of plays
• 10 rushing attempts by players other than Herbert that resulted in 6 first downs
• 1 first down gained by defensive penalty -- not sure what the play call was, but the penalty was unnecessary roughness, and it resulted in exactly +15 yards from the previous line of scrimmage, which suggests that the play gained 0 or negative yards
• 1 false start by Feiler -- don't know what the play call was

So, ignoring the penalties, the team was successful on 75% of the plays when they put the ball in Herbert's hands and 60% of the plays when they didn't.

For these reasons, I would adjust the success rate of going for it to 60%. I admit this is a gut feel, not scientific.

Here is some additional info from Popper about the play call:

The play call was a dive handoff to Kelley. Going a bit deeper, it was the exact same formation and design as the Chargers’ fourth-down conversion pitch to Derius Davis in the second quarter against the Tennessee Titans last week. The only difference in personnel was Allen was in the game for Williams, who had gone to the locker room with a knee injury. (Williams was carted to the bus after the game.) This time, instead of faking the handoff to Kelley and pitching to Davis, Herbert handed off to Kelley. It was stuffed.

There was intention in the play call. Offensive coordinator Kellen Moore was playing directly off a call and design from the previous week. And the Vikings had an edge rusher who jumped to the outside to defend Davis on the play. They had clearly studied the Week 2 fourth down play on tape.

The Chargers just did not win at the point of attack. Defensive lineman Jonathan Bullard got penetration between right tackle Trey Pipkins III and tight end Donald Parham Jr., blowing up the play.

So the Chargers ran a 4th down play they had put on film the previous game. Yes, they attempted a different option within that play, but (a) an option -- running Kelley into the line -- that had not worked well in the game and (b) clearly a play the Vikings had studied on film... does anyone think that studying the film is limited to the choice made on the play in the previous game, or do we agree that the Vikings would understand that Kelley could get the ball out of that play? IMO this seems like a poor decision. Not sure if this is solely on Moore or also on Staley.

Did Staley know the play call when he made the decision to go for it? In his press conference after the game, he refused to second guess the play, but that doesn't mean much... that could mean he knew or it could mean he simply didn't want to criticize Moore. For all we know, the Chargers game planned this during the week, planning to use the same play but hand off to Kelley instead of pitching. We can't know this, but knowing would definitely influence my thinking about this.

Also from Popper:

Staley said the Chargers considered a QB sneak. But the Vikings had four interior defensive linemen in the game.

"My job was just to hand the ball off," Herbert said.

This seems to undermine the perspective I have seen from many fans that the Chargers should have called a sneak or used the "tush push." Like us fans, the Vikings recognized that could be the play call, and they were prepared for it.

Last thing from Popper:

They could have put the ball in Herbert’s hands. They could have instead run behind the strength of their offensive line to the left side with Rashawn Slater, Zion Johnson and Corey Linsley.

They did not.

From Comments: "Does this Defense play better from a confined field"

My good buddy Alister posted this in the comments, and I have seen similar comments in other threads discussing this play:

I have no idea if this even factored into the decision matrix - probably not - but could it be argued that the 2023 Chargers D is more likely to concede a TD when the Vikings have an entire field to operate with, vs more confined spaces where some of our catch point specialists (Asante, Davis, and who would have thought Kenneth Murray/Nick Niemann) can focus less on coverage rules and all the playcalls an Offense could throw at you, and more on making plays on the ball from a presumably narrower base of conceivable offensive plays? In isolation I think the win probability would be higher if you asked the Vikings to march the entire field, but it’s something to think about

I disagree with this theory for a few reasons.

1. Prior to the decision we are discussing, the Vikings had 10 drives in this game. One was at the end of the first half, so perhaps it is fair to ignore that one. In the other 9 drives, they had 3 TD drives (33%). That is less than the win probability of the Vikings taking over on the 24 yard line.
2. The most immediate Vikings drive prior to the decision we are discussing ended in a turnover on downs. It was a 13 play drive that took 5:12 off the clock and ended when the Vikings could not convert a 4th and 2 at the LAC 2 yard line.
3. The Chargers have allowed 10 TDs to opposing offenses. 6 of them have come from inside the 5 yard line, and, despite the Chargers' propensity for allowing explosive plays, only 3 have come outside the red zone.
4. Ignoring penalties, the Vikings averaged 6.1 yards per play and 6.4 yards per pass play. This despite the explosive passes the Chargers allowed. Odds are they would have had to string together several positive plays to ultimately have a game winning TD drive.

IMO it would have been very unlikely the Vikings could have scored a TD after a typical punt scenario. I guess I have to trust the model baseline, but 20% probability of winning with 1:45 or so remaining and no timeouts seems high.

From Comments: "JK Scott who hadn’t had a great game to that point"

@FiatBux posted in the comments:

But my second thought has to do with JK Scott who hadn’t had a great game to that point. He’d averaged only 37 yards per punt, and while one of those (IIRC) was a bad bounce that went 15-20 yards in the Vikings’ direction. But a 40-yardish punt from the 24 would’ve put Minnesota around their own 35 with approximately 1:40 left in the game. I don’t know if, with the defense’s propensity to give up 20-yard passes, the Vikings aren’t at the Chargers’ 20 with 30 seconds left, but I have my suspicions.

Scott had punts of 27, 33, and 52 yards in the game. The 27 yarder is the one that bounced back about 10 yards before the Chargers could down it. The 33 yarder was fielded at the MIN 25 and downed there. His last punt before the decision in question was 52 yards, with the Minnesota returner fair catching at the MIN 17 yard line.

I think the 27 yarder was a fluke, so I discount that. Anywhere between 33 and 52 net yards would have been worth it IMO.

Revised Probabilities

With all that said, suppose the model probabilties were as follows:

• Chargers success percentage if going for it: 60%
• Vikings win percentage if Chargers punted: 22%
  • Increased by 10% over the model due to strength of Vikings offense and weakness of Chargers defense
• Vikings win percentage if Chargers went for it and failed to convert: 49.2%
  • Increased by 20% due to strength of Vikings offense and weakness of Chargers defense
  • Increased more than the punt scenario due to needing only 24 yards

If we do the math on that:

• Win probability if Chargers go for it: 79.7%
• Win probability if Chargers punt: 78%

This is not nearly as substantial as Baldwin's model projected. IMO it is narrow enough that is is within the noise of the model. This goes back to part of the Ben Baldwin quote I posted above:

...there are too many model choices that can swing things by small amounts to treat this as an exact science.

IMO this supports my view they should have punted, because IMO if the decision is that close, the right choice is the conservative choice. The conservative choice is to punt.

Conclusion

In real-time, I thought it was a bad decision before knowing the conversion would fail. I posted after the fact that I was shocked by the decision, and I was. Another poster and I got into a spat (maybe I had had a few beers by that time...) because he said I shouldn't have been shocked. But I honestly was.

Once it failed, I thought it was a virtual certainty the Chargers would lose. I'm obviously glad they held on.

Looking at this data doesn't really convince me the decision was right, but rather makes me think it was effectively a toss-up. I think in this game situation with a toss-up, I would punt.

And, having experienced this particular situation, I feel even more strongly about that for the next time this situation occurs and context is similar.

I found this discussion in the various threads very interesting, so hoping for some more interesting discussion here. I probably botched some of my thinking or overlooked some important factors; if so, hoping others will point out those things.

Thoughts?