First, a story. Brian, who many of you know as DaBolts, is a writer here at BFTB and was the "Head Writer" until I took over. I only took over because he no longer had the time to do it. It had nothing to do with talent, because the man is far better than me at everything he's done here at this blog/community. He's also a gentleman and a scholar.
So, being gentlemanly, Brian asked if it would be okay if he could contact the Football Outsiders guys about getting a free book (he would then review it here and they'd get some marketing out of it). I told him it'd be fine. As much as I've always admired what their website puts out and what they can get their crazy computers to predict for Pro Football Prospectus, I've never been able to understand all of it. It's a subculture of a subculture. Brian, however, gets it. There's the scholarly part of our departed-leader-turned-BFTB-columnist.
Apparently, his idea was one that FO guys liked. Actually, LOVED. Next thing I knew, I was getting e-mails from SB Nation saying that the FO guys were going to be sending a copy of their book to ever SB Nation NFL blog and answering team-specific questions. Hot damn!
So after reading the section about the Chargers....I still don't really get it. Well, I mean, I get it. They're predicting the Chargers to be good. Great, actually. Their computers think that the Chargers are going to be better than the 16-0 Patriots. I'll take that. Their computers predict 12.5 wins for the Chargers and give them an 87% chance of winning 11 or more games.
Their computers also are under the assumption that healthy players stay healthy (I believe), and everybody knows that San Diego has the league's most talented team when healthy. With Merriman back and Tomlinson, Gates and Cromartie staying injury-free, I have a hard time saying that the Chargers won't find great success. Anyways, the Football Outsiders Almanac does a fantastic job of predicting (their track record is unbelievable), but makes sure they back up everything with reasoning and past examples. If you're into football stats at all, it's a must-buy.
The questions were written up by Brian, since he had less of a chance of embarrassing himself around the super-human brains of Football Outsiders. The answers are written by Aaron Schatz, the Editor-In-Chief of Football Outsiders and the person responsible for writing the essay in the San Diego Chargers section of this year's FOA.
1. In the section on the Chargers I was surprised you didn't take note of the rather dramatic difference between the first and second half performance. The DVOA for both the defense and offense were substantially lower. Do you attribute this to coaching, or do you think it is just statistical noise?
What dramatic difference? I'm not sure what week cutoff you are using to determine the change in DVOA -- or perhaps this is something that existed in the old version of DVOA but not the newer version that we use in the book. San Diego's offense in 2008 had 24.8% DVOA in Weeks 1-9 and 23.4% DVOA in Weeks 10-17, virtually the same. The defense has 9.9% DVOA in Weeks 1-9 and 6.6% DVOA in Weeks 10-17, slightly better, but not really by much.
2. In the beginning of the year much of the ratings are based on the prior years performance. This would seem to overstate the value of players who are perhaps no longer on the team and understate the value of high draft picks. Do you attempt to adjust early season ratings at all for those factors?
Absolutely. First of all, a lot of the statistical trends that we consider exist precisely because talent tends to move around in the offseason in particular ways. Young teams get older and more experienced. Bad teams get rid of their worst players, which makes them better. Teams that are awful on third down tend to specifically go out and look for players who will help them improve on third down, which is one of the reasons for the "third down rebound" effect. In addition, the team projection system includes a number of variables related to player movement. This year for the first time, the team projections include a "no team variables" quarterback projection that helps us measure the impact of a team improving its quarterback situation. This is one of the reasons why Chicago's projection is higher and Denver's projection is lower. We have variables that consider top players switching teams, particularly top pass rushers -- and this "add a top pass rusher" variable (a.k.a. Merriman returning) is one of the reasons the Chargers are projected to improve on defense. We have variables that consider the specific draft value on different positions, both in the most recent draft and in previous drafts (this represents the importance of talent maturing in certain positions after two or three years). And there are variables that consider offensive line continuity -- that's one of the big reasons the Buffalo projection is so bad.
3. In some cases teams are phenomenally successful on a measure that they don't perform very often. For instance, if the Raiders run most of the time and DVOA is average or low there; but they rarely throw and the DVOA is high; the temptation is to say they should throw more. But it seems as though the reason they are more successful is at least in part due to the fact that it surprises the opposing team. Is there any attempt to factor that into your analysis?
Two notes here:
1) We should point out that team DVOA is not the average of run DVOA and pass DVOA. It includes all plays equally, so if a team is very good passing because they rarely do it, their team DVOA will still be low because the majority of plays will be (poor) runs.
2) We may play around with adjusting things more in the future, but for the most part the way we deal with this problem is common sense. We try to analyze and interpret our stats, rather than just assuming that any team that is better in stat X is automatically more talented when it comes to play type X. As we often say, our stats are better than the conventional NFL stats, but they aren't perfect, and they aren't meant to replace our own knowledge of how player talents and schemes interact on the field. That's why the chapters in the book have essays, not just tables of numbers.
4. Even trained scientists have been known to bias their observations towards their preconceived notions. Who actually records these statistics and do you think they might contain some bias in them?
Well, we've got two different groups of statistics here.
First, you have your standard NFL play-by-play statistics. These are recorded by a group of official scorers at each game. The standard offensive stats have pretty much no bias. The defensive stats are "unofficial" and may be a little bit biased, but that's less because the scorers want to benefit their own home team and more because the definitions can be subjective -- who gets the tackle in a pile, for example, or what counts as a pass defensed. The official scorers have gotten better about this over the past couple years, thanks to the efforts of Chris Hoeltge at the league office, who has tried to get things better standardized. In particular, the Philly people aren't putting in a pass defensed any time the pass goes 10 yards over a DB's head, which is a plus. The "QB hit" stats are still registered somewhat willy-nilly from stadium to stadium, but the FO game charting project has actually helped that by reporting mistaken hits or missed hits, which are then changed in the official gameboks.
Second, you have the stats we keep at FO through the volunteers in the FO game charting project. That's the stuff like formations, pass rushers and blockers on each play, when there's play-action, who's the defender in coverage, and so forth. There's no question this stuff is biased. Every year, we work to help standardize definitions so that there's less variation between our game charters about what counts as what. No team is entirely charted by any one person, so things tend to even out -- for example, if the Chargers have a few halves charted by somebody who doesn't mark enough quarterback hurries, they probably also have a few halves charted by somebody who marks quarterback hurries a little too often. After the season's over, before we count things, I go through and try to make sure that no team has its stats particularly skewed by one charter who is extreme in the way he marks certain events.
Do you have any thoughts about Bolts From The Blue? Any tips you want to send our way? Whenever you have something to say, don't hesitate to e-mail me directly.