How Little We Know about Football
PigSkinRevolution.com
It’s Monday night and millions of Americans have tuned their televisions into the weekly drama on the gridiron. As the
coaches labor over difficult decisions, so too are the commentators and fans discussing the relative merits of every
important play choice. Bill Belichick is considering "going for it" on 4th and 1, while John Madden and Al Michaels
question his motives with great conviction. Madden draws on his extensive experience as an NFL head coach.
Michaels reflects on his years as a commentator and the numerous NFL games he has witnessed from the booth.
Meanwhile, Belichick is on his headset debating the risks and rewards of the decision with his staff. Depending on
the outcome of the game, and just how much this play is perceived to have impacted the final result, the fans and
media will also hold trial on the decision. With all of the conflicting testimony, confusion, it seems, is the best we can
hope for. The question begs to be asked: Could there possibly be some definitive method to objectively assess such
a difficult decision? The answer as it turns out, is a resounding yes.
Beginning in the late 1970s, computers began to infiltrate various gaming disciplines. Board games such as
backgammon and chess were the first programming experiments. Much to the skepticism, and often condescension,
of the human experts, the programmers strove forth to build the omniscient “machine”. The goal was not only to create
a tool for the human expert to grow their knowledge, but ultimately to surpass the level of the top human. In 2005, the
top backgammon and chess programs are now the undisputed masters of the gaming universe. How have we made
this technological advance in just a few short decades? It turns out that the most important ingredient is humility.
Indeed when we accept the fact that our knowledge is incomplete, the door for new and better strategies swings
open. The first necessary process in the success of the backgammon and chess programs was to strip away much of
what the human evolution of the game had taught us. Certainly human experts were on the right track with many of
their strategies. This was evidenced by the very success that classified them as experts in the first place. But could
some of the widely accepted behaviors have been more peer driven than logically based? As it turns out, this was
strongly the case. The early breakthrough models were fed only the basic rules of the game with very limited human
coaching. It was through millions of trials and mutations, whereby the machines remarkably learned from themselves.
Successful strategies were developed through an independent and greatly accelerated evolution. Optimum decisions
were subsequently determined as those that had the most success in the virtual arena. In other words, the best
decisions had the highest probability of winning after very large numbers of trials were conducted. The term “Game
Winning Chance Expectation” emerged, or as you will often hear us refer to it; “GWC”. Ultimately, some deeply
engrained human strategies were confirmed while others were overturned. Computer models were not only capable
of lightening quick calculations and data processing but they never required sleep or fell victim to emotional blunders.
With perfect consistency, these machines focused on the only metric that really matters….winning the game.
So what does all this have to do with football? After all, we are talking about pieces on a game board and not living,
breathing, emotional human athletes. Surprisingly, while the pieces of the game may look different, in the eyes of a
computer model they appear to be quite similar. NFL coaches are statistical junkies. For every player assessment or
play-calling situation they may consider an historical observation or statistical record. When Bill Belichick is
contemplating his 4th down decision, much of this data is dangling in his head, and the heads of his assistants. The
distinction of the successful model is in the methodology of calculation. Using identical inputs, a program can
streamline the data toward the goal of winning the game far more accurately and tirelessly than its carbon-based
counterpart. Binary play-calling (i.e. Punt vs. Go, Field Goal vs. Go, On-Side Kicks, penalty acceptance, P.A.T.s,
Timeouts) in the NFL is perfectly amenable to computer modeling. As these types of “Critical” decisions constitute 15-
20% of all play-calling choices in a typical NFL game, their importance can not be overstated.
EndGame Technologies began the development of the ZEUS™ model in 2001. The goal of the project was to
provide a coaching tool that could answer the really difficult problems reliably and quickly. Before the project became
a reality, a number of important questions had to be answered:
1) Could a model replicate, in a virtual environment, an NFL football game between two typical teams?
Yes, but not before several years of painstaking research into NFL statistics and the behavioral traits of coaches. A
ZEUS™ generated game log is indistinguishable from an actual NFL game log. More importantly, ZEUS™
generated seasonal and multi-year statistics closely mirror actual NFL statistics in virtually every significant category.
2) Could the model accurately consider the skill differentials of the opposing teams?
Yes. Phase 2 of the development process entailed an implementation of team customization. For all major offensive,
defensive and special teams’ categories, a custom setting allows the input of particular attributes of the subject team.
In turn, these custom settings manifest themselves in probabilistic output. Ultimately the relative strengths and
weaknesses of the opposing teams are driven by these inputs.
3) If the typical NFL team was consistently erring in binary play-calling, would the magnitude be significant?
This was perhaps the biggest surprise we encountered. According to ZEUS™ seasonal error rates for the 2003
season hovered in a fairly tight range between 0.65 and 1.25 games. This data suggests the typical NFL team is
falling nearly 1 game short of their potential due to sub-optimal performance in Critical Play-Calling alone. Secondly,
it suggests great parity among the thought processes of NFL coaches.
4) Would the outputs be statistically reliable?
This is the million dollar question. With all of the variables, both tangible and intangible, that factor into a Critical Call,
how can one have confidence in the ZEUS™ output? This turned out to be one of the most challenging and significant
aspects of the development process. By applying the rigors of a “sensitivity” analysis, the model attempts to overturn
its own recommendations. The probability that a recommendation can be overturned by applying an extreme custom
case, results in the assignment of a “Confidence Factor” (rated from a low of 1, to a high of 10). For instance, if
ZEUS™ suggests a 2-point conversion is superior to a 1-point conversion with a Confidence Factor of 10. This
means it would still be correct for an NFL-worst offense to make this choice against an NFL-best defense. By
applying this stress test, extraneous circumstances such as field conditions, momentum and wind can be adequately
considered and placed in their proper perspective.
5) Could the model be developed in a user-friendly and real-time application?
Absolutely. The model can be installed on a typical laptop computer in a matter of minutes and most analysis can be
input and executed within seconds.
ZEUS™ has already taught us extraordinary lessons about the game of football, but we have barely scratched the
surface. With the ability to observe the effects of minor adjustments to so many aspects of the game, the possibility
for new discoveries seems endless. We will be examining our favorite pastime with both a microscope and a wide
angle lens in the coming weeks. Please join us at pigskinrevolution.com for insightful commentaries and the
upcoming launch of the “Critical Call Index” (CCI). We look forward to seeing you there.