RESPONSES TO LETTERS (ESPN, DEC.6, 2006)
PigSkinRevolution.com
After the publication of our analysis of the Cowboys/Giants game on ESPN.com (“What Would ZEUS Do”) on
December 6th, we were inundated with letters from a wide variety of readers. Some wanted more information on the
methodology behind ZEUS and more support for our conclusions. Others offered us words of encouragement, as
they too have often wondered about the status quo in NFL coaching theory. Still others view us as a threat to the
establishment. Many of the questions centered on several common themes, so we thought we would attempt to
provide answers to those questions here rather than to each individual letter. Many answers can already be found in
the FAQ section of www.pigskinrevolution.com. If we are still missing something, let us know and we will do our best
to provide answers on a future date. But please, no more letters about Russian Roulette.
1. What exactly is ZEUS’ criteria for stating a decision is right or wrong?
First let us state that a recommendation from ZEUS is not an opinion. It is an objective series of simulations and tests
with the best available input, to determine the relative weight of 2 or more choices in a critical decision. We may try
after the fact, as we did in the ESPN article to shed light on the model’s judgments but it is not always immediately
clear what ZEUS is “seeing” that humans may be overlooking. When a coach is assessing a simple problem; such as
kicking the field goal on the last play of the game when trailing by 2 from the opponent’s 10 yard line instead of
passing for a TD, he is doing some basic mental arithmetic. He KNOWS the expectation of winning the game is far
greater using the field goal option rather than attempting the TD based upon historical observations. When we add
complexity to these situations (i.e. more time on the clock, different ball position, different score differential etc.), the
ability of a human to properly assess the decision, let alone in real time, becomes nearly impossible. ZEUS is using
mostly the same assumptions as the NFL play-caller; the difference is its ability to objectively process those inputs to
make an optimal decision. ZEUS can play a game to conclusion from any configuration of variables (ball position,
down, timeouts, yards-to-first, score, etc.) millions of times in a matter of seconds. Not only is ZEUS playing fast, it is
playing accurately. This accuracy has been validated through many comparisons of computer generated statistics
and actual NFL data. More importantly, ZEUS is very critical of its own recommendations and performs a stringent
statistical analysis of the results. This statistical analysis includes an assessment of just how much extraneous factors
could have mattered. ZEUS is a vehicle for rationally processing information. It is not a black box.
2. Do we consider momentum, emotion, field conditions and all of the subtleties around the human
performance?
As mentioned above. ZEUS is very critical of its own recommendations. We can not precisely weigh every subtlety of
the human element of the game. Players have good days and bad days. Some statistics are very hard while others
somewhat soft. The Confidence Factor associated with any ZEUS conclusion is the answer to these uncertainties.
This stress test takes the base result of comparing 2 or more play choices through an extensive custom simulation
and then tries its best to overturn those results. Someone may say that ZEUS didn’t give full consideration to the skill
of the punter when it recommended “going for the first down” instead of punting the ball away. In fact the model did do
that….and more! We may not have been able to input a perfect distribution curve around the expectations of that
individual punter, but we did look at the best and worst case possible scenarios of a punter and punt returner. These
scenarios are often far in excess of the reality of that individual player. The often surprising result is that sometimes
these custom human traits matter very little and sometimes, in fact, they don’t matter at all in a particular decision. The
corresponding Confidence Factor indicates just how much they do matter. We provide the coach a recommendation
and he can decide how to implement that information. When the Confidence Factor is low (0-3), he may have much to
think about. When it is 10, the evidence is basically irrefutable.
3. Wouldn’t the Cowboys be more motivated and thus have a greater chance of scoring on their final
possession if they are trailing by 1 as opposed to a tie score?
Many astute readers mentioned this and they may indeed be correct. We do not have any historical information
supporting this hypothesis, but it does seem logical that this should be the case. Our math exercise in the article
suggested that “S” (The probability of the Cowboys scoring on the final possession of regulation) was the same
whether they were tied or trailing by one. This was an oversight in the commentary but not in ZEUS’s actual analysis.
ZEUS did downgrade the Cowboys scoring expectations when the score was tied and this fact was revealed in the
lengthy simulation. The model’s recommendation that the Giants should have gone for 2 is based on that assumption
and we stand by the original conclusion. Sorry for the confusion.
4. How can you possibly think that a computer can out-think a veteran NFL coach
It is this very question that got us started on this project over 5 years ago. Anytime you are introducing new technology
into an established “old school” culture you are going to rock the boat. There are countless examples of where
technology was finally adopted in a new arena after a long arduous battle. Again, we are not loosely throwing around
our opinions on the theory of football. ZEUS is a tool….a very GOOD tool that reveals some counterintuitive insights
into modern coaching strategy. It would be naïve at best for the NFL establishment to believe that play-calling has
already evolved to its highest state.
5. Can ZEUS be used for the College Game?
Yes, with some modification. We have been focusing mostly on the NFL game but will be doing more analysis of
college football in the future.
6. Can ZEUS evaluate players?
This is the most exciting development to date. We are currently in the process of developing an application based
upon the core ZEUS model to objectively and accurately evaluate player’s seasonal contributions across all
positions. The application will likely work in tandem with the breakdown of game film. The ultimate goal is to convert
every aspect of the NFL game into a common currency of GWC.
7. The betting line does not provide a true assessment of winning probability
ZEUS does not incorporate any input from betting lines into the customization of the model. In the commentary, we
referred to the point spread only to put some perspective on the relative strengths of the teams. However, it should be
noted that a liquid market often has the best predictability of any publicly available information. Anyone who doubts
this, only needs to witness how well a bettor will perform against the line over the long haul. 52.3% is the benchmark
for overcoming typical casino “juice” and very few bettors can make this grade after a statistically significant number
of trials.
8. Can ZEUS be applied to sports betting?
ZEUS was designed to weigh relative play choices in critical (or directional) decisions. We put stock in the model’s
ability to distinguish whether play A succeeds more often than play B. If for example, A has a GWC of 57% and B
has a GWC of 54%, we are more concerned about the differential than the absolute numbers. The true winning
expectations may vary a bit, but what is most important is the model's ability support the relative outcomes. This
again is accomplished by means of a stress test and associated confidence factor. A sports bettor is more
concerned with the accuray of the absolute expectation of the outcome. While we believe that ZEUS may provide
valuable information toward absolute expected outcomes, we have not yet conducted a detailed experiment to
support the possibility that it could overcome the "juice".
.
9. And finally, didn’t you know that a blank bullet can, and has, resulted in serious injury or death?
The example of Russian Roulette was of course intended to be a thought experiment. We certainly don’t advocate
conducting this exercise at home. Yes, firing a blank next to your head is a very bad idea and in reality our example
wouldn’t have worked very well.