Published: 12 October 2014
I've been trying to outsource 4th down analysis to the bot lately, but I'll chime in on this one for a couple reasons. The bot doesn't handle complex overtime situations, like the down-by-3-points-score-or-lose
situation the Panthers were in. Plus, this 4th down had enormous leverage in the game outcome. I'll use two different tools to see if Analytic Ron made the right decision--the new Win Probability model and the WOPR game simulator.
CAR was down by 3 points to CIN in the new OT system, which means they needed a FG to force sudden death or a TD to win. They faced a 4th and 1 from the CIN 17 with 2:24 to play. It seemed like a 'long 1' and not inches to go.
A tie is fairly likely given CAR elects for the FG attempt, so we need to value a tie as 0.5 of a win. Here's how the math works out with the WP 2.0 model:
WP(successful conversion) = 0.664
WP(failed conversion or FG miss) = 0
WP(good FG) = 0.475
FG success rate = 0.87
Break-even conversion probability = (0.475*0.87 - 0) / (0.664 - 0)
The WOPR simulator doesn't work quite in the same way. Running a very large number of simulated games yields a WP of 0.582 for a successful conversion. Going for the FG produces a WP of 0.388. In the end, those numbers mean the break-even conversion probability would need to be 0.66 for the 4th down attempt to be worthwhile.
The league-average success rate on similar 4th downs is 62%, so Analytic Ron faces a very tough decision. If anything, the WP model slightly favored the FG attempt, and the fact that the yard to go was longer than the typical 4th and 1 suggests the RiveraBot made a very defensible call.
I can't account for the differences between the WP model and the sim off the cuff. The WP model is based on some heavy assumptions due to the absence of a critical mass of data needed for direct analysis. And the input probabilities for the simulator's down-by-3-in-OT
situation suffer similar problems. But the fact that the probabilities turned out so close, and the final analysis produced very similar conclusions makes me optimistic that the two approaches are converging on useful, actionable insight.