Published: 04 December 2014
I've found that opening up my models to the public can often be a very good thing. It gives me essential feedback I need to find bugs or mistakes. It also builds confidence in the models. They often point to unconventional or counterintuitive results, and unless people understand how they work the analysis is likely to be dismissed as a black box.
I've been trying to think of a way to make the new WOPR game simulator available as a useful tool. Simulations work because they can run thousands of times to produce an overall rate or probability of each relevant outcome, and this takes time. Some simulations can be pretty simple, but the WOPR isn't one of them. It's really complex and it puts a big load on the server when it runs. It can take some time for it to chew through thousands of simulated games. However, I'd still like to make it available in some form.
For starters, I'm going to open it up to everyone, at least temporarily, but it will only run a single game. That's of very little analytical use, but depending on the response I may be able to open it up for relatively small batches of iterations. It still is useful for the reasons I mentioned above. It will expose it to criticism and help find any bugs that still hide inside the code. Also, it will (hopefully) inspire some confidence. When it runs once, it outputs the play-by-play.
Don't expect the kind of play-by-play you might see on nfl.com. There are no players. There are only times, downs, distances, field positions, play types and outcomes. The outcomes include gains and losses, and turnovers if they occur. The sim doesn't care if a turnover is a fumble or interception. It just needs to know that a turnover occurred and where the ball ended up. There are a handful of similar simplifications that don't affect the game outcome, which is the thing we care about. But eventual enhancements can address any consideration we're interested in.
Each play has an associated note. It mentions significant outcomes and gives you a window into the simulated 'mindset', such as 2-minute offense or the fact that a team needs a TD to survive and a field goal isn't helpful. Stuff like that is part of what governs the play selections and play outcome distributions.
The user interface should be familiar. There are a few noticeable differences from the Win Probability Calculator though. Instead of analyzing things from the perspective of an offense, there are two teams--the Home and Visitor. There's no home field advantage built in (yet), but it allows you to select which team has possession.
You can also override the next play in the game. Usually, when I want to investigate something interesting, I reach into the code and tinker with things to force a certain event. For example, last week when I investigated the strategy of kicking a field goal as soon as an offense reached field goal range when down by 9 or 10, I had to trick the sim into kicking a field goal on 1st down. You can now do the same thing by using a drop down menu. So if you want to investigate the wisdom of taking an intentional safety in a certain situation, you can add 2 points to the defensive team and force the next play to be a free kick.
You can change the number of replications the sim will do with the "n" variable. But for now it's only going to run once, no matter how high you make it.
What I'd ask of you is that you look through the simulated game and see if you detect anything that doesn't make sense. Come back to this post and make note of anything in the comments. I'm not worried about interface items. If you want to break it by entering a negative number for a down or whatever, I'm not concerned about that. I'm more concerned with something like "Every time it kicks off after a field goal it's a touchback. I tried 20 times."