Game Testing: Simulations From tXtFL 0.9.2

Our goal is to test our football game model...and make some fantasy football predictions while we're at it. Feel free to download a copy of tXtFL and start making predictions of your own.

Week 16 reveals some tweaking of the football game engine to improve realism in the AI for choosing between passing and rushing plays, weighting of the offensive line in rushing plays, and the game clock. Week 17 ran simulations of the essentially finalized 0.9.2 release.

Results.

Methods.

For each game, we recorded the points at the end of each quarter. At the end of the game, we recorded and highlighted stats of major players before reverting the spec files to original form. A summary includes each game and mean scores from all games. Along the way, we made notes about potential tXtFL bugs, and we intend to continue analyzing the stats to generate more realistic game models.

Analysis.

Week 16: Niners vs. Bucs

The simulation series was conducted on stats downloaded on 2007-12-22 (the day before the real game) and played simultaneously with the official game. Note that the simulation model was tweaked and the starting Niners QB was switched mid-stream through the game series to reflect the actual team assignments. Also note that in the actual game, TB subbed in McCown for Garcia at the start of the 2nd half, but Garcia was left in for the simulations, which may have slightly inflated predicted scores.

After tweaking the simulation model, passing statistics inched a little closer to reality, which run yardage appeared to mimic the game much more closely. The final scores for each team fell within a TD of the simulation total. Although the simulated scores were relatively close to the real scores, in the end the Niners bucked the tXtFL predictions by prevailing over the Bucs.

Week 17: Pats vs. Giants

No changes were made to the simulation during game play. The simulation series was conducted on stats downloaded on 2007-12-28 (the day before the real game) and played simultaneously with the official game.

The simulated scores came within 3 points, or one FG, of the final scores. Although the simulated scores bounced around, once reaching 50 points, they averaged out to more reasonable levels. This time, perhaps with help from simulation tweaking instituted midway through the previous (Week 16) simulation series, the tXtFL predicted winner came home victorious in the real match-up.

Nonetheless, the passer completion % and total passing yards remain dismally low, although rushing yardage and max receiving yards per player appear roughly accurate. Perhaps the number of short pass plays chosen should increase to raise passer ratings without inflating scores. Special teams yardage also seems unusually high. Decreasing these yards may improve punt/kick return predictive accuracy and offset any increase in total passing yards.

Conclusion.

The tXtFL football simulator appears to be well on the road toward its 0.9.2 release, due out at the start of 2008. Further model tweaking has been slated for the following release and will hopefully be ready in time for the next football season.