Back testing the original paper

Whilst the original paper uses additional statistical methods to attempt to correct for any issues with the data modelling I wanted to check some of the historical analysis they had used when creating their model so ran a back test.

The model uses the following formula for calculating the R.A.F. pilot strength BS on day t:

BS(t) := B0+ sbt - BL

Where B0 is the strength on the starting date of the simulation, s is the effective value of a new pilot compared to an experienced pilot, b is the average daily compliment of new pilots and BL is the number of pilots lost.

The value of s used is 0.85 which they have based on data from 501 squadron. For b an unsourced value of 11 has been used. As both of these are constants this gives a replacement value of 9.35 pilots per day, assigning different values to b and s doesn't seem to add any value to the model.

Working with the data published in the paper I have run a back test of their model.

Bootstrapped R.A.F. pilot strength (after Fagan et. al.)

This is similar to figure 3 in the original paper but what they have labelled as "Actual Battle" is what I have labelled as "Calculated".