Back testing the original paper

Whilst the original paper uses additional statistical methods to attempt to correct for any issues with the data modelling I wanted to check some of the historical analysis they had used when creating their model so ran a back test.

The model uses the following formula for calculating the R.A.F. pilot strength BS on day t:

BS(t) := B0+ sbt - BL

Where B0 is the strength on the starting date of the simulation, s is the effective value of a new pilot compared to an experienced pilot, b is the average daily compliment of new pilots and BL is the number of pilots lost.

The value of s used is 0.85 which they have based on data from 501 squadron. For b an unsourced value of 11 has been used despite another unsourced claim of 260 new pilots a month - i.e. about 8 per day (an actual value given in a primary source is 280 for the month of August which would be close to 9 per day). As both b and s are constants this gives a replacement value of 9.35 pilots per day, assigning different values to b and s doesn't seem to add any value to the model.

Working with the data published in the paper I have run a back test of their model.

(This is similar to figure 3 in the original paper but what they have labelled as "Actual Battle" is what I have labelled as "Calculated".)

Notably this diverges from the recorded figures from the start of August when the battle commenced in earnest so that does raise concerns with me as to the usefulness of this model for my purposes.