Survivorship Bias

Here’s a question to test your knowledge of the Australian equity market; what proportion of investment strategies have outperformed the index over the last three years?

If you guessed a number close to 50%, you’re on solid theoretical ground. It is generally accepted that about half of all active strategies will outperform, while the other half will underperform. After all, in a closed system, for every winner there is a corresponding loser[1]. However in this instance, you would be far away from the actual answer.

If we look at the universe of active Australian equity strategies according to eVestment^[2] over the three years to 30 June 2015, a whopping 78% have outperformed the S&P/ASX 300 index. Moreover, this is not extraordinarily high for this statistic. In the three years ending 30 June 2003 the figure peaked at 91%. In other words, nine out of ten strategies were ahead of the index!

As can be seen from the following chart, it is rare for this statistic to drop below the theoretically pure 50% mark. Taken at face value, this chart would argue that most strategies outperform the index most of the time.

This is clearly nonsense.

What this chart is actually telling us is that of the strategies still submitting performance data to this database today, most have outperformed the index over most time periods. This makes much more sense. It is the successful strategies that have “survived” to today and therefore continue to add to their performance histories.

This characteristic is often described as survivorship bias – a type of selection bias that is especially pernicious in the investment management arena. After all, it is hard to market a strategy that has persistently lost money and therefore these underperforming strategies are more often than not the ones that close down and drop out of performance surveys. Put another way, performance surveys often do not accurately reflect the entire universe of competing strategies through history.[3]

Arguably it is even more interesting is to note the peaks and troughs in the chart. There seem to be distinct periods where the majority of the universe has performed well. For example, from mid-2009 to mid-2011 roughly 70-80% of the universe consistently outperforms on a rolling three year basis.

If we highlight periods of market stress[4] some patterns begin to emerge.

Allowing for the trailing nature of the data^[5] it is not hard to argue that the highest peaks tend to coincide with the periods of market stress. In particular, as noted above, the gold line stays elevated through the GFC, indicating that the majority of strategies that are still around today are those that consistently outperformed the market in this period. Many of these strategies will have been run by highly skilled investment managers but a subset will have simply been those that “got lucky”.[6]

This analysis partly explains why it pays to be wary about using past performance to assess investment manager skill. It is fairly well-known that performance databases can be subject to survivorship bias, but the analysis above seems to suggest that over and above this there is also a bias towards those strategies that have, by luck or skill, navigated previous crises successfully.

Whether as many will survive the next crisis remains to be seen.

CPD: Actuaries Institute Members can claim two CPD points for every hour of reading articles on Actuaries Digital.