Survivorship Bias
How accurately do performance databases reflect competing investment strategies? Ben Trollip looks at the bias towards strategies that have, by luck or skill, successfully navigated past crises.
Here’s a question to test your knowledge of the Australian equity market; what proportion of investment strategies have outperformed the index over the last three years?
If you guessed a number close to 50%, you’re on solid theoretical ground. It is generally accepted that about half of all active strategies will outperform, while the other half will underperform. After all, in a closed system, for every winner there is a corresponding loser[1]. However in this instance, you would be far away from the actual answer.
If we look at the universe of active Australian equity strategies according to eVestment[2] over the three years to 30 June 2015, a whopping 78% have outperformed the S&P/ASX 300 index. Moreover, this is not extraordinarily high for this statistic. In the three years ending 30 June 2003 the figure peaked at 91%. In other words, nine out of ten strategies were ahead of the index!
As can be seen from the following chart, it is rare for this statistic to drop below the theoretically pure 50% mark. Taken at face value, this chart would argue that most strategies outperform the index most of the time.
This is clearly nonsense.
What this chart is actually telling us is that of the strategies still submitting performance data to this database today, most have outperformed the index over most time periods. This makes much more sense. It is the successful strategies that have “survived” to today and therefore continue to add to their performance histories.
This characteristic is often described as survivorship bias – a type of selection bias that is especially pernicious in the investment management arena. After all, it is hard to market a strategy that has persistently lost money and therefore these underperforming strategies are more often than not the ones that close down and drop out of performance surveys. Put another way, performance surveys often do not accurately reflect the entire universe of competing strategies through history.[3]
Arguably it is even more interesting is to note the peaks and troughs in the chart. There seem to be distinct periods where the majority of the universe has performed well. For example, from mid-2009 to mid-2011 roughly 70-80% of the universe consistently outperforms on a rolling three year basis.
If we highlight periods of market stress[4] some patterns begin to emerge.
Allowing for the trailing nature of the data[5] it is not hard to argue that the highest peaks tend to coincide with the periods of market stress. In particular, as noted above, the gold line stays elevated through the GFC, indicating that the majority of strategies that are still around today are those that consistently outperformed the market in this period. Many of these strategies will have been run by highly skilled investment managers but a subset will have simply been those that “got lucky”.[6]
This analysis partly explains why it pays to be wary about using past performance to assess investment manager skill. It is fairly well-known that performance databases can be subject to survivorship bias, but the analysis above seems to suggest that over and above this there is also a bias towards those strategies that have, by luck or skill, navigated previous crises successfully.
Whether as many will survive the next crisis remains to be seen.
[1] Although we are making a number of assumptions in this theoretically pure world, including the fact that the benchmark captures the entire investible universe. This is not necessarily true because off-benchmark positions can be held by many strategies. This would include stocks outside the index and cash.
[2] The universe has been sourced from eVestment and comprises all strategies classified in the “All Australian Equity” category.
[3] eVestment, like other performance databases, combats survivorship bias by retaining the performance of strategies that have stopped submitting performance data. In this analysis we have included strategies within eVestment’s database that are marked as “inactive”, so our sample incorporates some strategies that have not survived to present day.
[4] We have picked out the approximate dates covering three periods of “pain”: (1) July 1997 to August 1998: Asian financial crisis and Russian financial crisis, (2) March 2000 to March 2003: Dotcom bust and September 11th attacks, (3) January 2008 to January 2012: Global financial crisis and aftermath.
[5] That is, that performance through the period of stress will take some time to be expressed in the rolling one and three year figures.
[6] It is also worth remembering that there is a natural boost to relative performance in “bear” markets for strategies that can hold cash. In practice, most managers will have at least a small portion of liquidity in their portfolios.
CPD: Actuaries Institute Members can claim two CPD points for every hour of reading articles on Actuaries Digital.
