Making Sense of Survivorship Bias

Question: How does Morningstar deal with the issue of survivorship bias in calculating a fund's performance relative to its category?

Answer: Morningstar.com's fund data pages deal with the issue of survivorship bias in different ways depending on the time frame. But before we explain these differences, let's review what survivorship bias is.

Survivorship bias refers to how historical fund performance data can be skewed if you consider only existing funds rather than also including funds that have been liquidated or merged into other funds. By only including existing funds--or "survivors"--performance data might not accurately reflect a fund's performance relative to all of its competitors during the time period in question.

For example, let's say that 10 years ago a fund category consisted of 100 funds, but today only 50 of those funds survive. By comparing funds in the category only with their surviving peers for that time frame, one might come away with an inaccurate picture of a fund's performance relative to its competition. That's because funds that are liquidated or merged away tend to be poor performers, so not including them when comparing an existing fund's performance to that of its peers might make the fund look worse than it really is or was. For example, a fund might have delivered returns that rank in the second quartile when compared with its surviving peers, but that same performance might land it in the third quartile if nonsurviving funds are included.

Calendar-Year, Trailing Data Treated Differently
Some Morningstar performance data is free of this survivorship bias while some is not. In particular, category rankings for calendar years are free of survivorship bias, meaning that funds that no longer exist are included in these calculations provided they existed for the entire year in question. So a fund's ranking for 2006 would be based on all the funds that existed in its category as of Dec. 31 of that year, including those that have since been liquidated or merged.

However, rankings based on trailing performance are not survivorship-bias-free, meaning they do not include funds that no longer exist; the same fund's trailing 10-year total-return rank is based only on surviving funds in its category that have 10-year track records. The reason survivorship bias exists in trailing-return data is that it is very difficult to eliminate this bias for data that is tracked daily, not to mention the fact that trailing returns are calculated up to the most recent trading day, which makes comparison with funds that no longer exist irrelevant.

Funds Come and Funds Go
To see for yourself how the number of funds in a category can change over time, click on a fund's Quote page on Morningstar.com and scroll down to the Performance section. There you'll find information on the number of funds in the category during different trailing time periods (similar information can also be found under the Ratings & Risk tab). Keep in mind that only funds that have been in existence long enough to qualify are ranked for a given time period, so a fund that has only been around for five years won't be included in the category's 10-year performance rankings.

As an example, let's look at  Vanguard Wellington (VWELX), which resides in the moderate-allocation category. On the fund's Quote page you'll see that there are currently more than 1,000 funds in its category, whereas only about 400 have been around long enough to be included in the 10-year trailing return rankings. Again, the number of funds qualifying for comparison over shorter (more recent) time frames typically is higher than the number qualifying for longer periods because new funds are opening all the time while older funds sometimes are liquidated or merged into other funds and thus are excluded from trailing performance data. Therefore survivorship bias may be greater for longer-term trailing performance intervals than for those based on shorter time frames.

A Subject of Debate
It's easy to make more of survivorship bias than is really necessary. Yes, survivorship bias can skew data when comparing average performance between categories, especially if one category has seen more unsuccessful funds fall by the wayside over the years than the other. In fact, the subject of survivorship bias often comes up during debates about active versus passive fund management approaches. Investors who favor passive fund management--in which funds track an index as opposed to having a manager pick specific securities to buy and sell--argue that survivorship bias skews performance comparisons because many underperforming actively managed funds that no longer exist are excluded, making active management's track record appear better than it really is. We won't try to settle that argument here.

But despite being subject to survivorship bias, fund trailing-return rankings and Morningstar Risk ratings are still a useful tool investors can use to compare a fund's performance to that of its surviving peers over time. After all, for investors trying to decide whether to buy, sell, or hold a fund, how that fund ranks relative to its current competition--rather than its performance relative to funds that no longer exist and are no longer available--is usually what matters most.

---

Have a personal finance question you'd like answered? Send it to TheShortAnswer@morningstar.com.