• / Free eNewsletters & Magazine
  • / My Account
Home>Research & Insights>Investment Insights>In Practice: The Percentile Trap

Related Content

  1. Videos
  2. Articles
  1. New T. Rowe Target-Date Series Aims for Less Volatility

    T. Rowe's new target-date strategy differs from its flagship product by providing a lower equity allocation at retirement time and creating more predictability of returns, says manager Jerome Clark.

  2. Nygren: Value Managers Love Growth but Don't Overpay for It

    The Oakmark manager discusses a value case for Google, his team's three-pronged investment approach, and why capacity is not a major issue for the Oakmark Fund.

  3. College Expectations, Loan Rates at Crossroads

    Debates on Stafford Loan rates continue in Washington as the July 1 deadline approaches, while this year's college grads might be too optimistic of near-term job success.

  4. Johnson: Softer 1Q Doesn't Change My View

    Although the third read on first-quarter GDP came in lighter than expected, Morningstar's Bob Johnson is sticking with 2%-2.25% full-year GDP estimate.

In Practice: The Percentile Trap

When researching fund managers, put relative performance measures in their proper context.

Jeffrey Ptak, 06/18/2013

This article originally appeared in the June/July 2013 issue of MorningstarAdvisor magazine. To subscribe, please call 1-800-384-4000.

You can see it, but can you hit it?

It’s a baseball adage, but for manager researchers like us, it’s the pivotal question— we know that “good” managers are out there, but can we find them before it is too late? The historical record isn’t terribly encouraging in this regard. Examples of remarkably successful manager researchers—be they investment committees, gatekeepers, or private investors—do not exactly abound.

What’s the hitch? Well, manager researchers are human, after all. They chase returns; they misattribute outperformance; they rationalize underperformance; they get complacent; they dig in. That is, they act like the managers they’re paid to scout and monitor, whether they’re willing to admit it or not.

In our shop, we’re hardly strangers to such mistakes, though we try our best to minimize them. That’s why, when we reflect on our process and its blind spots, we consider not just the choices we make but also the context. Are we using the right yardsticks? Do they tell the full story? Do we have the proper frame of reference? Inevitably, our analysis has led us to reconsider that most ubiquitous of manager research tools—category peer rankings.

We’ve written previously about some of the quirks of peer groups (“Pitfalls of Peer Groups,”

August/September 2011). Our research found considerable flux within peer groups, with funds flitting between categories, raising questions about the substance of rankings themselves. We’ve also examined the fleeting nature of relative performance (“Performance Chasing, Evaluated,” April/May 2012), which tends to be mean-reverting (owing to stylistic biases, capacity constraints, “career risk,” or just plain luck). What we haven’t studied, however, is the convergence of the two and its implications on our ability to find successful managers in advance.

Our Study
We compiled rolling five-year net returns of all U.S. open-end mutual funds (existing and obsolete; oldest share-class only) that were members of the Morningstar large-value, large-growth, mid-blend, small-value, or small-growth categories on March 31 of any year from 2003 through 2013. We then ranked the funds within their peer groups based on trailing five-year returns. (Thus, we studied 11 distinct five-year periods: April 1998 to March 2003; April 1999 to March 2004; April 2000 to March 2005; and so forth.)

We compiled the returns using a “category true” method. We rebuilt the peer groups based on their historical Morningstar category classifications, rather than working backward from their current classifications (which might not represent how a fund was categorized in the past). The peer groups that we assembled should closely approximate the peer groups that actually existed at the end of each relevant five-year period.

In assembling the data, we sought to find out whether past top-quartile rankings predicted outperformance in successive five-year periods.

Our Findings
We found that unerringly consistent performance is rare: 40% to 50% of top-quartile funds promptly fell from the top quartile, on average, in the next consecutive rolling five-year period (Exhibit 1). That is, if a fund was top quartile in the five-year period ended March 2005, there was a roughly 50/50 chance that it would not be in the top quartile in the five-year period ended one year later in March 2006. A year further out, we found that only about one third of the original group of top-quartile funds remained. By the fifth year, almost none of the top-quartile funds were left. In short, top funds almost always fall from their perch.

What became of these top-performing funds? We tracked the subsequent performance of the 45 large-value funds whose trailing five-year returns placed them in the category’s top quartile as of March 2003 (Exhibit 2). One year later (i.e., rolling five-year period ended March 2004), 22 of the original 45 tumbled out of the top quartile, moved to a different category, or had been mothballed. Of the 23 top-quartile funds still standing, only 14 would remain by March 2005. By March 2008, just two of the original 45 funds remained in the top quartile.

True, this is a very exacting test, and the results aren’t shocking given the way relative performance tends to revert to the mean. So, instead of focusing just on consistency of relative rankings, we also compared funds’ five-year rankings at various points in time. We took a look at where top-quartile funds in these categories ended up five years later (i.e., rankings in the five-year period ended March 2003 versus rankings in the five-year period ended March 2008, and so forth). What we see isn’t just mean reversion, but also category flux and fund attrition on a significant scale (Exhibit 3). For example, on average, around 20% of top-quartile large-value funds had either exited the category or had been merged or liquidated away by the end of the subsequent five-year period. The churn was even more pronounced in the small-value, small-growth, and mid-blend categories, where nearly 30% of top-quartile funds didn’t even finish the following five-year period in the peer group.

All told, only 10% to 20% of top-quartile funds remained in the top quartile over the subsequent five-year period (versus 20% to 30% that fell to the bottom quartile), on average. About 30% to 40% finished in the top half (versus 40% to 50% in the bottom half, though that swells to 60% to 70% when we include funds that switched categories or died). In other words, you had far better odds of picking a future loser, category émigré, or casualty among top-quartile funds than you did a winner.

So, if top-quartile funds tend not to repeat in subsequent periods (or even survive them), where exactly do they come from? To answer that question, we examined the percentile rankings of all top-quartile funds in the five-year period immediately preceding the five-year period in which they landed in the top quartile. For example, if a small-value fund ranked in the top quartile in the five-year period ended March 31, 2010, we took a look at its ranking for the five-year period ended March 31, 2005, and so forth for other categories and periods. We summarize our findings in Exhibit 4.

A few aspects of the chart stand out. For one, 20% to 25% of top-quartile funds began the five-year period in a different category. In addition, new funds—those that lacked a five-year track record at the beginning of the five-year period concerned—accounted for 15% to 25% of eventual top-quartile funds, on average. Try as one might to predict which fund would beat its category peers in the future, the reality is that many of those outperformers are nowhere to be found—they reside outside of the category in question or lack a sufficiently long track record to be considered.

And then there’s the other striking aspect. You were likelier to find a top-quartile fund in a category’s bottom half than its top half. In fact, only about 10% of eventual top-quartile funds began the period in that category’s top quartile, which is roughly the same proportion that hailed from the category’s bottom quartile.

No Silver Bullet
The key takeaway isn’t that category rankings are unreliable and should be avoided at all costs. Rather, it’s that, like virtually anything in investing, they’re not a silver bullet. The vagaries of mean reversion and the changing nature of peer groups see to that, making it important that manager researchers place relative performance in the proper context. Put another way, a fund’s percentile ranking shouldn’t govern investment decisions, which is a trap that many of us have fallen into. What are the implications for manager researchers? We can think of a few:

Emphasize Benchmarks
Unlike managers who might move to and fro for a variety of reasons, a well-constructed index shouldn’t stray. It’ll simply reflect what managers own, in aggregate, within that defined area or style, no matter if it’s the same group of managers over time. This should make for cleaner, more-telling comparisons.

Avoid Granularity
Funds migrate between categories with some regularity. If the manager you’ve chosen isn’t flitting about, chances are that at least some of his or her peers are. This makes comparisons challenging, especially over longer periods. The more granular our classifications, the worse this flux is likely to be and the less meaningful our comparisons would be. So, to the extent possible, it makes sense to avoid getting too granular when classifying managers.

Make Choices
If manager research is hard, it stands to reason that we compound the difficulty when we weigh down a portfolio with numerous funds. Why? As mentioned, classifications that are too granular have less meaning. And while there’s theoretically a case to be made for diversifying into funds with uncorrelated alphas, the fact is that we as manager researchers have to make judgments about when to buy and sell, something we don’t excel at.

Embrace Flexibility
Consider managers who don’t pigeonhole themselves, but rather focus on a flexible, but identifiable, investing style. This presents its own challenges, such as finding a suitable benchmark for an eclectic manager. But as a practical matter, it avoids some of the problems associated with peer grouping, including head fakes caused by swings in relative performance. It also can mitigate issues associated with career risk, whereby a manager creates a portfolio in his or her peer group’s image, rather than investing with high conviction wherever the best ideas reside.

Incorporate Returns-Based Analysis
Holdings-based analysis is useful, and we use it in our manager-research process. However, to remove some of the noise of peer-group rankings, it can be worthwhile to create custom benchmarks for managers based on the sensitivity of the portfolio’s returns to various style-based indexes over time.

Be Contrarian
Our research strongly suggests that relative performance is mean-reverting. Leaders become laggards and vice versa (if they don’t switch categories or get merged away first). This should strike fear into the heart of any trend-follower, which is what manager research too often devolves to.


Guest Author

©2017 Morningstar Advisor. All right reserved.