Does the Star Rating for Funds Predict Future Performance?

Does the Morningstar Rating for funds, more commonly known as the star rating, predict future performance?

Before we tackle that question, let's briefly review: The star rating is a backward-looking, quantitative measure of a fund’s past performance compared with its category peers. We’ve long stated that the star rating can be a good starting point for research for a few reasons:

• It is based on funds' trailing three-, five-, and 10-year returns versus category peers, not just the most recent year. Longer-term returns are less unstable and, thus, a better predictor of future performance than short-term results.
• It compares funds' net-of-fee returns and incorporates any sales charges. Our research has shown that fees are one of the best predictors of future success.
• It adjusts fund returns to account for risk (technically, "downside volatility"). This acknowledges that some funds are harder for investors to use and factors that into the rating accordingly.

In other words, the star rating distills some of the more telling indicators of fund success into one easy-to-use measure. This, in turn, can help investors cut a larger universe of investment options down to size, clearing the way for more-detailed research from there.

But is it predictive? To answer that question, we recently conducted a detailed study into the star rating's predictiveness using two methods:

• A regression technique that examined the relationship between the star rating and fund returns in the month immediately following the rating;
• An event-study that measured funds' subsequent returns over various time horizons.

Here, we summarize the study approach and our key findings.

Study Approach The time frame of the study was January 2003 to December 2015 (January 2008 to December 2015 for alternative funds) and included all rated funds in our database during that span. In addition, the study included obsolete funds (that is, those that have been merged or liquidated) and thus does not suffer from survivorship bias.

Using this data, we conducted two tests of the star rating’s predictive power. The first was a regression procedure, known as a “Fama-MacBeth cross-sectional regression,” which sought to evaluate the relationship, if any, between a fund’s star rating and its return in the next month. In particular, we sought to assess whether higher-rated funds were associated with superior forward one-month returns (relative to 3-star funds) than lower-rated funds after controlling for numerous variables, including asset class, Morningstar Category, fund expense, and risk.

The second test was an event-study procedure in which we took a snapshot of funds’ star ratings at a given point in time, measured subsequent performance over varying time horizons, and then compared the returns of the ratings cohorts over those event horizons.

We ran twin tests for several reasons. We wanted to conduct a rigorous study that drew upon innovative research techniques that have been widely adopted and accepted in academic circles. This explains our use of the Fama-MacBeth cross-sectional procedure. However, recognizing that this regression technique could be less intuitive to some readers, we also sought an alternative that would be more accessible; hence, our inclusion of the event-study procedure.

Key Findings Using these two performance frameworks, we found that the star rating had moderate predictive power during the study period.

The Fama-MacBeth approach found that funds with higher star ratings had superior returns even after accounting for expenses and various risk exposures. Furthermore, these results held across all asset classes except alternatives.

• Among equity funds, the forward one-month returns of 5-star funds were 0.09% higher than those of 1-star funds, or 1.03% annualized. (To explain the exhibit below, the average forward month return of 5-star equity funds was 0.05% higher than that of 3-star equity funds; meanwhile, the average forward month return of 1-star equity funds was 0.04% lower than that of 3-star funds; hence, there is a 0.09% difference in the forward-month returns of 5- and 1-star equity funds.)
• Similarly, 5-star fixed-income funds registered 0.09% higher forward one-month returns than 1-star funds (1.09% annualized), and 5-star allocation funds notched 0.15% higher returns than 1-star funds (1.75% per year).
• The premiums observed for fixed-income and allocation funds were highly statistically significant (that is, they are highly unlikely to be the result of chance), but less so for equity funds.
• There wasn't a statistically significant difference between the performance of 5-star and 1-star alternative funds. Although it will be the subject of further analysis, these results could be explained by smaller sample size (January 2008 to December 2015) and model misspecification (owing to unique return drivers among alternative funds).

The next exhibit shows the cumulative returns of $1 compounded by the star rating premiums observed for rated equity funds over time. (The premiums cited above are averages for the full study period, whereas the next exhibit incorporates the premiums measured at the relevant points in time.) From this, we can surmise how the premiums associated with ratings would have translated into material cumulative differences over time.

The event study approach led us to similar but less-convincing conclusions. The returns of the 5- and 4-star rating portfolios were higher than those of the other star rating cohorts, but not substantially so. Similar to the regression approach, the event study reveals a consistent (though weak) monotonic pattern throughout all periods, with the higher-rated portfolios delivering better average cumulative returns, as shown in the exhibit below.

It is also worth noting that 5-star funds were far likelier to survive the full event-study horizon, especially the 60-month horizon, than lower-rated funds. Lower-rated funds are merged and liquidated away more frequently, sometimes at considerable inconvenience and tax cost to investors. By contrast, 5-star funds live longer, affording investors greater opportunities to succeed without interruption and forestalling the need to make an additional investment decision to select a replacement fund. In this way, one could argue the star rating confers the benefit of predicting survival.

Conclusion Taken together, our findings suggest that the star rating had moderate predictive ability for risk-adjusted returns in the short term. Over the long term, the event study indicates that the star rating has appeared to do more good than harm.

• At the one-month horizon of the Fama-MacBeth regression procedure, higher-rated funds exhibited strong persistence in performance that cannot be explained by their fees or risk exposures. The event study results corroborate these results over the one-month horizon.
• Longer-term event study results further suggest that, on average, investors who have bought higher-rated funds have tended to earn higher returns than those who have purchased lower-rated funds, though the magnitude of this outperformance is low.
• High-rated funds were far likelier to survive longer event horizons than low-rated funds.