Morningstar Rating Hits Five-Year Anniversary: How's It Doing?

This column is excerpted from the September issue of Morningstar FundInvestor.

Happy anniversary, Morningstar Rating for funds! The star rating is now five years old. That is, we adopted the current methodology five years ago.

In 2002, we put some complicated financial engineering under the hood to produce a better result, but the biggest change was a simple one: We moved from an asset-class peer group to a category-specific one so that mid-value managers were being compared with mid-value managers and intermediate-term bond funds with other intermediate-term bond funds.

The complicated bits involved a new measure for risk and a change in weighting multiple share class funds so they didn't count for more than single share class funds. The improved risk measure more accurately reflected investors' reactions to risk and did a better job capturing volatility as an indicator of risk. Previously we had used underperformance versus Treasury Bills to measure risk. The revised measure, based on the utility function, assumes that at any return point, investors would gladly trade some risk for a bit more safety, whereas the old measure only focused on one point on the risk/reward curve. Investor behavior backs up this assumption. We've seen that investors use high-risk funds rather poorly as they bail out after losses that were too great to endure only to miss out on the rally that ensued.

Because the rating is category-based, it's worth mentioning a couple of aspects about categories. First, we look at three years' worth of portfolios when we assign categories to ensure that a fund is in the category that best fits its long-term strategy and to make it difficult for funds to game the rating by shifting categories. In addition, when a fund changes categories, we raise the weighting given to the three-year period because that more closely reflects the fund's performance with a portfolio similar to its category peers. This has the added benefit of making it even harder to game the rating. The load adjustment and the time periods remain in place. The end result is that the star rating is much better at capturing which mutual funds add value and is, therefore, helpful in selecting funds.

Studying the Star Rating
Each year since the new star rating was launched in 2002, we have run a study to see how well it worked at predicting returns and risk-adjusted returns. We look at the ratings from the end of June in each year and gather data on ensuing total returns and relative performance. And the star rating itself serves as a measure of future load-adjusted, risk-adjusted performance. We now have five years' worth of performance on the new star rating. (To view a table, click here.)

As in past years, we found that 5-star funds have modestly outperformed 4-star funds, and 4-star funds have modestly outperformed 3-star funds, and so on down the line. When you compare opposite ends, you see that 5-star funds outperformed 1-star funds by a healthy margin. Specifically, on average for the periods studied, 5-star domestic-stock funds beat 1-star funds by 1.46% annualized. International 5-star funds outperformed 1-star funds by a smaller 0.91% annualized. The rating worked best in balanced funds, which outperformed by 3.12% annualized. Taxable 5-star bond funds outperformed by 0.73% annualized, and municipal-bond funds by 1.15%. We also found that higher-rated funds received a higher three- and five-year rating over the ensuing periods.

Adding in Low Costs
I was curious to see what happened when we compared 5-star funds whose expense ratios were in the cheapest quartile of their category with the whole group of 5-star funds for the same asset class. Interestingly, this time the additional expense ratio cut did not add much in total returns, but it led to better star ratings 14 out of 15 times. Why? Funds with higher costs often take on greater risks in order to compensate for those costs. In addition, funds with high-risk strategies often charge more because volatile returns lead investors to mistakenly tolerate higher expenses. Either way, higher-cost funds tend to have greater risk. In fact, funds that are both 5 stars and low cost have a significantly lower standard deviation than those that are just 5 stars.

Factoring in Survivorship Bias
As with our study on expense ratios, we found that the star rating looks even better when accounting for funds that have been merged away or liquidated. In short, 5-star funds are less likely to be eliminated than other funds, whereas lower-rated funds are more likely to do poorly and be merged away. To capture this effect, we asked what were the chances of funds in each star-rating class producing above-average performance. For example, 5-star-rated international funds in June 2002 survived and outperformed their peers 59% of the time compared with 19% of the time for 1-star funds--meaning you had triple the chances of success with 5-star funds. This varied over other time periods and asset classes, but there was always a dramatic improvement. We found that 5-star domestic-stock funds had a 50% greater chance of success, while 5-star muni funds had 4 times as great a chance at success as 1-star funds.

When we factored in low costs and survivorship bias, results improved significantly. Funds that were both 5 stars and low cost had a better chance of success than those that were just 5 stars. Because we know that costs didn't add much on the pure-performance screen above, that must mean that they are helping by making funds more likely to survive--something backed up by our recent study of expense ratios' predictive power. In short, finding cheap, highly rated funds will put you on the right path.

Where the Star Rating Leaves Off
Some fund managers run screens and other quantitative measures to narrow the field to promising candidates before they do fundamental research to select stocks. You can do something similar with the star rating. Let it narrow the field for you when you buy and when you decide to sell.

However, you still have work to do. Because the star rating looks backward, it's your job to look ahead. Look for funds with strong competitive advantages such as superior management and analysts. Look for low costs--still the best predictors of performance.

And, when paired with 5-star funds, low costs help lead you to lower-risk funds. Look for funds with high Stewardship Grades, and avoid those with low grades (something the star rating won't always capture).

Finally, look for a fund that should be able to sustain its success because its asset growth isn't out of hand and it isn't in an overheated asset class, such as real estate circa 2007. These are ideas embraced by our  Fund Analyst Picks. So, use the Morningstar rating along with a healthy diet of research, and you'll do just fine.

Get More Details on the Star Rating Methodology
Although some mistakenly think that the star rating is a secret formula, we actually publish the whole formula on our corporate Web site. If you want to drill down into all the details click on this link and you'll learn more about the star rating than you probably wanted to.