The Star Rating Gains Momentum

The following is adapted from an article that originally appeared in Morningstar FundInvestor.

We launched the revised Morningstar Rating for funds in 2002, and each year since then we've updated a study on how the rating has performed. For background, the rating is a quantitative measure of risk-adjusted performance relative to a category over the past three-, five-, and 10-year periods.

For this year we look at how the ratings as of June 2002, June 2003, and June 2004 have fared at predicting relative performance and future star ratings. Now that the ratings are four years old and we're on to our third study, we can be more certain about how it's working. Fund investing is about long-term success, so the best measure of the star rating will be long-term performance. We are starting to get there with four-year returns, but we're still a ways away. We looked at shorter periods, too, for purposes of tracking how the rating is faring, but we do not want to read too much into two-year returns.

How Do 5-Star Funds Perform?
Consistent with past studies, the new study shows that 5-star funds tend to outperform 4-star funds and so on. Thus, the biggest performance gap is from 5-star funds to 1-star funds. This was the case when measured by relative performance and the three-year star rating. (Click here to see the table.)

Looking out over the three time periods tested, it's apparent that this trend is holding up and may be getting stronger. Consider that 5-star domestic-stock funds from 2002 produced top-47% returns while 2003 improved to top-43% and 2004 to top-39%.

I'm not yet certain if this means that the rating is best at predicting near-term performance or if the rating simply performed better in the ensuing years. As you recall, 2002 was an inflection point--the last brutal year of the bear market. Thus, returns were less useful as a predictor of future performance that year than they would be in a more normal year.

The 2004 ratings are notable for the fact that they were the first to reflect our style-specific foreign-stock categories such as foreign large-growth. For the ensuing two-year period, 5-star international-stock funds averaged top-40% returns while 1-stars were in the bottom 45%.

Bond Fund Ratings Work Best
The rating for bond funds and balanced funds is even more powerful than it is for stock funds. Consider that the gap between 5-star and 1-star funds over the three time periods was an average of 26 percentage points for balanced funds, 27 percentage points for taxable-bond funds, and 40 percentage points for municipal-bond funds.

The most-likely explanation is that within a bond-fund category the returns are closely bunched because bonds tend to perform pretty closely to each other with only modest differences in returns. That means that expense ratios have a big impact on relative performance rankings and expense ratios hardly change at all. Thus, the star ratings capture a source of persistent outperformance. Moreover, the top bond shops are head and shoulders above the rest of the crowd, and they are more consistent in their outperformance than stock-fund managers again. These two factors combined make the star rating a strong measure for bond funds and balanced funds.

An Anomaly Disappears
I was interested to see that one anomaly that stood out in last year's study is no longer there. Last year, I reported that 1-star international funds from 2002 had produced a surprisingly strong top-43% return ranking over the ensuing three years. One year later, that top-43% has melted to bottom-63%. What happened? In two words: emerging markets.

The star rating is a risk-adjusted measure, but in 2003 to 2005 emerging markets were a risk that paid off wonderfully. Thus, international funds penalized by the star rating for risk were in the clover. In the second quarter of 2006, that risk came home to roost as emerging markets were clobbered and funds with excessive bets on them took a beating. That's an encouraging sign that our risk measure is working at capturing funds with big underlying risk.

What About Batting Averages?
In my report on performance of our  Fund Analyst Picks, I introduced batting averages as a way to track success. That is the percentage of Fund Analyst Picks that outperformed their category peers. As you may recall, our overall batting averages for the trailing three- and five-year periods ended June 30, 2006, were 67% for both periods.

I took a look at the batting averages for star ratings, too, and here's what I found. For 2002 through summer 2006 the average was 62%, for the class of 2003 it was 66%, and for 2004 it was 72%. So, on the whole, it looks like the Fund Analyst Picks have held up a little better than 5-star funds.