Skip to Content
Fund Spy

The Star Rating Performs Well but Our Picks Are Better

Our annual update shows the star rating and picks keep chugging along.

This article originally appeared in Morningstar FundInvestor.

In our latest evaluation of the Morningstar Rating for funds, we find that it continues to be modestly successful in forecasting performance.

The star-rating methodology was reconstituted in the summer of 2002, and each year since then we have produced a study examining total returns, relative performance, and risk-adjusted performance at the end of June. Then we look at how it did at predicting those factors over the ensuing years so that we can understand how investors using the rating to aid their decisions would have fared.

How the Rating Works
It's important to note that the star rating is a purely quantitative measure, updated every month, that's based upon past returns and volatility. Specifically, it measures load-adjusted total returns from the past three-, five-, and 10-year periods. Then it adjusts for risk so that high-risk funds are taken down a peg or two, and low-risk funds are moved up a bit. The risk-adjusted measure is then graded on a curve within a category. We give 5 stars to 10% of the funds, 22.5% earn 4 stars, 35% take the 3-star rating, 22.5% take 2 stars, and 10% earn 1 star.

You'll notice many important things aren't included in the rating, such as our analysts' opinion, the manager's record, and expense ratios. The rating is a handy measure of past risk-adjusted performance, but it is not a recommendation and it certainly is not the whole story. Rather, it's a handy way to narrow your search for a fund and a way to keep tabs on the funds you own. The data in our study show why.

How the Rating Performed
In the aggregate, 5-star funds beat 4-star funds, and 4 beat 3, and so on. (See table.) This is true of returns, ensuing star ratings, and batting average. For example, using ratings of domestic-equity funds from 2003, 5-star funds returned an average of 9.90% annualized over the next five years compared with 9.20% for 4-star funds, 9.04% for 3-star funds, 9.22% for 2-star funds, and 8.70% for 1-star funds. The gap between each star rating group is hardly the Grand Canyon, but by the time you get out to 1-star funds, which lag the 5-star group by 120 basis points per year, that's a pretty big deal. The gap for the class of 2003 was smallest for international equity (63 basis points) and largest for balanced (313 basis points).

For 5-year star ratings over the next five years, 5-star funds earned an average of 3.07 stars compared with 2.73 for 1-star funds--again a modest differential.

The same is true for batting averages--a measure of what percent of funds beat their peer group averages, though in our example of domestic-equity ratings from 2003, there was a surprise. We saw 54% of 5-star funds beat their peers over the next five years. That figure declined to 42% for 3-star funds, but then picked up slightly for 2-star funds to 44%, and 1-star funds had a nifty 53% batting average. What happened? Extinction. Many 1- and 2-star funds were liquidated over the five years so they didn't show up to bring the batting average down. However, if you instead ask what percentage of funds survived and beat their peer group, the figures make more sense: 49% of 5-star funds met that hurdle, while 43% of 4-star funds did, down to 33%, 30%, and 36% for the bottom three rating groups.

How the Rating Performed Combined with a Cost Screen
I can think of a number of things that might improve the star rating's predictive power, but the one I'm certain of is factoring in expense ratios. Slicing the 5-star groups to include just those funds in the cheapest quartile of their category led to higher three- and five-year star ratings over the next three or five years 80% of the time. Factoring out the survivorship bias, the success ratio was improved 90% of the time when the expense screen was added in. In other words, if you buy funds that have the cheapest-quartile costs and 5 stars, you have a much greater chance of getting a mutual fund that will survive and outperform its peers.

Why would this be the case when expense ratios are indirectly factored into the star rating because it uses after-expense returns? There are a couple of reasons. First, the star rating requires only three years' worth of performance, yet an equity fund is something you should hold for 10 to 20 years. Thus, the negative compounding effect of expense ratios is understated. Over time, the difference between a 0.60% expense ratio and a 1.25% expense ratio can lead to a very big sum after 20 years, but not so much for three years. In addition, expense ratios change over time, and the fund's most current expense ratio, rather than past expense ratios, is the best indicator of where it will be in the future. For example,  Primecap Odyssey Aggressive Growth  (POAGX) started out with a 1.25% expense ratio, but now it's down to 0.78%.

Star Ratings Versus Fund Analyst Picks
Our  Fund Analyst Picks are forward-looking fundamental-driven selections whereas the star rating is a backward-looking performance-driven measure; yet it's natural to want to see how the two perform versus one another. To do that, I looked at our picks from June 2003 and then looked to see how they performed over the ensuing five years.

Score one for the humans! Our Fund Analyst Picks topped 5-star funds in all asset classes. In domestic equity, the picks enjoyed a success ratio of 64% versus 49% for 5-star funds. In international equity it was 66% for picks and 54% for 5-star funds. In balanced, the tally was 100% for picks versus 59% for 5 stars. In taxable-bond, picks won 81% to 70%, and in munis it was 100% to 78%.

How about if I limited our picks to just the cheapest quartile? Performance gets even better and is also better than that of low-cost 5-star funds. Although our picks have lower costs than 5-star funds and the typical choice of fund investors, it's clear that we, too, should be more disciplined in our selection. Except for the two asset classes where we already had 100% success rates, our cheaper picks fared better than our pricier ones. Among domestic-stock fund picks, our success ratio popped from 64% to 79% when limited to the cheapest quartile. (The picks from outside the cheapest quartile actually had success rates below 50%.) In international picks, the improvement was marginal while in taxable bonds it went from 82% to 91%, and once more our picks from outside the cheapest quartile had a success rate below 50%.

Conclusion
The star rating isn't a complete solution but rather an aid that helps you to narrow the field and improve your chances for success. Our Fund Analyst Picks do an even better job of that than the star rating, and because there are fewer picks than 5-star funds, it also narrows the list to fewer choices.

In addition, it's more apparent than ever that low costs are another vital piece. No matter how you evaluate mutual funds, you need to incorporate costs into the equation.

 

 

 

FundInvestor Newsletter
Morningstar FundInvestor is
a 48-page newsletter dedicated to helping investors pick great mutual funds, build winning portfolios, and monitor their funds for greater
gains. This one-year subscription consists of 12 monthly issues, a Reader's Guide, and four free investing reports.
Learn more.
 $115.00 for 12 Print Issues $95.00 for 12 PDF Issues
  

Transparency is how we protect the integrity of our work and keep empowering investors to achieve their goals and dreams. And we have unwavering standards for how we keep that integrity intact, from our research and data to our policies on content and your personal data.

We’d like to share more about how we work and what drives our day-to-day business.

We sell different types of products and services to both investment professionals and individual investors. These products and services are usually sold through license agreements or subscriptions. Our investment management business generates asset-based fees, which are calculated as a percentage of assets under management. We also sell both admissions and sponsorship packages for our investment conferences and advertising on our websites and newsletters.

How we use your information depends on the product and service that you use and your relationship with us. We may use it to:

  • Verify your identity, personalize the content you receive, or create and administer your account.
  • Provide specific products and services to you, such as portfolio management or data aggregation.
  • Develop and improve features of our offerings.
  • Gear advertisements and other marketing efforts towards your interests.

To learn more about how we handle and protect your data, visit our privacy center.

Maintaining independence and editorial freedom is essential to our mission of empowering investor success. We provide a platform for our authors to report on investments fairly, accurately, and from the investor’s point of view. We also respect individual opinions––they represent the unvarnished thinking of our people and exacting analysis of our research processes. Our authors can publish views that we may or may not agree with, but they show their work, distinguish facts from opinions, and make sure their analysis is clear and in no way misleading or deceptive.

To further protect the integrity of our editorial content, we keep a strict separation between our sales teams and authors to remove any pressure or influence on our analyses and research.

Read our editorial policy to learn more about our process.