Skip to Content

How Morningstar's Analyst Ratings for Index Funds Have Performed

Morningstar Analyst Ratings have done a decent job guiding investors toward better-performing index funds.

Securities In This Article
SPDR® S&P International Dividend ETF
Vanguard Small Cap Index Inv
iShares S&P 500 Index Institutional
Allspring Index A
BNY Mellon S&P 500 Index

A version of this article previously appeared in the November 2019 issue of Morningstar ETFInvestor. Click here to download a copy.

The Morningstar Analyst Rating attempts to distill all relevant information about a fund's strengths and weaknesses into a signal of that investment's attractiveness relative to an appropriate benchmark. That's a tall order, and we don't always get it right. It's important to measure the performance of the Analyst Ratings, not only to gauge their efficacy, but also to identify ways we might improve them.

A previous study published by Jeff Ptak, Morningstar's global head of manager research, showed that the Analyst Ratings for mutual funds performed fairly well from December 2011 to March 2019. Performance was particularly strong among Gold- and Silver-rated funds. However, Bronze-rated funds didn't clearly perform better than lower-rated funds, and most of the funds covered in this study were actively managed. The analysis that follows takes a closer look at the performance of the Analyst Ratings for index funds.

Before diving into the data, some caveats are in order. Our sample of rated index mutual funds and exchange-traded funds is small and has a short history, so there's limited data to evaluate. The Analyst Ratings are meant to project funds' long-term performance, but Morningstar didn't start rating ETFs until November 2016. The sample of index mutual funds rated prior to that date is much smaller, which makes it difficult to draw conclusions about the ratings' longer-term performance. However, despite these qualifications, the available data still provide some useful information.

Study Design This analysis included the oldest share classes for all index funds (both mutual funds and ETFs) that Morningstar rated on three separate dates:

  • Sept. 30, 2014: This includes only index mutual funds, but it provides a longer lookback.
  • Nov. 1, 2016: This is the date when Morningstar launched Analyst Ratings for ETFs.
  • Sept. 30, 2017: This greatly increases the number of funds included.

I grouped the funds by their ratings on each date and calculated the average (capital asset pricing model) alpha for the funds in each ratings bucket from that point through September 2019. Alpha indicates whether a fund has beaten a benchmark after controlling for differences in market risk.

It may seem strange to expect index funds to deliver this type of outperformance (a positive alpha), as many of them passively replicate the composition of the market or a segment of the market. So, they often look a lot like the indexes assigned to benchmark their respective Morningstar Categories. For these traditional market-cap-weighted index funds, we used the category average as the benchmark in the alpha calculation. This is usually a lower hurdle than the category index because it's based on the average net-of-fees performance of all funds in the category, and most managers fail to earn back their fees. However, that's precisely the reason low-cost traditional index investing makes sense. If index Morningstar Medalists beat their category average, after adjusting for risk, we count it a success.

The appropriate benchmark for strategic-beta funds is different.[1] These are index funds that make active bets, usually in an attempt to beat the market. So, it's best to gauge their success based on their alphas relative to their respective category indexes.

Gold-, Silver-, and Bronze-rated funds should deliver positive alphas, and Gold-rated funds should perform the best. Neutral-rated funds should perform in line with or moderately lag an appropriate benchmark, while Negative-rated funds should do the worst.

Results Overall, the Analyst Ratings for index funds have done a decent job. That said, there is room for improvement, as Exhibits 1 and 2 show. As expected, medalists delivered positive alphas on average, and higher-rated strategic-beta funds tended to post higher alphas (with the exception of one Negative-rated fund).

However, the alpha rankings for the traditional index funds were less clean. Gold-rated funds tended to do the best in the most recent two sample periods, which cover more funds than the first. But Neutral-rated funds did better than every other cohort in the first period (though the sample size is small) and better than Bronze-rated funds in the second. Silver-rated funds didn't keep pace with Bronze-rated ones in the first and third periods. This illustrates that the results are sensitive to the period in focus.

Regardless of the rating, the index funds with an Analyst Rating were more likely to have positive alphas than unrated funds. That's because we tend to cover large, broadly diversified, low-cost funds. These tend to do better than more obscure, niche, and expensive funds.

Misses Expensive S&P 500 Funds One of our early mistakes in 2014 was assigning Neutral ratings to three S&P 500 trackers that were expensive at the time: Wells Fargo Index WFILX, iShares S&P 500 Index BSPIX, and BNY Mellon S&P 500 Index PEOPX. The Neutral rating was intended to signal that these funds were charging considerably more than the cheapest S&P 500 trackers and that there was no reason to pay up for them. However, this took our eye off what the ratings are really supposed to convey, which is how well a fund will do relative to its benchmark (the large-blend category average in this instance) over the long term. The S&P 500's strong performance more than made up for these funds' excessive fees, which hurt the ratings' efficacy.

The new ratings methodology largely resolves this issue by separating our assessment of the investment's gross-of-fees alpha potential from its fee, which will be mechanically subtracted from that estimate to arrive at the overall rating. This should more effectively balance fees against funds' gross performance potential. Under the new ratings framework, funds that track the same index should have similar gross-of-fees alpha estimates.

Wells Fargo Index and iShares S&P 500 Index currently carry Bronze ratings, while the lowest-cost S&P 500 trackers carry Gold ratings. However, these have not yet been rated under the new methodology.

Dow Jones Industrial Average Neutral-rated SPDR Dow Jones Industrial Average ETF's DIA strong performance also hurt ratings efficacy from November 2016 through September 2019. This has been one of the best-performing funds in the large-value category, but it really isn't a value fund in the traditional sense. It's based on a price-weighted index of 30 industrial leaders that has been around since 1896. Price weighting doesn't make much sense as the basis for an investment portfolio. It gives larger weightings to stocks with higher share prices, regardless of how many shares are outstanding. So, stock splits can reduce the firm's weighting in the portfolio, even though that's strictly a paper transaction.

Price weighting can also limit the stocks available to the index. For example, thanks to Alphabet's GOOG high stock price, it would account for more than 20% of the index if it were included, which could distort the Dow's representation of the U.S. market.

Concerns about the fund's concentration and price weighting have limited its rating to Neutral. However, the committee that runs the index has been able to deliver strong performance and keep risk in check by focusing on market leaders and keeping the index representative of the market's sector composition. Despite its problems, this fund's performance has been highly correlated with that of the S&P 500. That said, concentration is a very real risk for DIA. While we stand behind the fund's Neutral rating, we will continue to assess whether the fund's quirks are significant enough to keep it from beating the category average over the long term.

SPDR S&P International Divided ETF SPDR S&P International Divided ETF DWX is the only Negative-rated fund included in this study, and it posted a positive alpha over the trailing two years through September 2019. The Negative rating stems from the fund's aggressive pursuit of dividend yield, which can lead it to risky firms with deteriorating fundamentals and negative momentum. While it happened to beat the MSCI ACWI ex USA Value Index over the past two years, we don't think that's representative of the type of performance the fund will likely deliver over the long term. Its risk will likely come back to bite over a full market cycle, as it has over the past decade.

Hits It's easy to focus on the misses, but we tended to make more good calls than bad. There are lots of good examples, including Vanguard Dividend Appreciation ETF VIG, which is the only Gold-rated strategic-beta fund in the sample, and Gold-rated Vanguard Small Cap Index NAESX, which has consistently posted strong category-relative performance. There was also a string of strong-performing Silver-rated funds, including iShares Edge MSCI Minimum Volatility USA ETF USMV, iShares Edge MSCI USA Momentum Factor ETF MTUM, and Schwab U.S. Dividend Equity ETF SCHD.

We also called out some less-than-stellar funds, including Invesco Dynamic Large Cap Value ETF PWV and SPDR Bloomberg Barclays High Yield Bond ETF JNK, which both posted negative alphas after receiving their Neutral ratings.

Lessons Our record is far from perfect, but the Analyst Ratings have done a decent job guiding investors toward better-performing index funds. The new ratings methodology should improve ratings efficacy by separating the estimate of each fund's gross alpha potential from its cost, replacing a qualitative assessment of cost with an objective one. But there's more that we can do to improve. Some of our misses have shown that the ratings can get off track when we get hung up on portfolio construction details that aren't great (like price-weighting) when there isn't a clear link to performance. Above all else, the ratings should focus on how each pillar will likely affect future performance. The nitpicking should be done elsewhere.

It's hard to know which funds will win out in the short run. The Analyst Ratings probably won't be very helpful for that. But a strong investment process and low fee should pay off over the long term. Those factors will continue to drive the Analyst Ratings for index funds.

[1] This excludes market-cap-weighted value/growth funds.

Disclosure: Morningstar, Inc. licenses indexes to financial institutions as the tracking indexes for investable products, such as exchange-traded funds, sponsored by the financial institution. The license fee for such use is paid by the sponsoring financial institution based mainly on the total assets of the investable product. Please click here for a list of investable products that track or have tracked a Morningstar index. Neither Morningstar, Inc. nor its investment management division markets, sells, or makes any representations regarding the advisability of investing in any investable product that tracks a Morningstar index.

More in Funds

About the Author

Alex Bryan

Director of Product Management, Equity Indexes
More from Author

Alex Bryan, CFA, is director of product management for equity indexes at Morningstar.

Before assuming his current role in 2016, Bryan spent four years as a manager analyst covering equity strategies. Previously, he was a project manager and senior data analyst in Morningstar's data department. He joined Morningstar in 2008 as an inside sales consultant for Morningstar Office.

Bryan holds a bachelor's degree in economics and finance from Washington University in St. Louis, where he graduated magna cum laude, and a master's degree in business administration, with high honors, from the University of Chicago Booth School of Business. He also holds the Chartered Financial Analyst® designation. In 2016, Bryan was named a Rising Star at the 23rd Annual Mutual Fund Industry Awards.

Sponsor Center