How Morningstar's Analyst Ratings for Index Funds Have Performed
Morningstar Analyst Ratings have done a decent job guiding investors toward better-performing index funds.
A version of this article previously appeared in the November 2019 issue of Morningstar ETFInvestor. Click here to download a copy.
The Morningstar Analyst Rating attempts to distill all relevant information about a fund's strengths and weaknesses into a signal of that investment's attractiveness relative to an appropriate benchmark. That's a tall order, and we don't always get it right. It's important to measure the performance of the Analyst Ratings, not only to gauge their efficacy, but also to identify ways we might improve them.
A previous study published by Jeff Ptak, Morningstar's global head of manager research, showed that the Analyst Ratings for mutual funds performed fairly well from December 2011 to March 2019. Performance was particularly strong among Gold- and Silver-rated funds. However, Bronze-rated funds didn't clearly perform better than lower-rated funds, and most of the funds covered in this study were actively managed. The analysis that follows takes a closer look at the performance of the Analyst Ratings for index funds.
Before diving into the data, some caveats are in order. Our sample of rated index mutual funds and exchange-traded funds is small and has a short history, so there's limited data to evaluate. The Analyst Ratings are meant to project funds' long-term performance, but Morningstar didn't start rating ETFs until November 2016. The sample of index mutual funds rated prior to that date is much smaller, which makes it difficult to draw conclusions about the ratings' longer-term performance. However, despite these qualifications, the available data still provide some useful information.
This analysis included the oldest share classes for all index funds (both mutual funds and ETFs) that Morningstar rated on three separate dates:
I grouped the funds by their ratings on each date and calculated the average (capital asset pricing model) alpha for the funds in each ratings bucket from that point through September 2019. Alpha indicates whether a fund has beaten a benchmark after controlling for differences in market risk.
It may seem strange to expect index funds to deliver this type of outperformance (a positive alpha), as many of them passively replicate the composition of the market or a segment of the market. So, they often look a lot like the indexes assigned to benchmark their respective Morningstar Categories. For these traditional market-cap-weighted index funds, we used the category average as the benchmark in the alpha calculation. This is usually a lower hurdle than the category index because it's based on the average net-of-fees performance of all funds in the category, and most managers fail to earn back their fees. However, that's precisely the reason low-cost traditional index investing makes sense. If index Morningstar Medalists beat their category average, after adjusting for risk, we count it a success.
The appropriate benchmark for strategic-beta funds is different. These are index funds that make active bets, usually in an attempt to beat the market. So, it's best to gauge their success based on their alphas relative to their respective category indexes.
Gold-, Silver-, and Bronze-rated funds should deliver positive alphas, and Gold-rated funds should perform the best. Neutral-rated funds should perform in line with or moderately lag an appropriate benchmark, while Negative-rated funds should do the worst.
Overall, the Analyst Ratings for index funds have done a decent job. That said, there is room for improvement, as Exhibits 1 and 2 show. As expected, medalists delivered positive alphas on average, and higher-rated strategic-beta funds tended to post higher alphas (with the exception of one Negative-rated fund).
However, the alpha rankings for the traditional index funds were less clean. Gold-rated funds tended to do the best in the most recent two sample periods, which cover more funds than the first. But Neutral-rated funds did better than every other cohort in the first period (though the sample size is small) and better than Bronze-rated funds in the second. Silver-rated funds didn't keep pace with Bronze-rated ones in the first and third periods. This illustrates that the results are sensitive to the period in focus.
Regardless of the rating, the index funds with an Analyst Rating were more likely to have positive alphas than unrated funds. That's because we tend to cover large, broadly diversified, low-cost funds. These tend to do better than more obscure, niche, and expensive funds.
Expensive S&P 500 Funds
One of our early mistakes in 2014 was assigning Neutral ratings to three S&P 500 trackers that were expensive at the time: Wells Fargo Index (WFILX), iShares S&P 500 Index (BSPIX), and BNY Mellon S&P 500 Index (PEOPX). The Neutral rating was intended to signal that these funds were charging considerably more than the cheapest S&P 500 trackers and that there was no reason to pay up for them. However, this took our eye off what the ratings are really supposed to convey, which is how well a fund will do relative to its benchmark (the large-blend category average in this instance) over the long term. The S&P 500's strong performance more than made up for these funds' excessive fees, which hurt the ratings' efficacy.
The new ratings methodology largely resolves this issue by separating our assessment of the investment's gross-of-fees alpha potential from its fee, which will be mechanically subtracted from that estimate to arrive at the overall rating. This should more effectively balance fees against funds' gross performance potential. Under the new ratings framework, funds that track the same index should have similar gross-of-fees alpha estimates.
Wells Fargo Index and iShares S&P 500 Index currently carry Bronze ratings, while the lowest-cost S&P 500 trackers carry Gold ratings. However, these have not yet been rated under the new methodology.
Dow Jones Industrial Average
Neutral-rated SPDR Dow Jones Industrial Average ETF's (DIA) strong performance also hurt ratings efficacy from November 2016 through September 2019. This has been one of the best-performing funds in the large-value category, but it really isn't a value fund in the traditional sense. It's based on a price-weighted index of 30 industrial leaders that has been around since 1896. Price weighting doesn't make much sense as the basis for an investment portfolio. It gives larger weightings to stocks with higher share prices, regardless of how many shares are outstanding. So, stock splits can reduce the firm's weighting in the portfolio, even though that's strictly a paper transaction.
Price weighting can also limit the stocks available to the index. For example, thanks to Alphabet's GOOG high stock price, it would account for more than 20% of the index if it were included, which could distort the Dow's representation of the U.S. market.
Concerns about the fund's concentration and price weighting have limited its rating to Neutral. However, the committee that runs the index has been able to deliver strong performance and keep risk in check by focusing on market leaders and keeping the index representative of the market's sector composition. Despite its problems, this fund's performance has been highly correlated with that of the S&P 500. That said, concentration is a very real risk for DIA. While we stand behind the fund's Neutral rating, we will continue to assess whether the fund's quirks are significant enough to keep it from beating the category average over the long term.
SPDR S&P International Divided ETF
SPDR S&P International Divided ETF (DWX) is the only Negative-rated fund included in this study, and it posted a positive alpha over the trailing two years through September 2019. The Negative rating stems from the fund's aggressive pursuit of dividend yield, which can lead it to risky firms with deteriorating fundamentals and negative momentum. While it happened to beat the MSCI ACWI ex USA Value Index over the past two years, we don't think that's representative of the type of performance the fund will likely deliver over the long term. Its risk will likely come back to bite over a full market cycle, as it has over the past decade.
It's easy to focus on the misses, but we tended to make more good calls than bad. There are lots of good examples, including Vanguard Dividend Appreciation ETF (VIG), which is the only Gold-rated strategic-beta fund in the sample, and Gold-rated Vanguard Small Cap Index (NAESX), which has consistently posted strong category-relative performance. There was also a string of strong-performing Silver-rated funds, including iShares Edge MSCI Minimum Volatility USA ETF (USMV), iShares Edge MSCI USA Momentum Factor ETF (MTUM), and Schwab U.S. Dividend Equity ETF (SCHD).
We also called out some less-than-stellar funds, including Invesco Dynamic Large Cap Value ETF (PWV) and SPDR Bloomberg Barclays High Yield Bond ETF (JNK), which both posted negative alphas after receiving their Neutral ratings.
Our record is far from perfect, but the Analyst Ratings have done a decent job guiding investors toward better-performing index funds. The new ratings methodology should improve ratings efficacy by separating the estimate of each fund's gross alpha potential from its cost, replacing a qualitative assessment of cost with an objective one. But there's more that we can do to improve. Some of our misses have shown that the ratings can get off track when we get hung up on portfolio construction details that aren't great (like price-weighting) when there isn't a clear link to performance. Above all else, the ratings should focus on how each pillar will likely affect future performance. The nitpicking should be done elsewhere.
It's hard to know which funds will win out in the short run. The Analyst Ratings probably won't be very helpful for that. But a strong investment process and low fee should pay off over the long term. Those factors will continue to drive the Analyst Ratings for index funds.
 This excludes market-cap-weighted value/growth funds.
Disclosure: Morningstar, Inc. licenses indexes to financial institutions as the tracking indexes for investable products, such as exchange-traded funds, sponsored by the financial institution. The license fee for such use is paid by the sponsoring financial institution based mainly on the total assets of the investable product. Please click here for a list of investable products that track or have tracked a Morningstar index. Neither Morningstar, Inc. nor its investment management division markets, sells, or makes any representations regarding the advisability of investing in any investable product that tracks a Morningstar index.
Alex Bryan has a position in the following securities mentioned above: USMV, MTUM. Find out about Morningstar’s editorial policies.