A solid start for the Morningstar Quantitative Rating.
By Romir Agarwal
Funds
The Efficacy of Morningstar Ratings
Read Time: 4 Minutes
The Morningstar Quantitative Rating methodology for managed investments has delivered solid performance since its launch in June 2017.
Strategies with Gold, Silver, or Bronze ratings have outperformed their Morningstar Category averages. The Quantitative Rating model has also proved to be stable over the past five years.
As markets continue to be tumultuous in 2023, we expect the ratings to remain resilient as the model goes through its first prolonged bear-market cycle.
Newsletters
Smart Investor
A weekly rundown of Morningstar’s latest on the big trends driving the markets and new investment opportunities.
Background on Morningstar Quantitative Research
The Quantitative Rating uses a machine-learning model trained on Morningstar fund analysts’ past rating inputs and the data supporting those decisions. The model extends the reach of Morningstar's research to strategies that our analysts don’t cover.
The Quantitative Rating has the same Morningstar Medalist iconography as the Morningstar Analyst Rating and can be interpreted in the same way—the higher a fund's rating, the higher the confidence that it will outperform over a full market cycle.
The Morningstar Quantitative Ratings model is expanding coverage to keep up with the growing managed investments market.
The Quantitative Ratings launched for open-end and exchange-traded funds in the United States. Since 2017, the ratings have expanded across investment types, such as separate accounts and variable annuity accounts, and into new regions like Europe, Australia, South America, and Asia.
Evaluating the Efficacy of Morningstar Ratings
Do medalist strategies outperform Negative-rated strategies?
The portfolio study creates five portfolios that group the fund universe based on their ratings. Ratings buckets include strategies from different asset classes and categories. For example, the Gold portfolio shows the performance versus a category investing in every Gold-rated fund each month on an equal-weighted basis. Returns show the rated strategies’ returns minus the appropriate category average return.
Over different time horizons, the Analyst Rating and Quantitative Rating medalist products outperformed the Negative-rated products.
The ratings sort the universe, and the trend is more prominent for the Quantitative Ratings. This is partly due to the larger volume of share classes that get Quantitative Ratings compared with Analyst Ratings.
Exhibits 3 and 4 depict the performances of different ratings portfolios in discrete time periods. The trend persists across different time horizons; both ratings models show that medalist strategies outperformed the Negative-rated strategies.
In the three-year period and the nearly five-year period since inception, medalist products outperformed the Negative-rated products compared with their category peers in both ratings models.
The association of higher returns to higher rankings is more pronounced in Quantitative Ratings compared with Analyst Ratings. The Gold- and Silver-rated strategies outperformed their category peers by at least 0.5% in both time periods.
The shorter one-year period shows that medalist strategies haven’t been immune to the recent market volatility across both models. This has led to similar or worse performance compared to the Neutral- and Negative-rated strategies.
Are Morningstar ratings consistently effective?
An event study measures the typical experience an investor would have had using our ratings system on any given day across different holding periods. This study looked at the margin of outperformance among different ratings over various periods to see if managed investment products with the same rating outperformed over time. And if so, by how much.
The event study shows that investors in medalist strategies could experience this outperformance and that it isn’t the result of timing or coincidence.
Exhibit 5 below shows that buy-and-hold investors in time periods from one month to three years would have experienced gains from following the Quantitative Rating since its June 2017 launch. In all three-year holding periods, regardless of start date, Gold-rated strategies outperformed category peers by more than 2.5%. Negative-rated strategies underperformed category peers by nearly 1.0%.
Exhibit 6 shows that buy-and-hold investors in all time periods of one month to five years also would have experienced gains investing in strategies with Analyst Ratings of Gold.
In the five-year holding periods, regardless of start date, Gold-rated strategies outperformed category peers by approximately 1.00%. Negative-rated strategies underperformed category peers by nearly 6.0%.
This shows that the Morningstar Rating has served investors well in recent years. We now await results from the turbulent market conditions.
How stable are Morningstar ratings?
To measure ratings stability, we tracked how many strategies changed ratings groups each month over the past year. We used the number of changes to create a transition matrix that determines the stability of each rating group. Morningstar assigned Analyst Ratings on a yearly basis, but this study spreads the same rating over a 14-month period until an updated rating is found in future months or in past months.
Since the methodology update for the Quantitative Ratings in 2019, the ratings have been stable month over month. Exhibit 7 shows the monthly transition figures for the past year.
When a fund receives a Negative rating, we would expect that it has only a 0.05% probability of receiving a Bronze rating a year later and no probability of receiving a Silver or higher rating. Similarly, when a fund receives a Gold rating, we would expect that the fund has a 91.04% chance of retaining its Gold rating one year later and only a 0.32% probability of receiving a Neutral rating and a 0.01% probability of receiving a Negative rating a year later.
In other words, we tend to stick to our guns when quantitatively rating strategies.
Similarly, in Exhibit 8, the Morningstar Analyst Ratings are extremely stable across the rating groups. The Quantitative Rating can closely follow Analyst Ratings when it comes to predicting fund ratings, signaling higher accuracy.
The Morningstar Quantitative Rating for managed investment products is off to a good start as a complement to the Morningstar Analyst Rating. The rating is a reliable input for investors to use when Morningstar analysts don’t cover a fund.