The Morningstar Quantitative Rating™ for funds has officially adopted the changes made to the Morningstar Analyst Rating™.
The new methodology’s key changes, which took effect on Nov. 21, include:
- Funds’ price and performance information are captured in systematic ways that removed the need for those pillars. Instead, the new framework will focus on the more predictive pillars: people, process, and parent.
- Pillar ratings will utilize a five-point scale (High, Above Average, Average, Below Average, Low), which will help clarify the overall rating and make comparing funds easier.
- Each of the pillar models will be enhanced to improve the model’s ability to match analyst rating calls. Since the MQR relies on historical data to predict pillar scores, simply improving the input data can increase the model’s accuracy. In total, we added 13 new input-data points to the model and, in doing so, improved the out-of-sample accuracy for each pillar.
Below, we break down what these changes look like across pillars.
The new manager data points in the People Pillar
Under the original methodology, the MQR People Pillar used fund data as a proxy for data about the people on the fund’s portfolio-management team. For example, the model looked at the tenure across the fund but not the management team’s experience in the industry.
With the most recent changes, MQR’s People Pillar predictions are based on data points that directly measure the fund managers. To do so, we constructed a suite of new manager data points which combines the fund and manager-history data. Now, we can evaluate each member of a portfolio-management team on the dimensions of performance, experience, and turnover.
A couple examples of the new data points are:
- Manager Track Record. This data point captures a specific manager’s skill. For each month, we calculate their average ex-category average return across their portfolio of funds managed. From there, we can generate trailing three- and five-year excess return figures.
- Manager Experience. This data point captures a manager’s experience by measuring the amount of time since a fund manager was first listed as a portfolio manager in Morningstar’s database. With this calculation, a manager is given credit for having industry experience even if they just joined a fund or a fund is new.
The combination of these data points means that if a fund is new but has managers who are experienced industry professionals with a long-term track record of success, the fund can get credit for the management quality.
The chart below ranks the importance of each variable in the People Pillar. Manager Excess Return 5 Year is the most important variable for assigning a Negative People score, whereas Manager Experience is the sixth-most important variable for assigning a Positive People score. This shows how the new manager data points are impactful for rating the People Pillar.
Introducing the new data points in the Parent Pillar
The methodology enhancement also led us to expand our suite of firm data points. The new data points for the Parent Pillar include:
- % Funds Team Managed,
- % Funds with a Management Change,
- Average Number of Months Since Manager Change,
- Average Portfolio Transparency, and
- Firm Age.
Consider the % Funds Team Managed data point, which is a firm-level quantification of how Morningstar analysts evaluate a fund and a firm’s succession planning.
When we look at how analysts have historically rated Parent firms, the % of funds team managed is the ninth- and eighth-most important variable for assigning a Positive or Negative pillar score, respectively. The full rankings of data points are shown in the exhibit below.
How the Morningstar Quantitative Rating improves pillar accuracy
Since the MQR model is constructed to mimic the rating-assignment behavior of our manager research staff, we tested how accurately the Quantitative Pillar Ratings match the Analyst Pillar Ratings.
To evaluate the accuracy, we look at both the Positive and Negative directions.
- For Negatives, we look at how well the MQR can match an analyst assigning a Below Average or Low pillar score. Our accuracy here is near perfect, consistently above 95.0% for all three pillars. This makes sense, as the attributes of poorly analyst-rated funds are obvious: high fees, manager turnover, and bad performance.
- For Positives, we look at how well the MQR can match an analyst assigning an Above Average or High pillar score. Our accuracy for positively rated funds varies by pillar. Parent is the best-performing of the Positive pillars, with an accuracy score of 95.0%. This is unsurprising, as input data points about firms are the most widely available in the model.
Of all the tests, the People Positive and Process Positive ratings have the lowest accuracy, at 81.4% and 75.7%, respectively. This is a gain in accuracy of 4.3% and 1.3%, respectively, from the original methodology. The chart below maps out this accuracy across pillars.
There are very few instances of large disparities between the Analyst Pillar Ratings and the Quantitative Pillar Ratings. When the two systems disagree, it is often because the Quantitative system skews to Average.
A new lens for the Morningstar Quantitative Rating™
The enhancements to the Morningstar Quantitative Rating bring the system in closer alignment with the Morningstar Analyst Rating™.
Financial professionals using MQR benefit from the finer granularity of the pillar rating and the new data points advancing model accuracy. Its precision is helping us balance the desires to increase accuracy, avoid overfitting, and achieve strong future performance.