How Our Stock Star Ratings Have Performed

The Morningstar Rating for stocks recently passed the 10-year milestone, and we thought it would be a good time to take a step back and review our performance to see how we've done. Overall, we found that our ratings, particularly of firms we think have sustainable competitive advantages, have had a strong track record through the wide variety of market conditions during the last decade.

Performance Methodology
Before diving into the numbers, it is important to highlight exactly how we measure performance. Mark Twain has been attributed with the quotation, "There are three kinds of lies: lies, damned lies, and statistics." It certainly is easy to torture statistics to fit the story you would like to tell. But we prefer to take a more transparent approach. There are thousands of possible ways to look at the performance of research ratings, and of course, we must choose only a small number. We have chosen what we believe to be common-sense, easy-to-understand approaches that accurately and rigorously assess the value of what we do.

To that end, we build hypothetical portfolios of stocks that are constructed with some simple rules that are based on our ratings. We then look at the performance of these portfolios over time. The hypothetical portfolios below are reconstituted and rebalanced to equal weights using as-originally-reported ratings that were only available at that time (meaning, there can be no look-ahead bias).

This method is not without its own idiosyncrasies, however. Rebalancing hundreds of stocks to equal weights on a daily basis is not something a rational investor would do--transaction costs would be prohibitive. In addition, most common indexes, like the S&P 500, are rebalanced and reconstituted on a much less frequent basis. Trimming your winners and boosting your losers on a more frequent time scale can have such a profound effect on performance that it's futile to compare our hypothetical performance portfolios with any of these indexes (for our performance that is directly comparable to common indexes, jump to the "Investable Strategies" section below). Also, just because the portfolio uses equal weights doesn't mean it is always equally concentrated. The number of stocks with a given star rating fluctuates a lot over time. In fact, at the bottom of the market in March 2009, we had only roughly 30 1-star stocks, compared with more than 600 5-stars at that time. In periods of high concentration, each stock can have a much larger effect on the overall portfolio performance. This contrasts with common benchmarks that typically have a static number of holdings at all times.

Figure 1: Trailing Period Total Returns by Strategy
	Buy at 5,   Sell at 3	5-Star	4-Star	3-Star	2-Star	1-Star
Since Inception*	9%	11%	11%	8%	4%	9%
Trailing 10-Year*	15%	20%	14%	12%	9%	17%
Trailing 5-Year*	10%	17%	8%	5%	-1%	12%
Trailing 3-Year*	13%	16%	12%	10%	8%	4%
Trailing 1-Year	20%	17%	21%	18%	10%	18%
Trailing 3-Month	5%	4%	7%	5%	1%	4%
*Annualized, Data from Morningstar as of 12/31/2012.

In our Buy 5-Star, Sell 3-Star strategy, stocks enter the portfolio when they hit a 5-star rating and don't exit until they hit 3 stars. Our 5-star strategy works similarly, except it is equivalent to a Buy 5-Star, Sell Not-5-Star strategy. Meaning, stocks enter the 5-Star strategy when they hit 5 stars and don't exit until their star rating is no longer 5 stars. This logic is applied to our 4-, 3-, 2-, and 1-Star strategies, as well. As shown in Figure 1, over nearly all trailing periods, our 5- and 4-Star strategies outperform our 2- and 1-Star strategies, as we would hope. Things get a little more interesting when we move beyond just the star rating and begin incorporating our other ratings, as well.

Figure 2: Trailing 10-Year Annualized Total Returns by Star and Moat Ratings
		Wide Moat Only	Narrow Moat Only	No Moat Only
5-Star		19%	18%	18%
4-Star		11%	16%	14%
3-Star		8%	12%	13%
2-Star		10%	9%	7%
1-Star		-4%	6%	17%
Data from Morningstar as of 12/31/2012.

You'll notice two things from the performance in Figure 2. First, during the last 10 years, our 5 and 4 stars have outperformed our 1 and 2 stars. The outperformance is modest, but we'll take it. Second, and perhaps more interesting, is the huge boost to performance you get by limiting your universe to wide- and narrow-moat stocks. Not only do our 5-star stocks outperform by more in this higher-quality universe, our 1 stars underperform by more, as well! This implies that the information encoded in our star ratings is more accurate for higher-quality stocks.

The intuition behind that statement is a little tricky at first. Owning high-quality undervalued stocks appeals strongly to any value investor, but can it be true that you would do better to avoid high-quality overvalued stocks than low-quality overvalued stocks? According to our research, this is exactly the case because our moat ratings are indicative of the accuracy of our valuations.

Figure 3: Moat Ratings Over Time Total Return Indexes

 Data from Morningstar as of 12/31/2012.

Figure 3 further illustrates the idea that using moat ratings alone will not yield outsized returns. In fact, the wider the moat, the lower the return. This is because the market partially recognizes the lessened risk associated with wide-moat firms and compensates investors accordingly. However, what is not as obvious from Figure 3 is the degree to which risk is lowered by paying attention to moat.

Figure 4: Maximum Drawdown by Star and Moat Rating Since Inception
	Full Universe	Wide Moat Only	Narrow Moat Only	No Moat Only
5-Star	-65%	-41%	-62%	-64%
4-Star	-62%	-55%	-55%	-69%
3-Star	-62%	-63%	-54%	-67%
2-Star	-64%	-48%	-56%	-69%
1-Star	-78%	-67%	-90%	-80%
All Star Ratings	-59%	-49%	-54%	-65%
Data from Morningstar as of 12/31/2012.

When examining a portfolio over a long time period, maximum drawdown can be a useful risk measure. It measures the largest peak-to-trough loss your portfolio would have experienced. It cuts through the noise that is often embedded in a risk measure such as standard deviation where a jittery stock price, or even a rapidly increasing stock price, can contribute to risk.

Based on this measure, it's clear that paying attention to a company's moat helps from a risk perspective because an investor would've faced significantly smaller drawdowns. What might even be more interesting is that paying attention to our star ratings can lower risk, as well. In every moat bucket, the 1-star stocks faced steeper drawdowns than the 5-star stocks. This makes sense, of course, given that the market is a weighing machine in the long run and stocks eventually converge to their fair value.

Investable Strategies
As illustrative as hypothetical portfolios can be, they can still never fully reflect the performance an individual would have achieved using our investment research with real money on the line. As a result, we also present the results of real-money portfolios, managed by our strategists, using our best practices. An investor following along could have realized the results of any of the portfolios represented below.

Figure 5: Total Returns of Morningstar's Real-Money Portfolios
	S&P 500	Tortoise	Hare	Dividend Builder	Dividend Harvest	Wide-Moat Focus
Inception Date	6/18/2001	6/18/2001	6/18/2001	1/7/2005	12/29/2006	9/30/2002
Calendar Year  Returns
2001	-11.89%	9.11%	0.75%	N/A	N/A	N/A
2002	-22.10%	-1.61%	-23.94%	N/A	N/A	N/A
2003	28.69%	26.20%	26.28%	N/A	N/A	36.20%
2004	10.88%	13.05%	26.91%	N/A	N/A	27.82%
2005	4.91%	7.98%	3.17%	N/A	N/A	4.67%
2006	15.79%	13.70%	22.04%	N/A	N/A	17.72%
2007	5.49%	1.62%	5.25%	-2.84%	1.78%	-1.31%
2008	-37.00%	-22.16%	-32.35%	-18.27%	-31.10%	-19.58%
2009	28.26%	19.40%	45.60%	6.17%	34.04%	49.72%
2010	15.06%	9.74%	14.27%	14.43%	27.97%	8.57%
2011	2.11%	7.35%	1.42%	11.47%	17.70%	6.61%
2012	16.00%	14.93%	20.80%	14.85%	12.54%	24.50%
Trailing Returns
Since Inception*	3.41%	7.90%	7.26%	5.56%	8.07%	14.87%
Trailing 10-Year*	7.10%	8.40%	11.35%	N/A	N/A	13.68%
Trailing 5-Year*	1.66%	4.70%	6.64%	4.92%	9.39%	11.23%
Trailing 4-Year*	14.58%	12.75%	19.49%	11.68%	22.79%	20.63%
Trailing 3-Year*	10.87%	10.63%	11.87%	13.57%	19.23%	12.95%
Trailing 2-Year*	8.83%	11.08%	10.69%	13.15%	15.09%	15.20%
Trailing 1-Year	15.97%	14.93%	20.80%	14.85%	12.54%	24.45%
Trailing 3-Month	-0.38%	0.82%	2.09%	-1.44%	-1.77%	3.76%
*Annualized, Data from Morningstar as of 12/31/2012.

Paul Larson manages the Tortoise and Hare portfolios and they are available via Morningstar's StockInvestor newsletter. Josh Peters runs Dividend Builder and Dividend Harvest, and these are available in Morningstar's DividendInvestor newsletter. The Wide Moat Focus Index tracks the 20 cheapest (based on price/fair value ratio) wide-moat stocks.