Checking Up on the Star Rating for Stocks

It's been a little over three years since we launched the Morningstar Rating for stocks, and it's time for another performance review. In a nutshell, we're still doing a solid job of helping investors find undervalued stocks that can be held for the long haul. There's still some room for improvement--there always is, in this business--but the results are quite positive overall.

As in previous reviews of our performance, we created a hypothetical portfolio that buys 5-star stocks, sells them if they hit 1 star, and hangs on to them otherwise. However, we've changed the way we calculate the aggregate return of this portfolio by using a measure called "internal rate of return," which takes into account the flow of money into and out of our hypothetical portfolio. We didn't make this change to spruce up our performance--in fact, we actually look better under the old method. Rather, we used internal rate of return to adjust for the fact that our 5-star portfolio will have different amounts of money invested at different times. This happens because the stock star rating is an absolute measure rather than a relative one, and the number of stocks rated 5 stars can vary quite a bit over time. (This is in contrast to the Morningstar Rating for funds, which grades funds on a curve and consistently awards 5-star ratings to the top 10% of funds based on risk-adjusted performance.)

Let's cut to the chase. Here's how our "buy at 5-star, sell at 1-star" portfolio has done.

Portfolio Performance ( % )
	Inception*	2004*	2003	2002	2001**
Buy at 5, Sell at 1	8.44	6.22	35.89	-22.52	27.17
S&P 500	-0.70	1.50	28.70	-22.10	-3.80
S&P 500 Equal-Weighted	3.00	0.41	38.70	-19.43	-2.20
* Through Sept. 30, 2004; ** Star rating launched Aug. 6, 2001

Overall, these are good results. We've kept ourselves ahead of the market since we launched the Morningstar Rating for stocks, and we're doing well through the third quarter of 2004. I'm particularly happy with the above-market results in 2003. In a year when risky, low-quality firms tended to outperform their more-durable peers, our analysts produced great returns by recommending wonderful long-term holdings like  Moody's (MCO),  Progressive (PGR), and  McDonald's (MCD).

It's worth noting that our margin of outperformance relative to an equal-weighted index of S&P 500 stocks has been smaller than versus the regular S&P 500, in which big-cap stocks are weighted more heavily. Mega-cap stocks have lagged the average stock over the past few years, which has given us a tailwind of sorts relative to the S&P 500, since our hypothetical portfolio of 5-star stocks in not weighted by market cap. If mega-caps lead the market again as they did in the late 1990s, our performance relative to the S&P 500 would likely suffer.

Breaking Things Down
I'll look at some specific good and bad calls later in this review, but first I thought it would be interesting to slice our aggregate performance results by economic moat and by risk. The table below shows the hypothetical performance of our "buy at 5-star, sell at 1-star" portfolio if we had only bought stocks of a certain risk or moat rating. (We assign an economic moat rating to every company we follow, which is our judgment of the strength of the firm's competitive position and ability to earn excess economic returns in the future.) 

Performance of 5-Star Stocks by Moat and Risk ( % )
Moat	Inception	2004	2003	2002	2001
None	-2.81	0.18	42.25	-32.22	23.64
Narrow	12.00	6.57	40.89	-19.08	33.56
Wide	15.74	7.12	28.07	-1.11	20.38
Risk Level	Inception	2004	2003	2002	2001
Above Average	9.42	8.06	51.19	-23.39	35.37
Average	5.79	6.75	36.33	-26.48	26.33
Below Average	15.05	5.31	32.42	-4.46	21.47

In general, we've done much better at recommending lower-risk, wide-moat stocks than risky, no-moat companies. In fact, our wide-moat 5-star stocks have generated what I consider to be some very impressive results over the past three years. However, our no-moat 5-star stocks haven't kept pace with the market since inception, though they did do very well in 2003 when low-quality stocks were all the rage on Wall Street. A rising tide lifts all boats, after all.

There's also a lot more year-to-year variation in the performance of our no-moat stocks relative to narrow and wide-moat stocks. To some extent, this is simply a reflection of the fundamental difference between no-moat firms and companies with some kind of competitive advantage--the performance of no-moat firms is more variable in both good times and bad times, because their businesses are so poorly insulated from competition.

Of course, these results also highlight something useful about our track record, which is that we've given no-moat stocks the benefit of the doubt more often than we should have. I expect that over time, our long-term focus means that we'll likely do a better job at making stock calls on higher-quality companies, but our performance valuing and recommending lower-quality stocks is nonetheless something we need to work on.

Consistency Counts
Few--if any--investors are going to buy every single 5-star stock that we recommend. So, I also looked at our "batting average," which measures how often our 5-star recommendations do better than an investment in an index made at the same time. In other words, how often do our 5-star stocks beat the market? We think this is an important metric for our subscribers to know, because aggregate results alone can be pumped up by just a few good calls. Knowing our batting average helps us--and you--understand how consistent we are.

As you can see, we're doing all right on this front, though not nearly as well as we are in terms of aggregate returns. Also, our batting average is lower relative to an equal-weighted index (which our 5-star stocks have beaten 50% of the time) than it is relative to the market-cap weighted S&P 500 (which our 5-star stocks have beaten 55.24% of the time).

Overall Batting Average of 5-Star Stocks vs. the Market ( % )
	Inception	2004	2003	2002	2001
S&P 500	55.24	56.93	68.75	51.42	55.92
S&P 500 Equal-Weighted	50.00	51.09	40.63	47.17	53.55

Breaking out our batting averages by moat, we see results that are similar to those for our return performance by moat. We're more consistent picking narrow- and wide-moat stocks than we are recommending no-moat stocks, though this effect is smaller relative to an equal-weighted index than relative to a market-cap-weighted index. Again, the effect of the past few years' bear market in large caps is very visible.

Batting Average of 5-Star Stocks by Moat vs. S&P 500 ( % )
Moat	Inception	2004	2003	2002	2001
None	46.43	46.88	100.00*	43.18	47.52
Narrow	62.08	60.26	75.00	60.00	63.38
Wide	59.80	59.26	57.14	55.26	69.57

Batting Average of 5-Star Stocks by Moat vs. S&P 500 Equal-Weighted ( % )
Moat	Inception	2004	2003	2002	2001
None	44.64	40.63	100.00*	43.18	45.54
Narrow	56.25	53.85	37.50	58.67	60.56
Wide	47.06	55.56	35.71	34.21	65.22
* Only three no-moat stocks were rated 5 stars during 2003.

Putting these consistency figures together with the performance results, I think it's safe to say that we've simply done a better job overall with wide- and narrow-moat 5-star stocks, and that we can get better at recommending no-moat stocks. Still, even our no-moat picks lagged the market by only a small margin, and the narrow- and wide-moat stocks that we've recommended have done extremely well. On balance, this is a solid track record, and I think that our approach of buying solid companies at decent prices is working well.

Hits and Misses
Where have we excelled and where have we stumbled specifically? Over the past year, our bullish take on the  Chicago Mercantile Exchange (CME) stands out head and shoulders above the pack. Analyst Mike Ford-Taggart stuck his neck out with a very bold call on this wide-moat financial exchange just about a year ago, when the stock was trading for just $66; the shares finished the third quarter at around $150. (They've moved up even further since then.) Biotech  Biogen IDEC (BIIB), real-estate services firm  Jones Lang LaSalle (JLL), and GPS-gadget manufacturer  Garmin (GRMN) have also all appreciated very nicely from their 5-star prices over the past year--they're up 82%, 53%, and 44%, respectively. All three are solid businesses that we think still have bright futures.

Of course, we've had our share of misses.  Netflix (NFLX),  Level 3 Communications , and  Krispy Kreme Doughnut   have been our worst calls during the past year. All three are trading for about half our original 5-star price, and we've since changed our opinion markedly on all of them. We underestimated how fast competition would heat up for Netflix, and Level 3 simply has not generated the level of demand that we'd originally anticipated. Our miss on Krispy Kreme is particularly galling, because we were spot on with our assessment of the shares as being overvalued earlier this year. In any case, all three belong squarely in the "miss" column."

Conclusion
Overall, I'm happy with our performance. On balance, we're doing well at recommending undervalued stocks, and a portfolio of 5-star stocks would have handily beaten the market over the past three years. At the end of the day, that's solid proof that our process and our calls are generating value for investors.

However, we can do better on no-moat stocks, and I'd like to see our batting average improve. I also want to work on lessening the number of our "sins of omission"--those companies that we could have rated highly, but that we underestimated. I'm not thinking of low-quality stocks that have been bid up to irrational levels, but rather solid firms that outperformed our expectations. If we can improve our ability to get ahead of the curve on great businesses with sustainably high returns on capital, we'll recommend more stocks that can be held profitably for long periods of time as they compound wealth. And buying those kinds of companies should be every investor's goal.