P-CAPE: A Better Way for Investors to Estimate Future Returns

Editor’s note: Today, the reciprocal of the cyclically adjusted price earnings, or CAPE, ratio (1/CAPE) is the metric many investors use to estimate the long-term expected real return of the stock market. This metric is known as the cyclically adjusted earnings yield, or CAEY.

In their 1988 research paper, “Stock Prices, Earnings, and Expected Dividends,” John Campbell and Robert Shiller introduced the use of the CAPE ratio to forecast long-term stock market returns. Since then, the CAPE 10 (the 10-year average for earnings adjusted for inflation) has been used to estimate future equity returns. However, employing a 10-year average for earnings, instead of the most current 12-month earnings, wasn’t new. It was first suggested by legendary value investors Benjamin Graham and David Dodd. In their classic 1934 book, Security Analysis, Graham and Dodd noted that traditionally reported price/earnings ratios can vary considerably because earnings are strongly influenced by the business cycle. To control for the cyclical effects, Graham and Dodd recommended dividing price by a multiyear average of earnings and suggested periods of five, seven, or 10 years.

Providing support for the CAPE 10, in his 2016 study, “Predicting Stock Market Returns Using the Shiller CAPE—An Improvement Towards Traditional Value Indicators?,” Norbert Keimling concluded that it was possible to forecast potential long-term returns in the S&P 500 for periods of more than 10 years (he examined periods of up to 15 years) using the CAPE 10 and that the relationship also existed internationally in 17 MSCI country indexes: “In all countries a relationship between fundamental valuation and subsequent long‐term returns can be observed.”

The following chart shows the historical relationship between CAEY and the next 10 years’ real return of the US stock market.

Historical Relationship Between CAEY and 10-Year Return of US Stock Market

A Better CAPE

Victor Haghani and James White, authors of the June 2024 paper, “Introducing P-CAPE: Incorporating the Dividend Payout Ratio Improves Investors’ Favorite Estimator of Stock Market Returns,” sought to determine if the CAPE could be improved upon. They began by noting: “A shortcoming of Shiller and Campbell’s definition of cyclically-adjusted earnings is that it doesn’t take account of the fact that, in general, companies don’t pay out all their earnings as dividends each year. The fraction of earnings not paid out in dividends is either reinvested in the business or paid out via stock buybacks. Reinvesting earnings in the business is done in the expectation of growing future earnings, and this earnings growth should ideally be accounted for when smoothing earnings over the previous ten years for the purpose of predicting long-term future earnings.”

Their hypothesis was that a better measure of cyclically adjusted earnings is one that accounts “for the logic that retained earnings should increase earnings per share over time (whether through investment in the business or through share buybacks) in addition to the inflation adjustment already part of Campbell and Shiller’s measure.” They called their modified measure “payout and cyclically-adjusted earnings,” or P-CAE. The earnings yield is used to compute what they called “P-CAEY” and the price/earnings multiple “P-CAPE.”

“For example: if, 10 years ago, the dividend payout ratio was 60%, real earnings on the stock market index were $10, and the CAEY was 6%, the payout-adjusted earnings we’d use would be:

60% ∗ $10 + 40% ∗ $10 ∗ (1+6%)10 = $13.2.

We then take the average of each of those ten years of payout and inflation-adjusted earnings.”

They noted: “For dividend payout ratios of less than 100% and for positive earnings yields, P-CAE will be higher than Shiller and Campbell’s cyclically-adjusted earnings, which are only adjusted for inflation. From 1890 through May 2024, this new payout and cyclically-adjusted earnings metric was, on average, 19% higher than the standard cyclically-adjusted earnings metric.”

Following is a summary of their key findings:

• While the standard Shiller and Campbell metric underestimated future earnings by 13% over the period of 1890-May 2024 and by 15% in the more recent period beginning in 1950 (consistent with falling payout ratios, the shortfall was larger in the more recent period), their P-CAPE overestimated earnings by just 1% over the full period and was right on for the period beginning in 1950.
• The explanatory power of the P-CAPE was 11 percentage points higher over the full period (35% versus 24%) and 18 percentage points (33% versus 15%) higher in the more recent period.
• Not adjusting for dividend payout rates leads to increasingly poor earnings estimates as the window used for the estimate lengthens. Using the full US stock market earnings history from 1880 to 2024 as the window rather than 10 years, Campbell and Shiller’s CAE would be $45 per S&P 500 unit today, while the P-CAE estimate would be $210. Actual S&P 500 earnings at the end of 2023 were $197.
• The P-CAEY gives a more consistent measure across international equity markets with different dividend payout ratios.

The chart shows P-CAEY and the next 10-year US real stock market returns.

P-CAEY and Next 10-Year US Real Stock Market Returns

Why Forecasts of Returns Are Important

It is impossible to build an investment plan without estimating the returns to stocks (as well as to bonds and any alternative investments). One reason is that the estimate of returns determines your need to take risk—how high an allocation to equities and other risky assets you will need to reach your goal.

If your estimate is too high, it’s likely you won’t have sufficient assets to reach your retirement goal. If it’s too low, it could lead you to allocate more to equities, which means taking more risk than necessary. Alternatively, it could lead you to lower your goal, save more, or plan on working longer.

Unfortunately, many mistakes are made when looking at the historical record of the CAPE 10 (or any other measure) in terms of its forecasting accuracy. Perhaps the most common error is to consider the CAPE 10 not useful because there is much error in the forecast—there are still very wide dispersions of returns.

In their 2017 paper “The Many Colours of CAPE,” Robert Shiller and Farouk Jivraj found that the CAPE 10 was a better predictor than any of the alternatives considered, and it provided valuable information. As you can see in the table below, they found that 10-year-forward average real returns dropped nearly monotonically as starting Shiller P/Es increased. They also found that as the starting Shiller CAPE 10 ratio increased, worst cases became worse and best cases became weaker. However, it’s important to focus on not just the average returns, but the worst and best cases as well.

Starting CAPE Ratio vs. Real 10-Year S&P 500 Annualized Returns (%)

For example:

• When the CAPE 10 was below 9.6, 10-year-forward real returns averaged 9.8%, well above the historical average of 7.4%. The best 10-year-forward real return was 17.2%. The worst 10-year-forward real return was still a pretty good 4.2%. The range between the best and worst outcomes was a 13-percentage-point difference in real returns.
• When the CAPE 10 was between 16.1 and 17.8 (about its long-term average of 16.9), the 10-year-forward real return averaged 5.4%. The best and worst 10-year-forward real returns were 14.6% and negative 3.8%, respectively. The range between the best and worst outcomes was an 18.4-percentage-point difference in real returns.
• When the CAPE 10 was above 26.4, the real return over the following 10 years averaged just 0.9%—just 0.5 percentage point above the long-term real return on the risk-free benchmark, one-month Treasury bills. The best 10-year-forward real return was 5.8%, and the worst 10-year-forward real return was negative 6.1%. The range between the best and worst outcomes was a difference of 11.9 percentage points in real returns.

As tests of robustness, Shiller and Jivraj also found that the CAPE 10 had explanatory power for both sector and individual stock returns.

Valuations Matter

What can we learn from the preceding data?

First, starting valuations clearly matter, and they matter a lot. Higher starting values mean that not only are future expected returns lower, but the best outcomes are lower and the worst outcomes are worse. The reverse is true as well—lower starting values mean that not only are future expected returns higher, but the best outcomes are higher and the worst outcomes are less poor, too. However, it’s also extremely important to understand that a wide dispersion of potential outcomes, for which we must prepare when developing an investment plan, still exists—high (low) starting valuations don’t necessarily result in poor (good) outcomes. In other words, investors should not think of a forecast as a single point estimate, but only as the mean of a wide potential dispersion of returns. The main reason for the wide dispersion is that risk premiums are time varying (if they were not, there would be no risk in investing!). It is the time-varying risk premium, what John Bogle called the “speculative return,” that leads to the wide dispersion in outcomes.

The fact that a wide dispersion of returns occurs around the mean forecast is why a Monte Carlo simulation is a valuable planning tool. While the input includes an estimated return, it also recognizes the risks of that mean forecast not being achieved—which is why volatility is another input. Because most simulators allow you to examine thousands of alternative universes, they enable you to test the durability of your plan—you can see the odds of success (such as not running out of money or leaving an estate of a certain size) across various asset allocations and spending rates.

The time-varying risk premium is also why an investment plan should include a Plan B, a contingency plan that lists the actions to take if financial assets were to drop below a predetermined level. Actions might include remaining in, or returning to, the workforce, reducing current spending, reducing the financial goal, selling a home and/or moving to a location with a lower cost of living.

The bottom line is that CAPE’s explanatory power provides significant information that investors can use to build plans. However, you must not make the mistake of overestimating the forecasting power of the CAPE 10 metric (or the new P-CAPE) and treat that forecast as what will happen. Instead, it should be used to help determine your need to take risk and then, with the understanding that risk premiums are time varying and a wide dispersion of potential outcomes is likely, to help build a plan B.

Another mistake investors make when criticizing the use of the CAPE 10 is that it doesn’t work as a timing tool.

Timing the Market Using the CAPE 10

While valuations do provide information on future returns, the research has found that they do not provide information that allows investors to profitably time the market. For example, Cliff Asness, Swati Chandra, Antti Ilmanen, and Ronen Israel, authors of the study “Contrarian Factor Timing Is Deceptively Difficult,” found “lackluster results” when investigating the impact of value timing (in other words, whether dynamic allocations can improve the performance of a diversified, multistyle portfolio). They wrote: “Strategic diversification turns out to be a tough benchmark to beat.”

Despite the evidence, many investors ignore the findings and try to use CAPE 10 valuations to time markets. They might shift from stocks to bonds when the CAPE 10 is above its long-term average and shift from US to international when international valuations are lower (as they have been for a number of years now). As we just discussed, this has not worked well. However, blaming the CAPE 10 for its failure as a timing tool is like blaming the fork for its failure as a useful tool to eat soup. It is not intended to be used to time markets, only to forecast mean expected returns. Consider the following example.

Using data from AQR Capital Management, as of the end of the 2023, the CAPE 10 earnings yield was 3.5% for the US, 5.5% for developed non-US markets, and 7.1% for emerging markets. Some investors take this to mean you should underweight US stocks and overweight emerging markets. This is the wrong use of the CAPE 10 for the simple reason that doing so ignores risk. It’s the equivalent of saying junk bonds with higher yields are better investments than US Treasury bonds with lower yields. The information provided is that investors believe international stocks are riskier, and thus they have higher expected (not guaranteed) returns as compensation for that risk. It’s important to always keep in mind that the efficiency of the markets means that all risky assets should have similar risk-adjusted returns. Thus, your starting point when determining your asset allocation should be the global market capitalization, which is currently about one half US, three eighths developed non-US, and one eighth emerging markets. Having set your asset allocation, you should stick with it, rebalancing as needed.

Investor Takeaways

While Campbell and Shiller’s CAPE has provided valuable information as to expected returns, Haghani and White have found an improved tool for investors to use when estimating future returns. However, it’s important that the information be used in the right way, as misusing it can lead to bad outcomes and the failure of plans. You should not use a valuation metric in a deterministic way (that is, “I’m going to earn x percent”). Instead, the forecast should only be used in a probabilistic manner. And you should not use valuations to time the market, shifting allocations toward higher expected returning assets.