A Phantom Menace?

In the month of June, the market price of  iShares MSCI Emerging Markets (EEM) had a volatility that was 27% greater than its index and an annualized tracking error of 25%. While that might sound alarming, the exchange-traded fund has actually functioned superbly. The concern lies not with the fund, but with how volatility and tracking error are being used. Before we dive into the details of what happened in June to EEM, let's define volatility and tracking error and discuss some ways in which tracking error can be used.

Volatility tells us about the distribution or spread of returns. It can be calculated as the standard deviation of periodic returns. For example, the arithmetic average monthly return of the S&P 500 Index since 1926 through the end of last year is 0.93% and the standard deviation is 5.51%. The worst monthly return in the series was a 29.72% loss in September 1931, while the best month was a 42.56% return in April 1934. This spread of past returns can be seen in the histogram below. 

A rule of thumb for annualizing monthly volatility is to multiply by the square root of 12, which gives us 19.08%. Since we often think in terms of annual rates of return, annualizing monthly volatility puts it on a convenient time scale for comparison. Volatility is useful as a measure of risk as it gives a good idea of the spread of possible future returns. Generally, the more volatile a fund has been in the past, the more volatile we can expect its performance to be in the future.

Tracking Error and Tracking Difference
In concept, tracking error is similar to volatility; instead of referencing absolute returns, though, tracking error is in reference to a benchmark. A fund's excess return is its return minus the return of a benchmark. Morningstar defines tracking error as the standard deviation of a fund's excess return. Despite containing the term "error," tracking error in and of itself is not necessarily a bad thing for active funds where we are willing to accept some periods of negative excess returns with the expectation that the fund will outperform in the long term.

For index funds, we expect very little, if any, periodic deviation from the benchmark but are willing to accept that an index fund will lag the benchmark by an amount approximately equal to the fees. So, we expect a slightly negative excess return. Because tracking error is typically so small for index funds, it is common to focus instead on the degree to which the excess return is negative, sometimes called tracking difference.

Putting Tracking Error to Work
To get an idea of how tracking error can be used, we calculated the tracking error for the oldest share class of 275 mutual funds and the two ETFs that are currently categorized as large blend, list the S&P 500 as their primary prospectus benchmark, and have been in existence since the start of July 2009. We calculated the tracking error as the annualized standard deviation of the difference between the monthly return of the fund and the S&P 500 Index. We ran this calculation over the 36 months ended in July 2012. We use this period as an in-sample period and save the 12-month period over the past year as an out-of-sample period. Breaking the past four years into an in-sample period and an out-of-sample period allows us to see if the patterns observed in the first period carry over into the second period.

Past Tracking Error Is Indicative of Future Tracking Error
As the disclaimer goes, past performance is no guarantee of future results. But volatility and tracking error tend to persist. The scatter plot below shows the three year in-sample tracking error on the x-axis, and it ranged from nearly zero to about 14%. The y-axis shows the tracking error in the out-of-sample year. Funds with high tracking error over the three-year in-sample period tended to have high tracking error over the one-year out-of-sample period.

  - source: Morningstar Analysts

In the chart, the dots clustered around zero are index funds. As expected, active funds had higher tracking error in-sample, yet the wider range of returns persisted out-of-sample.  CGM Focus had the highest tracking error of any fund in the in-sample period at 13.7%. It also had the worst return, losing 1.38% per year over three years. While the tracking error remained elevated, the performance improved, as the fund posted a 41.51% return in the out-of-sample period, the best of any fund. This illustrates an appropriate way to think about tracking error, as a tool to gauge expectations about future performance around a benchmark.

ETFs Under the Microscope
A few issues arise when we attempt to calculate tracking error for ETFs. Like mutual funds, ETFs have an end-of-day net asset value. Unlike mutual funds, the average investor does not transact with the fund provider at NAV but rather must pay a market price on an exchange. Fortunately, for most ETFs, the arbitrage mechanism keeps the fund price close to the NAV. If there is a profitable gap between the market price of an ETF and the NAV of the underlying assets, arbitragers will trade to close that gap.

Even for ETFs holding liquid U.S. stocks, slight timing differences between when the last trade of the ETF occurs and when the NAV of the fund is struck can result in premiums or discounts. International ETF premiums and discounts will be much larger because fund NAVs are calculated based on prices when foreign markets close, many of which have stopped trading well before 4:00 p.m. in New York. Yet the ETFs continue to trade in New York, so their prices reflect information not yet embedded in their NAVs. For this reason, when an international ETF closes at a premium, we expect that premium to revert the next day when foreign markets open and the NAV incorporates the new information from the U.S. market.

In the month of June, if we measure the tracking error of EEM by annualizing the standard deviation of daily return differences between EEM's market price and the index NAV, we get a tracking error of 25%. But based on EEM's NAV, the tracking error drops to just 0.40%. For those unconvinced that premiums and discounts are largely mean-reverting noise, the table below shows the NAV and market price returns for EEM annualized for the one-, three-, five-, and 10-year periods ended June 30, 2013. Over the 10-year period, both the market price and NAV return nearly matched the index, net of fees.

 

Before dismissing ETFs that appear to have high tracking error, it helps to understand the root cause. High tracking error may result from technical and measurement issues such as those described above. High tracking error may also result when an ETF holds less-liquid underlying securities or follows an index that is not ideally investable, in which case you may want to reevaluate your thesis for investing in that asset class or your return expectations. Either way, a better understanding of the mechanics will make you a better investor.

 

 

 

	ETFInvestor Newsletter
	Want to hear more from our ETF strategists? Subscribe to Morningstar ETFInvestor to find out what they're buying—and selling—in their portfolios.	One-Year Digital Subscription 12 Issues | $189 Premium Members: $179 Easy Checkout

Disclosure: Morningstar, Inc.’s Investment Management division licenses indexes to financial institutions as the tracking indexes for investable products, such as exchange-traded funds, sponsored by the financial institution. The license fee for such use is paid by the sponsoring financial institution based mainly on the total assets of the investable product. Please click here for a list of investable products that track or have tracked a Morningstar index. Neither Morningstar, Inc. nor its investment management division markets, sells, or makes any representations regarding the advisability of investing in any investable product that tracks a Morningstar index.