Welcome to the New Morningstar.comSee what's new
Advisor Insights

What’s Our Real Impact?

Steve Wendel

In the financial-services industry, rarely do we rigorously assess whether our products and services actually help investors in practice. We know what drives our businesses, and whether a particular product could, theoretically, help investors. But our data on actual impact in practice is sorely limited, usually because we don’t have a clear vision of the rubber hitting the road: investor behavior after our products are bought or our advice is heard. That should change— to the benefit of investors and our industry.

Why Does It Matter?
Through my research career, I’ve been repeatedly amazed by the unintended and negative consequences of seemingly well-designed and well-intentioned systems. One of the most profound books on the topic is Dean Karlan and Jacob Appel’s More Than Good Intentions (2011). The authors detail how many of the world’s seemingly helpful international development efforts, like microfinance, seemed helpful, but were found to be ineffective or even detrimental once rigorous measurement was put in place.

Financial services provide many examples of a significant gap between what people in the industry intend and what actually happens— especially for the end investor. Among the many companies that claim to serve the investor, Morningstar included, this gap matters because it’s part of our mission to close it. Even for companies that don’t have an investor-centric mission, it matters because if they aren’t materially and measurably helping investors, eventually their business crumbles.

One example of such a gap is what numerous studies call the “behavior gap.” On average, investors receive far less from their investments than they should, because of common behavioral mistakes (e.g., Kinnel 2017). A well-structured investment plan can falter because of these behavioral biases, or even exacerbate them (Wendel forthcoming). Even passive investing falls prey to the same problems. For example, in a recent interview with Morningstar’s Christine Benz, Vanguard senior investment analyst Jim Rowley discussed how investors in passive vehicles make market-timing mistakes at the level of asset allocation, instead of investment selection (Benz 2017).

My own house is made of glass, too: Some powerful research in the behavioral science community shows how well-meaning efforts to overcome behavioral biases can backfire. Research by Beshears et al. (2015), for example, shows how encouraging plan participants to increase contributions via comparisons to their peers can cause them actually contribute less— even though behavioral researchers like myself had promoted such peer comparisons for years as a tool to increase contributions.

The overall lesson is this: Wanting to help investors isn’t enough. Over the past few decades, great strides have been made in helping investors reach their goals, including lower fees, more transparency, and better aligned incentives. To drive impact more effectively moving forward, though, we can significantly tighten the link between intentions and impact.

What would happen in our industry if we put impact front and center—if we considered practical, measured impact for investors as our North Star and created a feedback loop to facilitate improvement over time? It would mean new internal business processes and significant changes in how our industry works overall. Let’s delve into each of those two areas.

A New Impact-Focused Business Process
To understand the impact of our products and services, we obviously need to know what “impact” means and how to measure it. While there is no universal definition of what it means to help an investor (and, therefore, what it means to have impact in doing so), I like the simple, straightforward definition embedded in Morningstar’s mission: helping investors reach their goals. I’ll use that definition here, but in your company, you may have a different definition.

How do we know we’ve achieved an impact? Because investor success depends on the actual behavior of individuals, especially investors and their advisors, measurement must be done in practice—with real investors in real situations, and not in a theoretical model or an Excel spreadsheet. It requires a significant shift in mindset, though not necessarily a large operational shift.

The research community has developed a rigorous and effective way to measure impact despite the vagaries of individual behavior. It’s called the randomized control trial. It is the gold-standard approach and involves randomly selecting some people to receive the new product or service, and others not to. It is the same approach used to test the efficacy of medicines, in fact. In practice, companies can measure impact in this manner by making a small change to an existing standard practice: running pilot programs. If instead of a “normal” pilot program the recipients of the pilot are randomly assigned, it enables this rigorous measurement of impact. Both internal research staff and academic partners can help with the mechanics of the process.

There are implications for the upstream process of delivering products and services, as well. First, a business should determine what the product is supposed to do for investors, before it is built. Second, it must identify a way to quantify that benefit. For example, a provider of a new smart beta fund that says the fund “tracks the S&P 500 but tilts towards growth” isn’t saying enough. How concretely does that help an investor reach his or her goals? And how would the company quantify that benefit?

These pieces allow a company not only to measure impact (and verify the value of its work), but more importantly, to iteratively improve it. It sets up a feedback loop: We tried to do X, but Y happened. What can we do differently to make sure X happens in the future? From my experience in the field, many companies have no rigorous feedback loop like this, even for business impact, and are flying blind. Having this feedback loop also means that investor impact becomes part of the key performance indicators for the development process, and investor impact joins business impact as two of the central criteria for determining what gets built or marketed.

These are some of the major implications; there are certainly others. For example, companies should expect to find that certain products and services work better for some investors than others, and some do not have their advertised effect at all. Advisors may find that their efforts falter because investors don’t act on their advice or follow the agreed-upon plan. To have impact for investors means more than giving advice that is theoretically correct but isn’t effective in practice. An impact-centric segmentation process can arise—targeting people based on what’s impactful for them rather than just on their age or income—with significant implications for marketing and business success.

And there are many unanswered questions, including how to scale up impact assessments without driving up costs or slowing down execution time, and how to handle multilevel environments in which a company’s direct clients are not the end investor. These questions, and more, are ones our industry needs to tackle. At Morningstar, we are working to answer them as we apply this process to our own work, and we welcome partnership with others in the field.

An Impact-Focused Industry
The implications of this investor- and impactoriented process are considerable for the financial-services industry. Here are some of the issues to address:

The creation of a new product-by-product, service-by-service measurement of the impact would enable new transparency across the industry—showing what works and what doesn’t. This would naturally be backward-looking, based on measured actual impact, but it would help investors better judge the products that best help them reach their goals. The uncertainties of the market will never go away, but investors can better assess, in an apples-to-apples way, what each product means for them.

When investors have the information they need to evaluate products by their impact, that will naturally change the dynamics of industry competition. Instead of competing on a product’s features, companies would be increasingly pushed to compete on actual impact. That is clearly good for the investor. It also creates a business opportunity for those companies that can move and adapt quickly.

One central behavioral lesson is that often small details of how a topic is framed or described can drastically affect people’s decisions. For the financial services industry, we are increasingly seeing that the same holds true everywhere, from how companies talk about retirement savings to how the decision to sell a stock is presented. The implication for businesses is that product design, user interface, marketing, and delivery are all one integrated package that can influence outcomes for the individual. A company can (and should) improve outcomes for investors through their user interface and marketing, as much as the financial product itself.

Perhaps the most difficult issue for financialservices companies will be the redefining of accountability. Advisors in the United States are already facing the implications of being accountable for serving the best interests of clients. It’s a change that we at Morningstar have long held to be beneficial for investors and advisors. An industrywide focus on impact will, oddly enough, make meeting that standard easier by facilitating the development of a single clear metric of best interest. For asset managers and others in the industry, clear measurements of impact will necessitate changing business lines and practices that are shown to not drive impact for clients and the end investor.

With accountability and transparency come respect and trust. In many segments of American society, our industry is reviled. There is a general lack of trust of financial services, which has not historically been the norm. Through much of the 1980s, a majority of Americans had “a great deal of confidence” in banks; now that figure is 27% (Gallup 2016). We should acknowledge that that lack of trust has been well-earned by a variety of bad apples and by high-profile abuses. By focusing on the needs of the end investor and showing where industry efforts are—and are not—beneficial for investors, we can start to rebuild that trust.

What’s Coming Next
Measuring and focusing on the end-of-line impact we have on investors would have significant repercussions for financial services and investors alike. Some companies will not want to make the change for fear of being unable to compete when measured by that yardstick. But if investors are able to see which companies are most effectively helping them reach their goals, would we expect them to continue using any other yardstick? The companies that move quickly will be the ones to reap the reward.


Benz, C. 2017. “Passive Investors Aren’t Passive,” Nov. 25. http://www.morningstar.com/videos/836693/passive-investors-arent-that-passive.html

Beshears, J., Choi, J.J., Laibson, D., Madrian, B.C., & Milkman, K.L. 2015. “The Effect of Providing Peer Information on Retirement Savings Decisions.” The Journal of Finance, Vol. 70(3), pp. 1161–1201.

Gallup News. 2016. “Americans’ Confidence in Banks Still Languishing Below 30%.” June 16. http://news.gallup.com/poll/192719/americans-confidence-banks-languishingbelow.aspx

Kinnel, R. 2017. “Mind the Gap: Global Investor Returns Show the Costs of Bad Timing Around the World.” Morningstar Manager Research, May 30.

Wendel, S. forthcoming. “Using a Behavorial Approach to Mitigate Panic and Improve Investor Outcomes.” Journal of Financial Planning.

This article originally appeared in the February/March 2018 issue of Morningstar magazine. To learn more about Morningstar magazine, please visit our corporate website.