The 'Hot Hand' Finding, Revisited

The Big Splash Among the most influential behavioral-science papers is "The Hot Hand in Basketball: On the Misperception of Random Sequences," by Thomas Gilovich, assisted by Robert Vallone and Amos Tversky. Published in 1985, the article quickly became required reading. (It was for my MBA management class.) According to a recent missive from PIMCO, Hot Hand has received more than 1,200 academic citations.

The paper showed that, although basketball fans widely believed that players who had made their previous shots--who had the "hot hand"--were likelier to make the next shot that they attempted, such was not the case. The fans were wrong. They saw what they thought they would see, rather than what actually occurred.

The tale appealed. If basketball fans could not distinguish between fantasy and reality with something so simple as recognizing when baskets are scored, then perhaps they flubbed other aspects of reality. And not just basketball fans. Perhaps seeing mirages was a widespread error, occurring throughout society.

Such indeed was the belief of behavioral researchers; and Hot Hand provided important (although of course not sole) support for that thesis.

Two Naysayers However, it appears now that the paper was incorrect.

Several years ago, two economists--Joshua Miller and Adam Sanjurjo--realized that Hot Hand's counting method was flawed. By their analysis, if Hot Hand had used the same approach to measure coin flips rather than basketball shots, it would have shown that coins had reverse momentum. Coins that had mostly recently landed heads, by Hot Hand's calculations, would be likelier than not to yield tails on their next flip. Oops.

It took Miller and Sanjurjo a while to make headway with their claim. Not only was their argument difficult to explain, but Hot Hand had gravitas. Overturning the paper would require some doing. Miller and Sanjurjo made public their claims in 2015, but their refutation was not published until late last year, in the journal Econometrica. It was entitled, "Surprised by the Hot Hand Fallacy? A Truth in the Law of Small Numbers."

Let's Make a Deal Their critique, they explain, is equivalent to the Monty Hall problem.

In that puzzle, Hall shows the contestant three doors. Behind one door is a car and behind the other two doors are goats. The contestant is permitted to select a door, without knowing what is behind that door (otherwise, it wouldn't be much of a contest). Hall then opens one of the two remaining doors, behind which is a goat. The contestant then may switch her original choice, to pick the other door, if she so desires.

Mostly, the contestant does not. Few people make the switch. (The television show Mythbusters devoted an episode to this subject. All 20 participants opted to stay with their original selection.) But they should. While people tend to believe that now have a 50% chance of winning the car, and that each of the two remaining doors is effectively a coin flip, that is not so. Their current door gives them only a one in three chance of getting the car. The other door has two chances in three.

This is a deeply counterintuitive outcome. But a quick check illustrates that it is true. One time in three, the contestant's initial choice will be the car. Staying put would succeed. Twice out of three, however, the selection will be one of the two goats. In each of those two cases, because Monty has revealed the other goat, the remaining door will contain the car.

When Hall shows the goat, contestants believe that their odds have improved from 33% to 50%. In reality, though, their odds do not change, staying at original 33%. The probability that the remaining door holds the car doubles, however. Hall's apparently useless gesture contained information that altered the odds.

An analogous process occurred with the Hot Hand paper. As with Monty Hall, the author's activities altered the odds. In Hall's case, the action that added information--thereby altering the probabilities--consisted of showing a goat. For Hot Hands, the item was even more obscure: the act of identifying sequences when combing through the raw data.

Today Isn't Tomorrow To attempt to explain the mystery: If you watch somebody flip three straight heads with a coin and are asked, on the spot, to guess the probability that it will land heads again, you likely will respond 50%. That is correct. When observing events that are presumed to be random (as with coin flips) as they happen, history is immaterial. Assuming that the coin is not flawed, its next action is independent of its previous actions.

However, record what occurred with a series of coin flips, then extract sequences from those results (for example, isolating whenever three consecutive heads occurred), the calculations can function differently. That is because sorting through the history provides information that real-time observers lack. They cannot know when a given sequence will end--that is, which flip will conclude the string of heads (or tails). But the authors of Hot Hands did.

They possess foreknowledge, just as does Monty Hall. (He knows what lies behind each door.) And that foreknowledge alters the odds, as with the Monty Hall problem.

From there, I am stumped. I understand the Monty Hall puzzle; the spirit of Miller and Sanjurjo's approach; and how in the highly simplified example of a series that consists of three coin flips, the Hot Hand's methodology calculates that a flip of heads has a 42.5% probability of being followed by another heads, rather than the correct chance of 50%. That is as far as I have progressed in grasping the full argument--which, writes Miller in an email, involves "the added feature of adjacency." (Whatever that may be.) Thus, I cannot certify their finding.

But their paper has circulated for several years now and has been vetted by several dozen peers, including two Nobel laureates. In addition, Hot Hand's lead author Gilovich has conceded the point. Unbeknownst to Hot Hand's researchers, and to the thousands of subsequent readers, that paper's calculations understated the amount by which one successful basketball shot (or streak of successful shots) was followed by another.

In other words, the study's basketball players did exhibit hot hands. Perhaps not as powerfully as most basketball fans believed, but the effect existed. Basketball fans did not imagine it.

Lessons The upshot:

1) The existence of the hot hand in basketball is not a fallacy. When properly measured, without bias, Hot Hand's players showed a tendency to make baskets when on a positive streak (and vice versa, after missing shots).

2) Whether momentum exists in other endeavors remains unclear. But we should not use Hot Hand's findings to dismiss such claims.

3) Science is hard. Even the most closely scrutinized work, by top academics, may contain undiscovered errors.

4) Foreknowledge can be dangerous. Researchers who study the past have informational advantages over an event's contemporaries. Such effects may bias the findings.

John Rekenthaler has been researching the fund industry since 1988. He is now a columnist for Morningstar.com and a member of Morningstar's investment research department. John is quick to point out that while Morningstar typically agrees with the views of the Rekenthaler Report, his views are his own.