Mathematicians on the attack.
Firing a Broadside
Immediately after I finished yesterday's column on technical analysis, a related paper landed on my desk: "Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-Of-Sample Performance."
It's the first paper written by mathematicians that I have ever read. (Certainly, the first that was peer-reviewed for Notices of the American Mathematical Society.) And a pleasure it was. Based on a sample size of one, I can confidently state that mathematicians are much better writers than are finance professors.
They certainly know how to issue a call to arms:
Historically scientists have led the way in exposing those who utilize pseudoscience to extract a commercial benefit. As early as the 18th century, physicists exposed the nonsense of astrologers. Yet mathematicians in the 21st century have remained disappointingly silent with the regards to those in the investment community who, knowingly or not, misuse mathematical techniques such as probability theory, statistics and stochastic calculus. Our silence is consent, making us accomplices in these abuses.
Now, things aren't really that bad. The authors' critiques are familiar; even if mathematicians have been silent about the ills of investment back-testing, others have not. After all, the joke, "If you torture the data long enough, it will confess," was cracked by the economist Ronald Coase, not a mathematician. That said, the authors state their case well, and they offer a couple of solutions.
The paper begins:
Recent computational advances allow investment managers to methodically search through thousands or even millions of potential options for a profitable investment strategy. In many instances, the resulting strategy involves a pseudo-mathematical argument, which is spuriously validated through a simulation of its historical performance (also called a backtest).
A trivial example being the Super Bowl Indicator. According to the Indicator, a Super Bowl victory from a team from the old NFL equals a bull market for that calendar year, while a triumph from the AFL equals a bear market. That's a pseudo-mathematical argument, albeit of the simple kind, which we can validate by back-testing the Indicator's historical performance. Voila! The Indicator has worked on 35 of 47 occasions.