Though clearly not perfect, the finance professors' approach to investing has a lot going for it, particularly the notion of factors.
Though it sometimes is hijacked by ideologues, the scientific method works. The most successful societies entrust scientifically trained workers with the most specialized tasks, such as performing brain surgery, designing airplanes, and setting marketwide interest rates. And yet, many individuals regularly entrust their fortunes to the investing equivalents of witch doctors and astrologers. Or they take matters into their own hands for no good reason other than a vague belief that they can do it if they put their minds to it. Unlike good scientists, they're not skeptical enough of themselves or others.
Brains and education are no panacea. My father is a tenured professor of electrical engineering at one of South Korea's top research universities. When he designs a microchip, he draws on his years of education, consults industry journals, and relies heavily on the work of other engineers. When he speculates in small-cap technology stocks, his efforts are far more casual and sometimes include asking me which ones I like--I always profess ignorance. Despite admittedly subpar performance over 20-something years of investing, he refuses to give up control and index his holdings. It's as if the logical, skeptical part of his brain shuts down, and a more animal, overconfident part of his brain takes over in all matters investing. Would I, armed with nothing more than the efforts of a few spare hours each week, go into the chip-design business to compete with the likes of Intel and ARM? Of course not.
There is a science to investing. Though you may not know them by their technical names, chances are you're familiar with the fruits of Modern Portfolio Theory, the connection between risk and return, the theory of interest, and the efficient market hypothesis. Parts of financial theory are so integral to the practice of investing that most investors have forgotten they originated in academia. That said, some domains are more amenable to scientific expertise than others. The sweetest fruits of biomedicine originate from trained scientists; the best investment results don't always originate from finance Ph.D.s. In fact, some of the greatest investors are derisive of "scientific" approaches to investing. Warren Buffett warned, "Beware of geeks bearing formulas."
Why aren't finance professors dominant in investing? I can think of several reasons. The foremost reason is certainly emotion, which can consume even the finest minds. Famed logician Kurt Gödel was terrified of being poisoned and ate only food prepared by his wife, Adele. When she was hospitalized for six months, he starved himself to death. Even very smart investors can kill their retirement plans when they're in thrall to their animal brains. Investing requires unusual discipline that, by definition, most people lack. Moreover, this quality, in my experience, seems unrelated to brainpower.
Second, finance and economic researchers often don't have powerful enough tools to divine as much meaning from the available data as they'd like. That doesn't stop them from trying, though, and they often end up making astrologers look good by comparison. Not helping matters is how their work can have profound economic, social, and political implications, tempting researchers into the morass of politics, where their impartiality often withers and dies. Find me an economist who assumes nuanced positions that can't be neatly described as Democrat or Republican, and I'll show you someone who's practically irrelevant in the nation's political discourse.
Third, most of the information in the markets is not quantifiable with the tools at our disposal. By restricting themselves to hard numbers, scientific investors sometimes miss what qualitative investors see clear as day. Franco Modigliani, of the Modigliani-Miller theorem, was puzzled that so many firms paid dividends. Investors have known since the days of Ben Graham that dividends impose discipline on corporate managers, keeping them from doing too many stupid things. Soft qualities such as culture and incentives matter, even if you can't easily assign numbers to them or model them in a closed-form solution.
There are good reasons be skeptical of the things that come out of finance researchers' mouths, but it's a big mistake to completely dismiss them. I'd go as far as to say evidence-driven investing is the best approach for the majority of investors, because it's based on an efficient learning strategy. Many investors pick a terrible learning strategy: personal experience. Experience is unreliable; colored by emotions and the zeitgeist, it captures a period that's short by the standards of history. Investors traumatized by the Great Depression learned that stocks are dangerous and should be avoided; investors who rode the bull markets of the 1980s and 1990s learned that stocks are unstoppable engines of wealth. Both learned the hard way that personal experience is a flawed teacher.
A better strategy is to learn from history, so you don't repeat the mistakes past generations made. Scientific investing, at its best, is about engaging the data honestly, with the intention of learning something new, hopefully something discordant with previously held beliefs. Science as it's currently practiced has plenty of flaws, but it's still the most reliable method of acquiring the truth that I know of.