This post was first published in May 2020 and updated in March 2021
The biggest refrain you hear from people who think you can beat the stock market goes as follows:
“By definition, 50% of investors do better than the market and the other 50% do worse. Surely it cannot be so hard to be in the first group.”
There are a few problems with the above statement.
First of all, the actual proportion of investors that beat the market is far less than 50%. Here’s why:
Assume there is a $1m trade in Apple shares. On one side of the trade, you’ve got a thousand retail investors, each selling a share of Apple for $1k (yes, I know Apple is just north of $100 at the moment after all the stock splits but let’s keep it simple).
On the other side, you’ve got an institutional investor with $1m of capital, buying those 1,000 shares.
If Apple outperforms the market, the institutional investor will have beaten the market, while all the individual investors will have underperformed.
In terms of money traded, the split is 50-50. There was a cool million changing hands on each end.
But in terms of investors, the split is 0.01% winners to 99.9% losers. Sure, it could also be the other way around.
Except it isn’t, as you will see further down below. When it comes to active investing, retail investors rarely beat professional money.
Second of all, no one questions the fact that some people can beat the stock market. That happens every minute, hour, and day.
The question is, can you outperform the market on a consistent basis?
Or is it Adam getting lucky today, Joe tomorrow, and Jenny next week, yet all of them lose money over the long run?
As the data proves, doing so over years and decades is much harder than scoring a few lucky shots:
Not as easy as it appears
And yet, some people do beat the stock market. Over the years, I’ve come across a number of them, which I can broadly bucket into four categories.
Let’s take them in turn.
The People Who Beat The Market
Category #1: Insider Traders
This is the easiest one.
Notwithstanding all the technological progress in tracking illicit activity, insider trading still happens, as evidenced by the data.
Source: Patrick Augustin, Menachem Brenner, Marti G. Subrahmanyam, University of Montreal
In the graph above, T-0 is the moment in time when inside information becomes public knowledge.
That pesky line creeping up? That’s all the naughty people who have the inside information, know they aren’t supposed to trade on it – and do it anyway.
Whether it’s Ivan Boesky, Raj Rajaratnam, Rajat Gupta, or those naughty brothers at Goldman and Clifford Chance a few months ago, history is full of people who took the shortcut – and paid the price.
According to the evidence above, quite a few also manage to get away.
Make no mistake – this is not a victimless crime.
Insider traders are people who literally reach into your pocket and appropriate a share of your investment returns.
They bid up share prices of the winners – and so by the time the rest of the market (i.e. you) gets around to buying, they end up paying a higher price and getting a lower return.
And they short sell the losers – so by the time you get around to selling your stock, you absorb a bigger loss.
Some insider traders take this to the next level.
While Steve Cohen was never convicted of insider trading, the accepted wisdom is that he actively encouraged his employees to engage in insider trading.
No, he isn’t going to a party
One gets a ten-year sentence, the other – a minority stake in the New York Mets.
Whatever the circumstances, seeing “investors” like these eradicated as a class ranks pretty high up on my wish list.
Category #2: High-Frequency Traders
As it turns out, starting in the early 2000s, another group of investors figured out a way to beat the stock market.
They managed to keep it on the down low for a long time – until Michael Lewis caught wind of them and wrote an excellent expose titled Flash Boys.
In a nutshell, high-frequency traders (HFTs) set up infrastructure as close to stock exchanges as follows. By getting an earlier look at all the buy and sell orders in the market, they can figure out which way the prices are heading.
They then jump in, buying and selling stocks before the price moves – and selling them right after.
You can think of HFTs as a bunch of lightning-fast piranhas swimming next to a freshwater bull shark.
No matter how formidable the bull shark may be, piranhas will jump on prey much faster, leaving the bull shark with leftovers.
Reminds you of anyone? Not that different from insider traders, are they… The word frontrunning also comes to mind.
Founded by Ken Griffin, Citadel Securities is probably one of the most famous HFT shops in the world, but there are many others.
Despite all the recent spotlight and the talk of regulatory intervention, they still operate in broad daylight. I suppose the most expensive real estate in the world doesn’t pay for itself.
Ken Griffin’s latest real estate purchase here in London
Sorry to say this, but the bull shark is you.
Category #3: Fundamental Investors
Of everyone who manages (or claims to) beat the stock market, this is perhaps the most high-profile group.
Hundreds of thousands of highly educated, intelligent, and motivated individuals spend their days and nights analyzing companies to discern the winners from the losers.
Some succeed – but most don’t.
For those that do crack the code, it may be a case of skill – or yet more evidence of the Infinite Monkey Theorem.
Unfortunately for star-struck investors, even the most successful money managers tend to flame out after a period of time.
Neil Woodford (though he says he’s sorry and wants to try again), John Paulson, Bill Miller. Even Bill Gross crashed and burned in the end.
And many of the ones who didn’t (Peter Lynch comes to mind) simply operated in very different market conditions.
No one wants to admit it now, but in an era of less stringent disclosure requirements, they often had an informational edge over retail investors.
And yet, the likes of Graham, Buffett, Soros have a well-deserved place on the pantheon of the greats. New faces may well join them over time.
The only challenge is figuring out who they might be – ahead of time.
Category #4: Quantitative-Driven Hedge Funds
And now, let me introduce you to the Moby Dicks of the stock markets.
Keeping a low, almost invisible profile, they lurk beneath the surface, using powerful supercomputers and bleeding-edge algorithms to identify and exploit stock market inefficiencies.
It could be predicting the price of crops based on daily weather patterns for the preceding year.
Tracking container ship movements to bet on exchange rates.
Scraping corporate job advertisements to predict R&D progress and forecast new product launches.
Day after day, they run correlations on thousands of variables to try and discern even the weakest cause-and-effect relationships which are all but invisible to the human eye.
In his lengthy, yet excellent book The Man Who Solved The Market, Gregory Zuckerman lifts the veil on this secretive space.
For those of you who enjoy peeking behind the scenes, this will be a worthwhile read.
It’s fair to say that Renaissance didn’t seek the spotlight.
If anything, they tried their best to avoid it – but thanks to Zuckerman’s dogged reporting, we now have an excellent view into what it takes to crack the stock market – and how lucrative it can really be.
Meet the man who actually did solve the markets (courtesy of WSJ)
Here’s a piece of data to drive the point home: between 1988 and 2018, Renaissance generated an annualized return of 66%.
Yes, you read that right.
66% on average, or a total of $104 billion in trading profits over 30 years.
Even after Renaissance’s eyewatering fees (5% management fee and 44% of the profits), it worked out to 39.1% in annualized returns to investors.
While I’ve never met Jim Simons, I have come across a few of his contemporaries.
After all, even the most secretive traders can’t run their business without accessing the markets, which they typically do through broker-dealers.
My perceptions of the industry certainly jive with what Zuckerman has to say. For those of you who don’t feel like going through an entire book, these are the most salient points:
The people involved are incredibly smart
World-class mathematicians, military codebreakers, physicians, astronomers, and statisticians, the players here aren’t your typical Wall Street types.
And yet, they possess exactly the right skills needed to beat the stock market in today’s incarnation.
They have access to world-class infrastructure
Hundreds of millions of dollars are invested every single year to upgrade the machinery that helps collect, process, and analyze the information – on a near real-time basis.
They don’t try to make sense of the market
The most successful quants have long realized that they cannot explain most of the correlations they identify.
They don’t look for underlying answers. If the relationship is statistically meaningful, they trade on it and move on.
Their success rate is just north of 50%
This is perhaps the most startling revelation of them all.
For all their intellectual ability, experience, data sets, and capital, the odds are just ever so slightly in their favour.
Yes, it’s this hard to beat the stock market.
Unfortunately, when something seems to be too good to be true, it probably is.
No, Renaissance and other quant hedge funds aren’t running a Ponzi scheme of some kind.
The answer is much more prosaic is much more disappointing.
They Don’t Need Your Money
Yes, this is the biggest letdown of them all.
When the quantitative hedge funds were just starting out as an investor class, they did accept outside investors.
The issue, of course, is that market inefficiencies are hard to find. And when you do find them, the very act of exploiting them reduces the arbitrage opportunity to zero.
You can’t milk it forever. If you could, Renaissance wouldn’t have paid out $100bn+ in dividends.
Instead, they would have reinvested the money and dominated the entire world instead of just lurking in the corridors of power of its most powerful country.
As their trading profits piled up, the quant shops realized they are much better off keeping the spoils to themselves.
Today, access to the best funds is limited to existing and former employees only, plus a small club of privileged insiders.
And yes, this applies to anyone else who figured out a way to make a superior risk-adjusted return.
I hope that people reading this blog know this already, but just in case, here goes:
No, that Instagram bro with pics of cars and mansions in his profile doesn’t know how to do it.
Neither does the Twitter “expert” with proprietary option trading courses that you can buy for £99 (but only if you hurry up).
And nor do the Facebook “commodity visionaries” that seem to have overtaken my timeline with sponsored posts on crushing it with gold through the next debt supercycle.
The only thing they’ve figured out is how to speak to that ever-present human desire to find a shortcut instead of taking the tried and tested route.
The problem is, despite anything you may have seen on the internet, you only have two possible paths to riches:
If you are a world-class scientist with a stellar track record, you may want to chuck in a job application here.
Everyone else might as well stick with index funds.
About Banker On Fire
Enjoyed this post?
Then you may want to sign up for our exclusive updates, delivered straight to your inbox.
You can also follow me on Twitter or Facebook, or share the post using the buttons above.
Banker On FIRE is an M&A (mergers and acquisitions) investment banker. I am passionate about capital markets, behavioural economics, financial independence, and living the best life possible.
Find out more about me and this blog here.
If you are new to investing, here is a good place to start.
For advertising opportunities, please send an email to bankeronfire at gmail dot com
23 thoughts on “The People Who Beat The Stock Market”
Nice article. I remember reading Michael Lewis’s book liars poker many years ago and loved his style. Flash boys is an eye opener and it’s amazing his revelations are still legal.
I think there was a statistic that said the HFT average time they “hold” a share is 0.000x of a second.
They also had an unbroken record of something like 300 months with no loss.
A friends work place all got bought out of their pensions. About 30 people all well paid professionals. Many DB pensions were to provide £80k per year for life…. do the maths!
One of the employees wife worked for a very well know financial planning firm. She basically got commission (5%) from 30 people all taking the money from a DB to SIPP product.
This book said it all
Where Are the Customers’ Yachts? (Wiley Investment Classics) https://www.amazon.co.uk/dp/0471770892/ref=cm_sw_r_cp_api_i_SiKUEbG6T8G3D
I have all the respect in the world for Brad Katsuyama and IEX.
Unfortunately, the initiative isn’t gaining nearly enough traction – disappointing but not surprising given the balance of power in the market infrastructure space.
The book you are referencing has been on my list for ages – time to move it up the ranks!
Loved the article and all the flair with your writing. You are good!
Cheers Shana. I’ve got to admit, being able to look behind the curtains is a pretty cool aspect of my day job.
Thanks for another awesome article.
You mention passive investing and index funds in your articles quite a lot, which this post clarifies the reason behind that.
When you talk about index funds, however, are you referring to index mutual funds (not traded) and ETFs (traded)? I think a lot of people confuse the two. Although both are passive, each has its advantages and disadvantages. Which one would you recommend and why?
I think you mentioned in a previous post that you hold VUSA which is an ETF, so based on that I guess you prefer that option.
Thanks and keep up the writing. Your blog is super helpful.
Hi there fellow banker!
Very good point. Reality is that both mutual funds and ETFs get you to the same place. JL Collins did an excellent summary of how they differ here:
I don’t have a preference between the two to be honest. Key is to pick something well diversified (I like US but global is also an excellent choice) and with low fees.
Thanks for the quick reply as well as for the link.
1) Do you invest in SP500 through both vehicles or only through ETFs (VUSA as you mentioned in one of your previous posts)?
2) Also, aren’t you “afraid” of the exchange rate risk? Similarly to Buffet, I am also a long-term bull on the American market and, just like you, I am investing in VUSA (EUR denominated in Euronext). However, our returns are not only dependent on the performance of the U.S. market, but also on USD/EUR (or USD/GBP). Of course, there is the interest rate parity theory which says that in the very long-term, exchange rates should not matter. Is this what makes you comfortable with this risk?
Personally, I like to diversify a bit more (also in terms of forex), so I also hold a world index ETF of which ~50% is SP500 basically. You may ask why VUSA then? That’s because of my bullishness on the US market, so I want to be overweight on that part.
I only use ETFs but I really don’t think it makes a difference, as long as you use a vehicle that minimizes trading and management fees.
On your second question – I think FX risk is overrated. There’s a section at the end of this post that explains why: https://bankeronfire.com/the-case-for-a-100-us-equity-portfolio
Pingback: Weekend reading: Would you lend yourself money in an emergency?
Another top article. Love your writing style and honesty! Keep it up!
Thanks Alex, glad you enjoyed it.
Have a good weekend!
Aged 73 retd 17 years -took me a while to work out that I had no chance beating the market
Been in index funds ever since -20+ years
They did the job for me
I however enjoy reading about finance -if nothing else it reinforces my investing stance
Michael Lewis,s books are terrific
John Bogle of Vanguard helped a lot of people and his ideas are slowly turning the tide for the average investor
The clever and smart are always with there-no doubt often advancing the cause of humankind but they are very different from the rest of us
It takes a very unique combination of skills and intelligence, these days often augmented by supercomputing power, to outwit everyone else in the stock market.
The rest of us are best off controlling what we can – our savings rate and time horizon.
Hi, I’ve read few of your blogs on active investment, and the evidence that you cite to prove that beating the market is futile does not apply to the retail investor.
The evidence which you use that- for the last 15 years. 91.6% of large cap fund had fail to beat the market certainly shows that only a few of active fund managers can only beat the market.
But most of us are not fund managers, we are retail investors- big difference.
For fund managers, their investment time horizon is much much shorter, most of the time they have to provide positive return for their clients every quarter- otherwise their clients would just withdraw from their funds. To achieve this, fund managers will have to sell out great companies for a quick profit early.
For retail investors, we are not constrained by this, and thereby we can invest for the long long term, thus beating the market if you choose the right companies through value investing. (which you don’t need a 150 IQ to do so). For example, through both skill and lucky, I am now sitting on 100% gains year to date ( way outperforming any index funds) by investing in great companies during the March lows. (Tesla, Facebook, JP Morgan, and other Hong Kong companies).
Hopefully I can provide a different perspective on you investment strategies.
Thanks a lot for your comment Michael. I am not dogmatic by a stretch – this blog is about what I do to build wealth but I always welcome different perspectives and opinions, so I do appreciate your point of view.
You have a point on the time horizon with institutional players. That being said, hedge funds don’t have a great track record of beating the market either, and their investors are much more constrained with it comes to redemptions.
Turning to retail investors, by default some will beat the market. As you point out, it could be due to skill, luck – or both. But the point is that most retail investors probably won’t, on account of not having the right skill set, access to management (which professional money managers have more of than people appreciate), risk tolerance, length of perspective and most importantly – time to research and make investment decisions.
In addition, on a retail portfolio of let’s say £100k, a point of outperformance adds up to £1k a year. Most folks are better off putting their money in index funds and using the free time to either make more money or enjoy life. Unlike us money geeks, not everyone enjoys researching stocks.
Cheers and good luck with your portfolio going forward, as an industry insider I’m a big fan of JP Morgan in particular, and Tesla was a great call so far. Always happy to take a contrarian point of view on anything that I say 🙂
Hi thanks for the reply :),
I honestly believe that investing in stocks is simple, and Charlie Munger had said that the temperament of an investor is more important that his IQ.
And my strategy is – buy and hold great companies at a fair for the long term and let them compound. And they are not hard to find.
As me and you said earlier, luck comes into play in investing too. For example, the March crash was an extremely lucky opportunity for me to invest into great companies.
But after all good luck with your FIRE journey.
It’s fascinating and shocking that if you just beat the stock market 0.5% year after year, you will become richer beyond your wildest dreams. Compound investing works in mysterious and wonderful ways so to those people who beat the stock market on a consistent basis, hats off to them. Just hope that they are not lying 😉
I tend to believe the people who aren’t asking for your money 🙂
For example, Jim Simmons was trying his best to avoid doing the Zuckerman book. Once you actually know how to beat the market, you don’t want other people to start deconstructing your methods and competing with you.
Not quite like the Twitter bros who will spill give away all their market-beating techniques for just $99 per course 🙂
saw your recent article on POF and was directed here. Really enjoyed your articles and congratulations on your RE success! I thought about IB, but don’t think I would have lasted. Also, it doesn’t look as glamorous as I imagined! thanks for peeling back the curtain.
Glad you enjoyed it Brian!
Yes, true IB is definitely not what you imagine when you watch Billions, Wolf of Wall Street, or Succession (though there are some parallels and I’ve got quite a few stories to tell).
The actual “craft” is quite different and if you don’t enjoy the day to day, it’s very tough to survive the pressure and the pace.
Pingback: Betting on a player. Betting on a team. Betting on a league.
Though I picked this article to read at random, category #2 is really interesting because I’m considering of doing a career transition into the world of HFT.
I don’t know too much about the technical details of high frequency trading but it seems like a fascinating world and will definitely pick up Flash Boys.
Hopefully I can do the following:
1) Just invest in the market.
2) Transition into an HFT career where I can get an outsized paycheck to repeat 1).
The real prize is to invest with the HFTs as well
Those who invested with Simmons have made a killing, so much that he eventually closed his fund to outside money