The History of Quant Trading S&P 500SPCFD:SPXSteverstevesCenturies ago, back in 2023 (man, time flies fast around here), I did a post on the history of the S&P using SPX to illustrate key moments in history, from the 1900s to the current day of 2023, you can read it here. This was one of the biggest posts I had done that required the most amount of research. And one of my objectives this year was to give more education content, so I thought about doing another history post. You can probably imagine that I am, well, a nerd. I like math and applying math to markets. I actually hate trading, the joy I get from trading is using math and statistics to solve complicated real world problems, like where is ticker Y going to go and where will ticker X find a bounce. This is what keeps me engaged. Well that, and the payout. But another thing I enjoy are computers. Since I was a kid in the 90s, I had a real fascination with computers. My father, being a Ford executive here in Canada, would frequently bring home old industrial computers that were used on the production floor and I would tinker around, take apart the insides, swap components and the like. It became a huge fascination of mine and that actually went dormant until recently, when I entered the realm of high impact, quantitative analysis, which required me to build up my own server and rig with very specific components to handle the task. This brought back memories of me being a kid in the 90s and it was surprising how little has changed in terms of the internal components and building of computers between the 90s and now. Only, my new setup smelled less like machines and oil. So I thought what better than to talk about the history of Quantitative finance, and in particular, the evolution of how data availability, computing and, of course, AI has shaped the financial markets as we know them today. The Absolute Beginning It started in 1900 in a French university basement with a PhD thesis on agricultural commodities, drawing heavily on physics and fluid dynamics. A gentleman by the name of Louis Bachelier (1900), a French mathematician, published his dissertation, The Theory of Speculation (or Théorie de la spéculation). In this research, as an effort to model option prices on the Paris Stock exchange (or la Bourse), he described Brownian motion, a concept and method commonly applied today in indicators and quantitative models. Keep in mind, this was five years before Einstein famously used the exact same mathematics to model the physical movement of pollen floating in water. Bachelier hypothesized that asset price changes were unpredictable and followed a continous “random walk” (again, another term and theory commonly used today in 2026). This left the door open for the future of modern stochastic calculus applications. Why did this not revolutionize the market at the time? While the stock market was very much alive and moving in the 1900s, Brownian motion is an incredibly complex process, requiring hundreds if not thousands of random simulations to find a central tendency. During this time, there were not even calculators, never mind computers, so this was not a realistic theory to apply to markets. It would take days if not weeks/months to do the required calculations by hand, at which point, the time had already passed in the market and the predictions would be old. Then, in 1952, along came a man named Harry Markowitz. Markowitz introduced the Modern Portfolio Theory or MPT. Instead of just picking stocks that were constantly winning (*cough* NVDA *cough*), Markowitz proposed diversification. But he went many steps further than just “proposing”, he mathematically proved that investors cold construct an optional portfolio by balancing expected return against variance using covariance matrices! The most interesting thing about Harry Markowitz isn’t actually his theory and his conclusion, but how he came to solidify this theory as a conclusion. To prove that asset diversification actually meaningfully contributed to better outcomes, Markowitz used an IBM 701 (one of IBM’s first commercial mainframe computers) to calculate his thesis. This computer used physical punch cards to solve quadratic programming problems. And 1952, just 6 years after WWII ended, marked the moment of the Quantitative Finance and Computing intersection. Towards 1964, along came a man named William Sharpe. And I know you’re naturally thinking, “Oh is this the founder of the Sharpe Corporation?” and no, he is not haha. But what he did, was simplified Markowitz’s math into what he termed “Capital Asset Pricing Models” or CPAMs. This introduced the concept of systematic market risk and the process for separating market risk from stock-specific risk. Around the same time, in 1970, Eugene Fama formulated the Efficient Market Hypothesis (EMH), asserting that asset prices reflect all available information, making it impossible to consistently “beat” the market without taking on higher risk or utilizing algorithmic edge. This also reflected the general sentiment that the 70s were revolutionary and the halmark of a global connected world, with telephone systems now spanning globally, you could call your friend in Europe from New York, and the rise of more complex computers that could process more data and faster! These breakthroughs were seen in: The IBM 370 in 1970: Replaced magnetic core memory with silicon microchips and introduced Virtual Memory (VM), allowing programs to use far more memory than was physically installed. Also, letting my nerd show again, bears an awfully similar resemblance to the Red Queen in Resident Evil. Hmmm… The Cray-1 (1976): Designed by Seymour Cray, the Cray-1 was the world's first commercially successful vector supercomputer, operating at a record-breaking 160 MFLOPS (million floating-point operations per second). Other groundbreaking computers of the 1970s that directly impacted and lead to advancements in Quantitative and Banking analysis/data: And we would be nowhere in our discussion of the history of quantitative finance without discussing the fringe Edward Thorp, a math professor at MIT who used an IBM mainframe to invent card counting in Blackjack. When he successfully managed to do this, he turned the algorithms toward Wall Street, publishing “Beat the Market” in 1967. He went on to launch Convertible Arbitrage and founded Princeton/Newport Partners, running the first fully quantitative hedge fund using delta-neutral hedging. He was the first to also heavily use computers within his fund to perform his algorithm. Princeton/Newport went on to do an unbelievable 20 year run without a single losing quarter, only to be shut down due to some scandals unrelated to its actual success. Crazy honestly. If you compare Warren Buffet to Thorp, Buffett produced a higher raw annualized return over a longer time horizon, Thorp achieved something almost mathematically impossible: high returns with virtually zero downside risk. You can see why Math is king when it comes to markets. I will now idollize Thorp as my role model. Not to mention he is currently 94 and looks max 60. Jeeze. Anyway, in 1973, three interesting cats, Fischer Black, Myron Scholes, and Robert Merton, published what became known as the Black-Scholes Model for pricing European options. Options had been traded for centuries, but pricing them was pure guesswork because every existing model relied on predicting the underlying stock’s expected return, a nearly impossible variable to pinpoint. What this trio did was revolutionize financial math through a concept called Dynamic Delta Hedging (or no-arbitrage pricing). They proved that if you continuously buy and sell tiny fractions of the underlying stock to offset the risk of an option, you create a completely risk-free portfolio. And because the position becomes risk-free, the stock's expected return drops out of the equation entirely, replaced by the risk-free interest rate. Fischer Black even derived the core partial differential equation by transforming the financial problem into the Heat Equation from classical physics, treating option price movement like heat diffusing through a solid object! Their timing couldn't have been better: 1973 was the exact same year the Chicago Board Options Exchange (CBOE) opened. Within months, traders were walking onto the floor carrying Texas Instruments calculators programmed with the Black-Scholes formula, using it to instantly spot and exploit mispriced contracts on the fly. And from the work of those 3 Cats, came the arrival of TXN or Texas Instruments. In 1973, traders on the Chicago Board Options Exchange (CBOE) began carrying customized Texas Instruments Calculators onto the trading floor, programmed with the Black-Scholes formula to instantly spot mispriced options. Let’s roll up a picture of these 70s TXN calculators: The 80s Now on to one of my favourite decades, the 80s! Though I never lived them being a 90s child, I still find it interesting. So what happened here? Well, a man by the name of Jim Simons, a former Cold War codebreaker and MIT mathematician, founded RenTech and launched the Medallion Fund in 1988. Instead of using contemporary economic theory, Simons hired astrophysicists, signal processing engineers, and cryptographers, using Hidden Markov Models (originally used for speech recognition) to extract statistical noise patterns from market data. At the same, Morgan Stanley came up with the Automated Trading Desk or ATD. Nunzio Tartaglia, a nuclear physicist at Morgan Stanley, assembled a team of mathematicians to invent Pairs Trading, executing automated trades whenever two historically correlated stocks drifted apart. One of the more groundbreaking advancements in the 80s was the Bloomberg Terminal in 1981. Michael Bloomberg introduced it in 1981, brining real time price feeds, analytics and historical data directly onto trading desks. Meanwhile UNIX workstations (like Sun Microsystems) began replacing central mainframes on trading floors. The 90s Oh yeah, my year! The year of the Pentium processors and the Steversteves Birth. Some real advancements happened in the 90s in terms of quantiative finance, both in technological advancement, and in practice and process. One such example is the FIX protocol of 1992. The creation of the Financial Information eXchange (FIX) protocol standardized digital communication between brokerages and exchanges, laying the digital plumbing for modern algo order routing. The 80s and 90s were also a simplier and easier time for Quantitative finance. If you think about it, today we have billions of people accessing the markets, from their laptops, desktops, phones, anywhere. With thousands of trading platforms, access to high speed internet globally, the market has seen participants the likes of which it has never seen before! This was not the case in the 90s Dial up era. Market participants were first and foremost large market makers themselves, with indie retail mostly acting as investors, that is until the dotcom bust where more “retail” started entering into the market with software like Watcher. But that is for another post. Because of smaller participation, the 90s early algorithmic traders exploting NASDAQ’s small order execution system (SOES) to front run slow human floor brokers. Other things like Long Term Capital Management (LTCM) (1994 – 1998), founded by John Meriwether, used extreme leverage to exploit tiny fixed income arbitrage spreads. However, this ended badly for them when in 1998, Russia experienced a financial crisis and their theoretical models broke down, triggering a 3.6 billion dollar Wall Street bailout, the first major systemic warning about over-leveraged quantitative risk models. But the 90s were also interesting in its more democratic approach to quantitative finance. Having read this far, if you have, you will see that the major advancements were done on computers the size of living rooms and the costs of 50 homes back in the 70s and 50s. This left Quantitative computing out of reach for retail and only in reach for enterprise and academic institutions. However, in the 90s, computing was fairly even. Corporate and commercial servers utilized the same “Pentium” and “PowerPC” processors that consumers had access to. This was the true democratization of quantitative finance. In fact, famously a man by the name of David E. Shaw lead the way in monopolizing computing power in the 90s. Instead of buying expensive proprietary hardware (like Cray supercomputers), David Shaw realized he could assemble massive computational power at a fraction of the cost by buying standard, off-the-shelf consumer Intel PC components, specifically Intel Pentium and Pentium Pro processors, and running them in parallel using customized Linux/Unix cluster configurations. These are known as Beowulf clusters, with an image from the 90s below: There was one little nuance that was the halmark of a transition from analogue memory (up until the end of the 80s) to digital memory, and that was data archiving. In the 1990s, quantitative finance hit a brutal wall that had nothing to do with smart algorithms and everything to do with data architecture and accessibility. Before modern cloud APIs, real-time data feeds, and pre-packaged historical datasets existed, early quants spent over 80% of their time acting as data plumbers rather than mathematicians. Financial data was stored on massive physical reels of magnetic tape, floppy disks, or early CD-ROMs shipped via mail from vendors like CRSP or Compustat. Ingesting this data into early Unix servers or consumer PC clusters required writing custom parser scripts just to read raw byte streams, where a single corrupted bit on a magnetic tape could corrupt an entire backtest. Beyond physical access, the sheer "dirtiness" of historical data made quantitative backtesting a minefield: Survivorship Bias: Early databases only included companies that were currently active. If a firm went bankrupt in 1993, vendor databases simply erased it from history, making backtests look unrealistically profitable unless quants manually hunted down old microfiche records to rebuild dead stocks into their models. Corporate Actions & Splits: Stock splits, dividends, spin-offs, and ticker changes weren't automatically adjusted by data providers. A 2-for-1 stock split would register as a sudden 50% crash in price, triggering false algorithm sell signals unless clean, split-adjusted adjustments were applied by hand. Intraday Tick Data Gaps: Minute-by-minute tick data was practically non-existent or prohibitively expensive to store. Early high-frequency pioneers had to set up their own local hard drives to capture live satellite feeds in real time, dealing with frequent packet drops, missing timestamps, and out-of-order quotes. Firms like Renaissance Technologies and D. E. Shaw built their early competitive moats not just on better mathematical formulas, but on employing armies of engineers solely to clean, align, and store clean historical datasets that no one else on Wall Street possessed. So where is this data today? Good question, because I really want to know. As a quant trader myself, I would pay anything to get my hands on a CD ROM from the 90s that contains tick level or minute level or even 5 minute level data from the 90s, specifically before and during the dotcom boom! I have scoured online to find this data, or if I can purchase it from some source, but alas it really seems to be an enigma! If anyone is a keeper of archaic computers (like me) and comes across something like this, feel free to send it my way! YTK and GPUs There is a lot to unpack from this era, from regulations, the introduction into High Frequency Trading (HFT) and GPU hacking leading to the introduction of CUDA. So bear with me for this continued lengthy post! Regulation NMS and the HFT Big Bang (2005) Before 2005, human floor brokers on the NYSE could still manually hold onto orders for tens of seconds. That changed when the SEC enacted Regulation NMS (National Market System). The Order Protection Rule: Reg NMS mandated that a trade must be executed at the best available price across any electronic exchange in the country (the National Best Bid and Offer, or NBBO). The Result: If Exchange A had a stock listed for $100.00 and Exchange B had it for $99.99, an algorithm had to route the trade to Exchange B. This resulted in the Birth of HFT. Suddenly, speed was everything. Early HFT firms (like Getco, Tradebot, and Citadel) built ultra-low latency infrastructure, laying specialized fiber-optic cables through mountains and placing servers right inside exchange data centers (co-location) to front-run quote changes by microseconds. GPU Hacking Long before official machine learning tools existed, quants ran into a major CPU bottleneck: running thousands of simultaneous Monte Carlo simulations for derivative pricing or risk management took hours on x86 server farms. Around 2001, clever researchers realized that consumer 3D graphics cards (like the NVIDIA GeForce and ATI Radeon) were essentially massive parallel matrix math engines built to calculate light and pixels. Clever coders developed what was called the “Shader Hack”. Because GPUs only spoke "graphics languages" (like OpenGL and DirectX), early quants disguised raw financial data matrices as 2D image textures. They mapped linear algebra equations onto pixel shader operations, forcing a gaming card meant for Quake III or Doom 3 to crunch option pricing matrices 10x to 20x faster than an Intel CPU. NVIDIA was very much alive during this time and very much aware of these Shady Hacks, I mean, SHADER hacks. In fact, NVIDIA saw this as a marketing advantage for their components. In November 2006, NVIDIA officially unveiled CUDA (Compute Unified Device Architecture). This was the exact turning point for modern GPU compute. CUDA allowed software engineers to write standard C/C++ code directly for the GPU without needing to trick 3D graphics engines. Instead of running 4 or 8 threads on a high-end dual-socket server CPU, a single GPU could run thousands of lightweight parallel threads simultaneously. For quantitative desks, algorithms that previously took an overnight batch run on a server rack were suddenly executing in seconds on a single workstation fitted with NVIDIA Tesla cards. The Dark Side of YTK The 2000s also exposed the catastrophic tail-risk of relying too blindly on mathematical models: The August 2007 "Quant Quake": Over three trading days in August 2007, several major multi-billion-dollar statistical arbitrage funds suffered massive losses simultaneously. Because everyone was using similar factor models and statistical arbitrage algorithms, a single fund liquidating positions caused a domino effect that triggered automated stop-losses across the entire quant industry. The 2008 Financial Crisis & The Copula Failure: Wall Street used a famous mathematical formula called the Gaussian Copula (developed by David X. Li) to price complex credit derivatives like Collateralized Debt Obligations (CDOs). The model assumed historical correlation between mortgages would remain stable. When housing prices dropped nationally, mortgage defaults correlated at unprecedented levels, breaking the models and triggering the global credit freeze. This really highlighted then and currently, why Math isn’t always King, contrary to my personal bias. Fundamentals play a key role still in sifting through exuberance and over-extension to answer the “how”, “why” and “what if” questions that lead to bubbles popping and markets imploding (i.e. Michael Burry). Fast Track, Quantitative Trading of Today Between the late 1990s and the 2010s, the arrival of broadband internet, standardized financial APIs, and ultra-fast data feeds changed everything. Suddenly, historical stock prices, tick data, and market depth were no longer physical commodities locked inside specialized terminals; they were accessible to any algorithm via a web connection. Quants built high-frequency trading engines, co-located their servers right next to exchange matching engines, and ran automated statistical arbitrage strategies around the clock. Business as usual meant writing code to scrape raw price feeds, backtesting signals against decades of clean numeric data, and racing in microseconds to exploit tiny pricing mismatches. However, as every firm gained access to the exact same high-speed internet infrastructure and API data streams, the industry ran into a wall. The biggest structural problem quants faced as computing power exploded was Alpha Decay. The moment a pure mathematical pricing anomaly was discovered, thousands of automated algorithms front-ran it, arb’d it out, and crushed the profit margin down to zero. Pure statistical price-action models hit a wall because every algorithm on Wall Street was staring at the exact same structured price and volume data. To survive this wall, quant firms throughout the 2010s and early 2020s turned to mathematical engineering, feature expansion, and machine learning. Quantitative analysts shifted from simple statistical arbitrage to factor-based multi-factor models, statistical signal processing, and supervised machine learning algorithms like Gradient Boosted Decision Trees and XGBoost. They expanded their feature matrices by sourcing specialized, high-cost alternative datasets, tracking credit card transaction feeds, web-scraping e-commerce inventories, and logging supply chain manifests. The core objective of a quant analyst during this era was to hand-craft hyper-specific numerical features, feed them into mathematical models, and squeeze tiny, uncorrelated micro-edges out of structural market behavior before competing algorithms could discover them. This operating model reached another inflection point around 2023 with the sudden explosion of Generative AI and Large Language Models. Suddenly, firms scrambled to integrate LLMs into their investment stacks, expecting a gold rush of automated trading profits. But as the honeymoon phase settled into reality, many quant funds discovered a harsh truth: using AI to directly trade financial markets is surprisingly unhelpful, and often unsuited for generating direct trading alpha. Large language models are fundamentally non-deterministic, prone to hallucination, slow at low-latency execution, and notoriously poor at making numeric time-series predictions. When firms simply asked LLMs to generate trading strategies or predict price movements using public datasets, the models hallucinated, fit noise, or produced conventional, highly crowded signals that decayed almost immediately. Instead of discovering a magical AI money-printing machine, the quantitative finance world of today has adapted AI into a hyper-efficient operational backend rather than a primary trader. Rather than replacing quantitative mathematical models, AI serves as an intelligence filter, processing unstructured real-world data like SEC filings, transcript audio, news feeds, and legal disclosures, then converting that qualitative chaos into clean, structured numeric vectors. The modern quant stack uses AI models as pre-processors to digest complex qualitative realities, while leaving the actual pricing, execution, portfolio optimization, and risk management to traditional, time-tested mathematical and statistical engines. AI didn't replace quantitative math; it simply gave math a way to comprehend unstructured human language. And The Future is Now And that is where it stops. The evolution of quantitative finance. While there really is no more risk free trading like Thorp had in the 60s and 70s, arbitrage in one way or another still remains one of the gold standards for trading, it just looks different. From high frequency trading pricing microsecond mispricings on order flows, to AI scanning fundamental revenue sheets for sectors and finding PE ratios and revenue that is falling behind others in its basket (a current strategy I also use), we are kind of at a crossroads in Quantiative finance, in my honest opinion. Scrounging for a little bit of Alpha in an over-saturated market is hallmark of what is currently happening. Ultra-competitive firms, hiding their processes for fear of them being arbitraged out and using data that mostly is only available to HFT firms (such as direct line time and sales) are how most major Quantitative firms operate to this day. It makes it incredibly hard for a lil’ ol’ retail quant trader like me to pick up the scraps left behind because obviously we cannot compete with these HFT firms. But we can find signals buried in that noisy data, if you look in the right place and find the right filtering metrics. So what is next for Quant Finance? The biggest shaker-uper will be the release of Quantum computing. This will slam retail harder than they can expect, and hobbyist quant traders like me. Consumer Quantum computing will never be a thing in our lifetime, at least by the looks of it. The introduction of quantum computing will likely bring us back to 1950s, where the elite institutions continue the processing power needed to solve impossible questions. How quantum computing will work, is by generating billions of simulations and fitting price action to the single most similar simulation that matches. As well as tracking multiple order flows simultaneously across thousands of stocks. I am personally excited about Quantum computing, but preparing for it to completely destroy even more of the market than has already been destroyed with the current onslaught of quant firms and algo trading. I suspect it will be a dark day for retail quants when Quantum hits the trading stage, it will surpass anything that AI is capable of doing in terms of arbitrage and trading execution. Final Thoughts If you read this far, thanks! This was a long post and took some time for me to research and put together over the course of last week. I am a bit of a vintage computer collector myself, I still have my iBook Clamshell from 2001 from when I was a Teen and my Macbook Pro from 2009 as well as my Thinkpad from 2011. I was going to, as part of this post, run benchmarks of current quantitative methods I use to see how computing changed over the years. I was genuinely interested to see a PowerPC Clamshell at 466 MhZ process a 1 million simulation Monte Carlo, but that will probably be an idea for another post as this turned out to be long enough. I really hope you enjoyed, please like if you found this interesting to keep me motivated to do these long researched out posts. As always, safe trades everyone!