We gave 5 LLMs $100K to trade stocks for 8 months

Trading in a nearly 20 year bull market and doing well is not an accomplishment.

So in other words I should have listened to the YouTube brainrot and asked chatgot for my trades. Sigh.

This is really dumb. Because the models themselves, like markets, are indeterministic. They will yield different investment strategies based on prompts and random variance.

This is a really dumb measurement.

Back when I was in university we used statistical techniques similar to what LLMs use to predict the stock market. It's not a surprise that LLMs would do well over this time period. The problem is that when the market turns and bucks trends they don't do so well, you need to intervene.

lolol Gemini

Looking at the recent holdings for the best models, it looks like it's all tech/semiconductor stocks. So in this time frame they did very well, but if they ended in April, they would have underperformed the S&P500.

Could they give some random people (i volunteer) 100k for 8 months? ...as a control

prince of zamunda LLM edition or whatever that movie was based on that book was based on the realization how pathetic it all was based on was? .... yeah, some did a good one on ya. just imagine evaluating that offspring one or two generations later ... ffs, this is sooooooooooooooo embarrassing

“Everyone (including LLMs) is a genius in a bull market.”

Yea, so this is bullshit. An approximation of reality still isn’t reality. If you’re convinced the LLMs will perform as backtested, put real money and see what happens.

>We gave each of five LLMs $100K in paper money

Stopped reading after “paper money”

Source: quant trader. paper trading does not incorporate market impact

tl;dr https://www.aitradearena.com/blog/llm-performance-chart.png

If I'm reading this, almost all of Grok's advantage comes from heavy bets into semi-conductors spiking: ASML, INTC, MU.

Update with Gemini 3. It's far better than its predecessors.