I can so relate to this guy.

I'm "only" in my 50s now, but I am in my 50s now. For a couple of decades I've had this grand vision about how to build the perfect algorithmic trading system. The practical expression of this has come in fits and starts.

During the Pandemic, I finally devoted some time to building a bespoke server and hosting it co-located facility near a high-quality market data provider. I chose to do this before the software was ready in order to "light a fire under my a$$" and help motivate me to move things along. By the end of 2021 I had written, tested, and deployed a set of specialized clients written in rust to consume the market data and perform basic parsing of the real-time information. I stored the transformed output of these processes in log files as a temporary data sink that could represent the data "someday" moving downstream in a custom stream processing platform that would perform trades via an online broker.

And then, it stalled. There always seemed to be good reasons why I couldn't commit to one implementation strategy or another. Would I pass the data downstreeam via pipes? With Kafka? Via shared memory? Would the parsed input be represented via protobuf records? A custom binary format? Apache Arrow/Parquet format? Would the algorithms be expressed in custom rust code? Using Pandas(numpy)/Polars? Would I focus on my imagined "insights" into how prices move or make the transition to Machine Learning by trying to produce "special sauce" custom features that could be fed into models of various types? In the meantime, I have only collected a high volume of daily log files.

There has been some progress, of course. Last year, with LLMs as my pair programmer, I added vector processing to the ingest components, which made them run even more efficiently. I certainly enjoy the sporadic work I dedicate to working on the system. But there always seems to be a fog of possibilities ahead of me obscuring my view towards a broker API getting called.

My family and closest friends look upon this as the "hot rod in his garage" which I sometimes work on and which will probably never get finished.