logoalt Hacker News

quaintdevyesterday at 5:09 PM7 repliesview on HN

Begs the question if we should move on to minimal microservices so that whole project lives in context of llm. I hardly have to do anything when I'm working with small project with llm.


Replies

mlsuyesterday at 5:14 PM

Why not take it a step further? Make each function in the codebase its own project. Then the codebase can fit into the context window easily. All you have to do is debug issues between functions calling each other.

show 1 reply
Aurornisyesterday at 5:15 PM

In my experience, the result is just more crawling across the separate microservices and additional reasoning to confirm how it all fits together.

The monolithic codebases are easier to crawl for any problem that can't be conveniently isolated to a single microservice.

show 2 replies
dwedgeyesterday at 6:05 PM

Ironically this is accidentally begging the question - that breaking them up into LLM context windows would be good because it would be to fit them in LLM context windows.

Maybe you're right but I'm aghast at how much of engineering over the last 15 years has been breaking up working monoliths to fit better within the budget of an external provider (first it was AWS). Those prices can change.

There are good reasons to use microservices but so often they're used for the wrong reasons.

hadlockyesterday at 5:30 PM

I've done the opposite, moving multiple tightly coupled repos into a single monorepo. Saves the step of the llm realizing there's a bigger context, finding the repo, then also scanning/searching it. Especially for fixes that are simply one line each in two repos.

show 1 reply
swader999yesterday at 5:51 PM

Generally speaking no. Treat your IP (the code that runs your business, makes your business competitive or special) as precious and don't make it subservient to infra. It should be in the format (code, architecture, structure) that best serves it.

show 1 reply
giantg2yesterday at 5:14 PM

Orchestration between those services and the integration testing for any reasonably complex change can still be quite large.

Retr0idyesterday at 5:14 PM

The whole service might fit in a context window but the details of the system around it will still be relevant.