Kubernetes is a complicated solution to a complicated problem. A lot of companies have different problems and should look for different solutions. But if you are facing this particular problem, Kubernetes is the way to go. The trick is to understand which problem you are facing.

> all of those choices have been made and agreed upon

Have they really? I have a few apps deployed on k8s and I feel like every time I need something, it turns out it doesn't do that and I'm into some exotic extension or plugin type ecosystem.

Something as simple as service autoscaling (this was a few years ago) was an adventure into DIY. Moving from google cloud to AWS was a complete writeoff almost - just build it again.

I'm sure it captures some layer of abstraction that's useful but my personal experience is it seems very thin and elusive.

Agree. For years I had developed my own preferred way of deploying Rails apps large and small on VMs: haproxy, nginx, supervisord, ufw, the actual deploy tooling (capistrano and other alternatives) and so on... and if those tools are old or defunct now it's because my knowledge of that world basically halted 8 years ago because I've never had to configure anything but k8s since then.

I've used it every day since then so I have the luxury of knowing it well. So the frustrations that the new or casual user may have are not the same for me.

lol, the big problem with kubernetes is that none of the choices have been made, it's not opinionated at all, there's no conventions. It's all configuration and choices all the way down. There's way too much yaml, and way to many choices for ever tiny component, it's just too much.

I do run a k3s cluster for home stuff... But I really wish I could get what it provides in a much simpler solution.

My dream solution would effectively do the same as k3s + storage, but with a much simpler config, zero yaml, zero choices for components, very limited configuration options, it should just do the right ting by default. Storage (both volume and s3), networking, scale to zero, functions, jobs, ingress, etc... should all just be built in.

> However, the big win with Kubernetes is that all of those choices have been made and agreed upon, and now you have an entire ecosystem of tools, expertise, blog posts, AI knowledge, etc, that knows the choices Kubernetes made and can interface with that. This is VERY powerful.

Yep! I am now using k8s even for small / 'single purpose' clusters just so I can keep renovate/argo/flux in the loop. Yes, I _could_ wire renovate up to some variables in a salt state or chef cookbook and merge that to `main` and then have the chef agent / salt minion pick up the new version(s) and roll them out gradually... but I don't need to, now!

I just feel like "you can do this with Kubernetes" is a slippery slope. "You can do X with Y, so use Y" is a great way to add a dependency, especially if it is "community vetted" already. Sometimes simple is better - you don't need to add anything that implements some of you logic as a dependency to stay DRY or whatever you want to call it.

It really feels like we are drowning in self-imposed tech debt and keep adding layers to try and hold it for just a while longer. Now that being said, there is no reason not to add Kubernetes once a sufficient overlap is achieved.

Ephemeral user accounts were agreed upon before that. The OG container

Docker and k8s are just wrappers around namespaces, cgroups, file system ACLs, some essential cli commands, which can also be configured per user.

We may be headed back there. Have seen some experiments leveraging Linux kernels BPF and sched_ext to fire off just the right sized compute schedule in response to sequences of specific BPF events.

Future "containers" may just be kernel processes and threads... again. Especially if enough human agency looks away from software as AI makes employment for enough people untenable. Why would those who remain want to manage kernels and k8s complexity?

Imo its less we agreed on k8s specifically and more we agreed to let people use all the free money to develop whatever was believed to make the job easier; but if the jobs go away then it's just more work for the few left

If you can solve the same problem in a simpler way without using k8s, that means k8s is not a zero cost abstraction.

It's not obligated to be, but it's also obvious why people would want it to be.

Yeah, I spent quite a bit of time learning Kubernetes, but now I'd use it to host a static webpage on a single server, over alternatives. It's so awesome.

Honestly the main problem is people using k8s for something that's like... a database, and an app, and maybe a second app, that all could be containers or just a systemd service.

And then they hit all the things that make sense in big company with like 40 services but very little in their context and complain that complex thing designed for complex interactions isn't simple

> This is VERY powerful

No argument there. The Toyota 5S-FE non-interference engine is a near indestructible 4 cylinder engine that's well documented, popular and you can purchase parts for pennies. It has powered 10 models of Camrys and Lexus and battle proven. You can expect any mechanic who has been a professional mechanic for the last 3 years know exactly what to do when it starts acting up. 1 out of 4 cars on the road have this engine or a close clone of it.

It's not what any reasonable person would use for a weedwhacker, lawnmower, pool pump or an air compressor.