alt Hacker NewsGreat points, and this is a problem we have spent a lot of time thinking about as well. Our current hypothesis, is that the models will reach (and have already arguably come very close) a level where problems like these can be resolved without any human intervention needed. That doesn't mean that problems won't arise, they for sure will (nothing will ever get to 100%), but the increasing strength of the agents will allow resolving those issues to become very consumer-friendly.
I would also encourage you to take a look at our benchmarks (linked in the post), that show how the gaurdrails we have in-place allow us to deliver pretty strong results to our users. And as a last point, most of our users are not creating business-critical features with Vendo right now. For a lot of them, it is an additional feature that they wished they had but didn't. And for the companies we work with, the benefit of providing their customers with this level of customization outweigths the possible risk that you mention.
Would love to hear more of your thoughts, thanks for the feedback!
> Our current hypothesis, is that the models will reach (and have already arguably come very close) a level where problems like these can be resolved without any human intervention needed.
Frankly I don’t think this is that relevant. When software breaks, most people don’t want to get support from another piece of software. They want human support. As the models get better, I’d wager the problems that fall through will get increasingly messy by nature of a) being hard enough to stump the model or b) the model running in circles making changes that obscure or worsen the problem. Especially if you have non-technical users who can’t accurately describe what they want or the problem with the current state.
Benchmarks and guardrails are nice and all, but as you said problems will never be 100% self correctable. Models are good enough until they aren’t. I like to take a “what’s the worst that could happen” approach to planning support. Murphy’s law and all that.
> And as a last point, most of our users are not creating business-critical features with Vendo right now.
Just give it some time… you’d be surprised what becomes business critical in the eyes of a customer. Sooner or later a report will become part of an accounting workflow etc, and it’ll break on the wrong day and suddenly payroll is late. You know how many Excel sheets are “business critical”? Same thing.
An alternative model you might consider is selling direct to the end user companies and taking a partnership approach to the apps you work with. End users come straight to you for help, you have a list of integrated apps you already support with the required plan/license from that vendor plus custom MCP support and features that don’t need an API, then sell professional services engagements if an end user client wants you to work with one of their vendors to set up a new integrated app. That way you reduce support friction, and your end users are aligned with your buyers.