I'm a non-coder who works in a small SaaS business. I first heard about Claude at this Substack, and recommended it when my team decided to subscribe to an LLM less than a year ago (This was based on my superficial impression that it was superior to ChatGPT when it came to coding tasks). Even though I'm not a heavy user, I've been able to translate customer asks into Product Requirements Documents and build/test/ship actual working software and/or product enhancements in as little as a few days. Some of these accomplishments would have taken our lead developer weeks or months to build. Reading your essay my first thought was "have you used Opus 4.6 or Sonnet 4.6 in the last month?" It is scary close to being able to do right now what you are suggesting. A better audience for your essay might actually be Claude itself. I generally detest when folks post AI-generated responses to articles or blog posts, but for those curious to how Claude responded to your essay, here goes:
"This resonated with me — especially the principle that the AI should prompt the human rather than the other way around. That's the right mental model, and I think it applies well beyond software development.
I wanted to share some thoughts on how this could work in practice, because I think there's a version of your vision that's very achievable right now — and it's worth being clear-eyed about where the boundaries are.
The AI-as-business-analyst concept is the strongest part of this. Today, you could build a structured AI workflow that interviews a domain expert exactly as you describe — asking about entities, relationships, statuses, and CRUD permissions — and produces concrete deliverables: an entity-relationship model, a CRUD matrix, role mappings, and user stories. That's real and valuable. For someone building courseware, that interview process could save weeks of requirements work.
Where I'd temper expectations is the jump from requirements to finished application. You write that after the interview, the AI "ought to be able to develop the application without much further direction" — but that's where the hardest problems live. Edge cases, security (FERPA in your example), error handling, integrations with existing university systems, performance under load, deployment, and ongoing maintenance. These aren't things a better requirements process eliminates; they're engineering realities that emerge during build-out.
I'd also offer a gentle counterpoint on the "UI should be English" idea. Natural language is powerful but inherently ambiguous. Sometimes a structured interface — a dropdown, a calendar picker, a status badge — communicates more clearly than a text box, because it shows the user what's possible without them having to guess. The goal isn't zero UI; it's UI that doesn't require a manual.
The practical path I'd suggest: use AI for the structured interview and requirements deliverables, then feed those into a coding tool like Claude Code to scaffold the application. A human stays in the loop for judgment calls. It's not the full autonomy you're describing, but it dramatically compresses the timeline and keeps the non-technical person in the driver's seat. That's a version of your dream that works today."
My son, whom I trust on software issues (Ph.D. in EECS, teaching software at UC Berkeley), doesn't think AI software is that good, and the architecture is even more problematic.
The underlying question might be why we vibe code at all. If the answer is that it is trendy or simply easier, that probably misses the point. What it really reflects is a shift in what we need from software development. Vibe coding is interesting because it reduces some of that distance between the need and the product, and shortens the feedback loop …
I honestly don't see the gain here. I suppose it'd be fun but programmers don't eat that much. In the software engineering literature, we're 5% of cost. The very process of mapping the informal, human-language part onto the formal, machine-ey-language part seems fraught as a task for AI. If an AI has a github-like set of templates, maybe but even then...
The hard part is renormalizing disparate requirements into a coherent whole. Even with humans, that's iterative and a great deal of work.
I'd say the real gain would be for AI to emerge to enable closing the gap with provably correct software.
And your "menu bloat" problem is very often to offer a scripting interface for obscure cases. Works a treat.
I like the notion of AI learning to prompt the human instead of the human learning to prompt the AI.
At work, I’m experiencing a roll-out of AI (a “walled garden” implementation of ChatGPT plus Microsoft Copilot) and it is clunky at best.
I’ve done the training for pseudo-ChatGPT, but don’t see the value yet. I can do the tasks I would prompt it to do in the same time it would take to sort out the prompts. And I’m still skeptical of its summarizing ability (plus, I feel I don’t learn as much or as thoroughly having AI summarize things).
As for Copilot…it’s the return of Clippy, and I despise it.
Bear in mind the accountability aspect. Since an AI does not currently have any kind of long-term relationship with you--it gets restarted, upgraded, and discarded all the time--there needs to be some other way for the operator to verify that what they wanted is what they are getting. When I grope for a job that remains for humans to do, these long-lived relationships seem like one of them.
Reading the code seems like a big one. Even without the skill set to write a lot of code by hand, it is within the grasp of just about anyone to read through code and understand the overall structure and what the different routines are. If that's not possible, then distrust the results. If the outline looks like a giant confusing hash, then distrust the result. This is a standard technique already for working with an intern or an overly enthusiastic PhD. Experienced engineers will do this to their own work once they realize it's going to be applied to them from outside. A reliable solution really ought to have a set of major components that form it that you can make sense of.
Another big one is acceptance tests. Describe in English the acceptance tests you want, and ask it to generate machine-checkable tests that match what you want. Then, go read the tests!
* For a UI-heavy app, there are frameworks that will give you screenshots in a story-board style for a UI test. You can ask the AI for these.
* For a data processing app, you might want examples of input and output data that use a small enough amount of data you can verify it.
Again, these things are also done with human engineers. Getting back to the beginning, though, the big difference is how easy an AI agent can cut tail and run compared to a human agent.
> Even without the skill set to write a lot of code by hand, it is within the grasp of just about anyone to read through code and understand the overall structure and what the different routines are.
I don't think it is within the grasp of just about anyone. Programming languages have too many subtle and not-so-subtle differences for any random stranger to pick up on what they do. Functions have side effects. They can invoke external programs in different ways. Even loops and blocks and scope differ. Some function calls can pass local variables by reference.
It's like saying any random stranger can understand how an engine works. Not only are there different kinds of engines (piston, turbine, rocket, electric, steam), there are subcategory differences (flathead, overhead cam, double overhead cam; axial, compressor, ramjet, pulsejet) and all sorts of accessories (water pump, oil pump, fuel pump, spark plugs, distributors). A random stranger might be able to recognize the drive shaft, but how can a differential be understood without taking it apart?
Imagine asking for a car, though. You may not know how the engine works but should be able to inspect that it has one engine. If it has zero engines or 100 engines, you do not have to know how cars work to know that's wrong.
I'm a non-coder who works in a small SaaS business. I first heard about Claude at this Substack, and recommended it when my team decided to subscribe to an LLM less than a year ago (This was based on my superficial impression that it was superior to ChatGPT when it came to coding tasks). Even though I'm not a heavy user, I've been able to translate customer asks into Product Requirements Documents and build/test/ship actual working software and/or product enhancements in as little as a few days. Some of these accomplishments would have taken our lead developer weeks or months to build. Reading your essay my first thought was "have you used Opus 4.6 or Sonnet 4.6 in the last month?" It is scary close to being able to do right now what you are suggesting. A better audience for your essay might actually be Claude itself. I generally detest when folks post AI-generated responses to articles or blog posts, but for those curious to how Claude responded to your essay, here goes:
"This resonated with me — especially the principle that the AI should prompt the human rather than the other way around. That's the right mental model, and I think it applies well beyond software development.
I wanted to share some thoughts on how this could work in practice, because I think there's a version of your vision that's very achievable right now — and it's worth being clear-eyed about where the boundaries are.
The AI-as-business-analyst concept is the strongest part of this. Today, you could build a structured AI workflow that interviews a domain expert exactly as you describe — asking about entities, relationships, statuses, and CRUD permissions — and produces concrete deliverables: an entity-relationship model, a CRUD matrix, role mappings, and user stories. That's real and valuable. For someone building courseware, that interview process could save weeks of requirements work.
Where I'd temper expectations is the jump from requirements to finished application. You write that after the interview, the AI "ought to be able to develop the application without much further direction" — but that's where the hardest problems live. Edge cases, security (FERPA in your example), error handling, integrations with existing university systems, performance under load, deployment, and ongoing maintenance. These aren't things a better requirements process eliminates; they're engineering realities that emerge during build-out.
I'd also offer a gentle counterpoint on the "UI should be English" idea. Natural language is powerful but inherently ambiguous. Sometimes a structured interface — a dropdown, a calendar picker, a status badge — communicates more clearly than a text box, because it shows the user what's possible without them having to guess. The goal isn't zero UI; it's UI that doesn't require a manual.
The practical path I'd suggest: use AI for the structured interview and requirements deliverables, then feed those into a coding tool like Claude Code to scaffold the application. A human stays in the loop for judgment calls. It's not the full autonomy you're describing, but it dramatically compresses the timeline and keeps the non-technical person in the driver's seat. That's a version of your dream that works today."
My son, whom I trust on software issues (Ph.D. in EECS, teaching software at UC Berkeley), doesn't think AI software is that good, and the architecture is even more problematic.
The underlying question might be why we vibe code at all. If the answer is that it is trendy or simply easier, that probably misses the point. What it really reflects is a shift in what we need from software development. Vibe coding is interesting because it reduces some of that distance between the need and the product, and shortens the feedback loop …
I honestly don't see the gain here. I suppose it'd be fun but programmers don't eat that much. In the software engineering literature, we're 5% of cost. The very process of mapping the informal, human-language part onto the formal, machine-ey-language part seems fraught as a task for AI. If an AI has a github-like set of templates, maybe but even then...
The hard part is renormalizing disparate requirements into a coherent whole. Even with humans, that's iterative and a great deal of work.
I'd say the real gain would be for AI to emerge to enable closing the gap with provably correct software.
And your "menu bloat" problem is very often to offer a scripting interface for obscure cases. Works a treat.
I like the notion of AI learning to prompt the human instead of the human learning to prompt the AI.
At work, I’m experiencing a roll-out of AI (a “walled garden” implementation of ChatGPT plus Microsoft Copilot) and it is clunky at best.
I’ve done the training for pseudo-ChatGPT, but don’t see the value yet. I can do the tasks I would prompt it to do in the same time it would take to sort out the prompts. And I’m still skeptical of its summarizing ability (plus, I feel I don’t learn as much or as thoroughly having AI summarize things).
As for Copilot…it’s the return of Clippy, and I despise it.
I agree about the direction.
Bear in mind the accountability aspect. Since an AI does not currently have any kind of long-term relationship with you--it gets restarted, upgraded, and discarded all the time--there needs to be some other way for the operator to verify that what they wanted is what they are getting. When I grope for a job that remains for humans to do, these long-lived relationships seem like one of them.
Reading the code seems like a big one. Even without the skill set to write a lot of code by hand, it is within the grasp of just about anyone to read through code and understand the overall structure and what the different routines are. If that's not possible, then distrust the results. If the outline looks like a giant confusing hash, then distrust the result. This is a standard technique already for working with an intern or an overly enthusiastic PhD. Experienced engineers will do this to their own work once they realize it's going to be applied to them from outside. A reliable solution really ought to have a set of major components that form it that you can make sense of.
Another big one is acceptance tests. Describe in English the acceptance tests you want, and ask it to generate machine-checkable tests that match what you want. Then, go read the tests!
* For a UI-heavy app, there are frameworks that will give you screenshots in a story-board style for a UI test. You can ask the AI for these.
* For a data processing app, you might want examples of input and output data that use a small enough amount of data you can verify it.
Again, these things are also done with human engineers. Getting back to the beginning, though, the big difference is how easy an AI agent can cut tail and run compared to a human agent.
> Even without the skill set to write a lot of code by hand, it is within the grasp of just about anyone to read through code and understand the overall structure and what the different routines are.
I don't think it is within the grasp of just about anyone. Programming languages have too many subtle and not-so-subtle differences for any random stranger to pick up on what they do. Functions have side effects. They can invoke external programs in different ways. Even loops and blocks and scope differ. Some function calls can pass local variables by reference.
It's like saying any random stranger can understand how an engine works. Not only are there different kinds of engines (piston, turbine, rocket, electric, steam), there are subcategory differences (flathead, overhead cam, double overhead cam; axial, compressor, ramjet, pulsejet) and all sorts of accessories (water pump, oil pump, fuel pump, spark plugs, distributors). A random stranger might be able to recognize the drive shaft, but how can a differential be understood without taking it apart?
I agree in general.
Imagine asking for a car, though. You may not know how the engine works but should be able to inspect that it has one engine. If it has zero engines or 100 engines, you do not have to know how cars work to know that's wrong.
But that kind of superficial inspection is useless with hundreds or thousands of lines of code.
Positive vision. Enjoyable writing style. Problem solving mindset. Thanks again. Who will implement?