Every developer I know uses AI tools now. Copilot suggestions appear mid-keystroke. ChatGPT tabs stay permanently open. Claude conversations stretch across multiple projects. The adoption curve was vertical, faster than any technology shift I have witnessed in two decades of software engineering.
But here is the uncomfortable truth: most of us are using these tools without any real methodology. We treat AI as fancy autocomplete, accepting plausible output and hoping for the best. We have adopted revolutionary capability while keeping amateur practices.
This is the unstructured AI problem, and it is quietly undermining our codebases.
How Developers Actually Use AI Today
Let me describe what I observe in my own work and in conversations with other developers. I am not here to judge; I am here to name what is happening.
The typical AI interaction looks something like this: You have a task. You open your AI tool. You describe what you need, often in a single sentence or a rough paragraph. The AI produces something. You glance at it. It looks reasonable. You paste it into your codebase, maybe tweak a variable name or two, and move on.
Sometimes you iterate. The output was not quite right, so you add a follow-up prompt. “Actually, can you make it handle null values?” The AI adjusts. You accept the new version. Done.
This workflow has three characteristics that define unstructured AI usage.
First, the input is vague. We describe what we want in the same casual language we might use with a human colleague who already knows our codebase, our constraints, and our preferences. But the AI knows none of these things. It fills gaps with assumptions, and we rarely notice because the output looks syntactically correct.
Consider how differently we would approach a specification for a junior developer. We would explain the context: what the system does, how this piece fits, what patterns we use. We would state constraints: which libraries to use, what performance characteristics matter, what edge cases to handle. We would describe the interfaces: what the inputs look like, what the outputs should be, what errors might occur.
With AI, we skip all of this. We write “create a function that validates email addresses” and accept whatever the AI produces. We do not specify whether we need RFC-compliant validation or simple pattern matching. We do not mention whether we need to handle internationalized domain names. We do not explain what should happen with edge cases like empty strings or strings with leading whitespace. The AI fills these gaps with its own assumptions, and we inherit those assumptions without knowing what they are.
Second, the evaluation is shallow. We check that the code compiles. We check that it does roughly what we asked. We do not systematically verify edge cases, security implications, performance characteristics, or architectural fit. We trust the AI because its output is confident and coherent.
This shallow evaluation often comes down to time pressure. We asked the AI for help because we wanted to move faster. Taking time to carefully evaluate the output feels like it defeats the purpose. So we skim. We look for obvious problems. We check that the happy path works. And we move on.
Third, the integration is immediate. Code moves from AI output to production path with minimal intermediate steps. We skip the practices we would apply to our own code or to code from a new team member: careful review, testing against specifications, consideration of system-level impact.
When a new team member submits a pull request, we review it carefully. We ask questions about their choices. We suggest alternatives. We ensure it fits our patterns. AI-generated code receives no such scrutiny. It arrives pre-formatted, pre-commented, and seemingly complete. The very polish that makes it easy to accept is what makes us skip the steps that would reveal its flaws.
This is not carelessness. It is the natural result of tools that are easy to use and produce output that appears correct. The path of least resistance leads directly to unstructured usage.
The “Looks Reasonable” Trap
AI-generated code has a dangerous property: it almost always looks reasonable.
When a human writes problematic code, it often shows visible signs of confusion. Awkward naming. Commented-out experiments. Inconsistent patterns. These signals invite scrutiny. We know to look more carefully when code looks uncertain.
AI-generated code carries no such signals. It presents with uniform confidence. Variable names are sensible. Structure is clean. Comments, when present, are articulate. The code reads like it was written by someone who knew exactly what they were doing.
This creates a psychological trap. Our brains use fluency as a proxy for correctness. When something is easy to read and understand, we assume it is more likely to be right. This heuristic serves us well in most contexts, but it fails catastrophically with AI-generated code.
I have seen AI produce beautifully structured functions that silently fail on boundary conditions. I have seen it generate elegant API handlers with subtle security vulnerabilities. I have seen it write clear, readable algorithms that are O(n²) when the problem demands O(n). In each case, the code looked reasonable. It looked like code I might have written myself. And that appearance of competence made it harder to catch the underlying problems.
The “looks reasonable” trap is especially dangerous because it compounds. Each piece of AI-generated code that passes casual review reinforces the habit of casual review. We build confidence in a process that does not deserve our confidence.
Hidden Costs That Emerge Over Time
Unstructured AI usage does not produce immediate disasters. If it did, we would stop. Instead, it produces slow accumulation of problems that become visible only in retrospect.
Subtle bugs that escape initial testing. AI-generated code often handles the happy path correctly while mishandling edge cases. These bugs may not surface for weeks or months, emerging only when specific conditions align. By then, the code has been integrated, built upon, and is expensive to change. Tracking the bug back to its source reveals a piece of AI-generated code that was never rigorously evaluated.
Security vulnerabilities that await discovery. AI models are trained on vast amounts of code, including code with security flaws. They reproduce patterns without understanding their security implications. A function that “looks reasonable” might use deprecated cryptographic approaches, fail to sanitize inputs properly, or expose timing vulnerabilities. These issues hide in plain sight until a security audit or, worse, an actual breach exposes them.
Architectural drift that accumulates invisibly. Each piece of AI-generated code brings its own implicit assumptions about structure, patterns, and conventions. When these assumptions differ from your codebase’s architecture, the drift is rarely dramatic enough to catch attention. But over dozens of AI-assisted additions, your codebase becomes inconsistent. Patterns that should be uniform are implemented three different ways. Abstractions that should be shared are reinvented repeatedly. The system becomes harder to understand and maintain, not because any single addition was wrong, but because the additions were not coordinated.
Technical debt that defies measurement. Traditional technical debt comes with awareness. You know when you are cutting a corner. You might even leave a TODO comment. AI-generated technical debt is invisible because you never knew the corner was being cut. The AI produces code that works but is not optimal: inefficient data structures, unnecessarily complex logic, or approaches that will scale poorly. You accept this debt without knowing it exists, and you carry it forward with each build.
Skill erosion that happens without notice. In my first post in this series, I discussed how relying on AI for implementation erodes the skills needed to evaluate AI output. This is perhaps the most insidious hidden cost. The more you use unstructured AI assistance, the less capable you become of catching its mistakes. Your evaluation abilities atrophy precisely when you need them most.
These costs do not announce themselves. They accumulate quietly, manifesting as systems that are harder to maintain, harder to extend, and harder to trust. By the time the costs become visible, they are deeply embedded.
The Gap Between Capability and Reliability
Here is a distinction that unstructured AI usage fails to make: AI tools are highly capable but not highly reliable.
Capability means the AI can produce correct, useful output for a given task. Current AI tools have remarkable capability. They can write functioning code in dozens of languages. They can implement complex algorithms. They can generate tests, documentation, and infrastructure configurations. The ceiling of what they can do is impressively high.
Reliability means the AI consistently produces correct output across varied conditions. This is where current AI tools fall short. The same prompt might produce excellent code on one attempt and subtly flawed code on another. A task that worked perfectly yesterday might fail strangely today. The AI does not know when it is uncertain, and it cannot reliably distinguish between tasks it can handle well and tasks where it will struggle.
Unstructured AI usage treats capability as if it were reliability. Because the AI can produce correct code, we assume it will produce correct code. We skip verification because verification seems redundant when the tool is so capable.
This is the fundamental category error. Capability tells you what is possible. Reliability tells you what to expect. Professional software development requires reliability. We cannot ship code that might be correct. We need code that we have verified to be correct.
The gap between capability and reliability is not a temporary limitation that will disappear with the next model release. It reflects something deeper about how these systems work. AI models generate plausible output based on patterns in training data. They do not have access to ground truth about your specific requirements, your specific codebase, or your specific constraints. They cannot verify their own output against your actual needs.
Think about what the AI actually has access to when it generates code for you. It has your prompt, which is typically incomplete. It has whatever context you provided, which is typically minimal. It has patterns learned from millions of code examples, most of which have no relationship to your system. What it does not have is understanding. It does not know why your system is architected the way it is. It does not know which constraints are load-bearing and which are historical accidents. It does not know which parts of your codebase are well-tested and which are fragile.
The AI generates code that is statistically similar to correct code. Most of the time, statistically similar is close enough to be useful. But most of the time is not the same as reliable. Reliability requires understanding, and understanding requires context that the AI simply does not possess.
This gap must be bridged by human judgment and structured process. That bridging is exactly what unstructured AI usage omits.
Why Ad-Hoc Worked Before and Fails Now
Developers have always used external resources to help write code. Stack Overflow answers. Documentation examples. Code from other projects. Library sample code. We copy, adapt, and integrate code from outside sources constantly.
For these sources, ad-hoc usage worked reasonably well. Why does it fail for AI?
The key difference is scope and confidence.
When you copy code from Stack Overflow, you typically copy small, focused snippets. A regex pattern. A configuration block. A utility function. The snippet solves a narrow problem, and its narrowness makes it easier to evaluate. You can read five lines of code and understand what they do.
AI-generated code can be arbitrarily large and complex. A single prompt might produce dozens or hundreds of lines. The scope of what you need to evaluate expands dramatically, while the time you allocate to evaluation does not expand to match.
When you copy code from documentation or sample projects, that code has been written for explanation. It is explicit about its assumptions. It often includes comments describing limitations or requirements. The code signals its own context.
AI-generated code comes with no such signals. It looks like production code, not like example code. It presents its assumptions implicitly, buried in implementation choices rather than stated explicitly. You must infer what the AI assumed, and those inferences are easy to miss.
When you adapt code from other sources, you typically recognize it as foreign. You know it came from elsewhere. This awareness triggers scrutiny. You mentally flag the code as “needs review” because it was not written with your specific context in mind.
AI-generated code triggers no such flag. It arrives in response to your prompt, addressing your described need. It feels like it was written for you, even though the AI has no real understanding of your situation. The appearance of customization suppresses the scrutiny that foreign code would receive.
There is also a psychological dimension. When you copy code from Stack Overflow, you know you are borrowing. You maintain an appropriate humility about the borrowed code. AI-generated code feels collaborative, almost co-authored. You described the problem; the AI wrote the solution. This sense of partnership creates false confidence. You feel like you understand the code because you participated in its creation, even though your participation was limited to describing the desired outcome.
These differences mean that practices which were adequate for other external code sources are inadequate for AI-generated code. The scope is larger, the signals are weaker, and the apparent customization is deceiving. We need new practices for this new source of code.
Setting Up the Need for Methodology
I have spent this post diagnosing the problem. Let me preview the direction of the solution.
What we need is methodology: a structured approach that matches the nature of AI-assisted development. Not tips and tricks. Not prompt engineering hacks. A coherent discipline that addresses the gap between AI capability and AI reliability.
This methodology must do several things.
It must front-load clarity. If vague input produces unreliable output, we need practices that ensure our input is not vague. We need specification disciplines that surface ambiguity before generation, not after.
It must enforce evaluation. If the “looks reasonable” trap leads to accepting flawed code, we need systematic evaluation practices that catch what casual review misses. We need checklists, heuristics, and habits that apply human judgment consistently.
It must structure integration. If immediate integration bypasses quality gates, we need defined checkpoints that code must pass before reaching production. We need the same rigor we would apply to any other high-risk addition to our codebase.
It must maintain skills. If AI usage erodes evaluation capacity, we need deliberate practices that preserve and develop those skills. We need to remain capable of the judgment that methodology demands.
In the next post, I will trace the history of how methodologies emerge when existing approaches no longer fit reality. Waterfall gave way to Agile. Test-Driven Development inverted assumptions about when tests should be written. Each methodology emerged because practitioners recognized that new conditions demanded new discipline.
AI-assisted development is such a condition. The methodology I call ADD (AI-Driven Development) is the discipline it demands.
Let’s Continue the Conversation
I am curious about your experience:
Have you noticed the “looks reasonable” trap in your own work? What hidden costs have emerged from AI-generated code in your projects?
What evaluation practices, if any, have you developed for AI-generated code?
Share your thoughts in the comments or reach out directly. These observations from practicing developers will shape how I present the methodology in upcoming posts.