What I learned vibe coding with Claude Code and Codex

I’m a programmer. I know how to code and have earned a living from my coding skills. I also head up a company well stocked with professional programmers who have many more years of experience and far more ability than I do. Why, then, would I vibe code with an AI coding assistant like Claude Code or ChatGPT Codex instead of writing the code myself or handing the job to one of them?

The short answer is that the tools take me from idea to working software almost instantly. The longer answer is that 10 weeks of managing two AI programmers taught me more about managing programmers than I expected. I’m not saying these tools are about to replace the people at Watkyn. They aren’t. For my own personal projects, however, they let me test an idea in an evening that I would otherwise never have tested at all.

Why I vibe code

America’s economic growth in recent decades owes a lot to technology that made it cheaper for an entrepreneur to try an idea and find out quickly whether it was any good. Move fast, break things, see what works, and spend no more time or money than the experiment deserves. AI coding tools accelerate that process. In fact, if I get what I think is a great idea, I take it straight to an AI and talk it through from several angles: what’s the simplest version, who has already done it, what’s wrong with it. Vibe coding, for me and in my areas of personal interest, is a way to test an idea and sharpen a vision quickly. (It also means I don’t pull a Watkyn developer off client work to build a toy for the boss. Our people are paid to build systems that clients depend on, not my hobbies or personal interests.)

What I’ve used

Mainly, I use Claude and ChatGPT, the chat apps, for thinking a project through and organizing my own thoughts about something. I then turn to Claude Code and ChatGPT Codex, the coding agents, for actually building it. I’ve used Grok to write code alongside the other two as well. The agents run on a personal Mac mini, with access to a private GitHub repository in my personal account. Claude Code did most of the design-heavy early building; Codex did most of the release engineering afterward, through branches and pull requests that the other one reviewed.

The guitar app

The first thing I vibe coded was a web app for my guitar practice. (I’m a jazz vocalist and guitarist in my free time.) There are thousands of websites and apps for guitarists, but I wanted one specific thing with nothing else to distract me: a drill for quickly identifying and forming jazz chord voicings up and down the guitar fretboard. No lessons, no tabs, no social features, just the one exercise I was weak at, that I could practice over and over.

This app was built almost entirely with ChatGPT (the chat app, not Codex) on my laptop. I was amazed at how quickly it understood what I was trying to accomplish and even more amazed at how fast it got a real working app up and running for me in just a few hours of chatting back and forth. As I saw the app taking shape, I realized I needed to make little changes to how the app worked, and it was able to do that much faster than I could have myself.

This was an auspicious start with AI programming, but it also allowed me to see some of the potential pitfalls of relying on AI coding agents. I had it make some changes to the user interface, but a few turns later, while making another change I asked for, it reverted to the UI design I had explicitly told it I didn’t want. This same kind of thing happened over and over again.

Despite appearing to have immense “knowledge” of music theory, as well as an understanding of how a guitar fretboard works, the agent would regularly create chord voicings that weren’t quite right or would have been impossible for any guitarist with fewer than seven or eight fingers to play. When I would ask it which notes should be played in a particular chord, it always knew the right answer, but somehow that information was not consistently available to it when it was populating our database of chord voicings.

It was humorous to see how a system could be superhumanly capable and also innocently inept at the same time. Despite the setbacks, it was a useful experiment for me that produced a working app that I continue to enjoy using in my practice.

The app that argues with itself

The second project I want to talk about came much later and has been more helpful to me, both as an experiment in using AI coding agents and because the software product we created has been more useful.

Two heads are better than one, and I realized some time ago that I get better analysis and sharper criticism from several AI models from different vendors working on the same problem than from any one alone. In order to get that benefit, I adopted a manual method that was simple and tedious: copying and pasting between several AI models in different browser tabs. One flagship model would draft, another would critique, and the first would then evaluate the critique and choose how to revise. Meanwhile, I would be busy monitoring, clarifying, redirecting (and mostly copying and pasting) around and around 8 to 15 times until I had the document just how I wanted it.

The most important thing I learned from this process was that I kept going past the point where both models declared the document finished and couldn’t be improved. Pushing them past the point they both thought the document was “done” almost always turned up more real problems or considerations that had previously been unknown. I was always glad I didn’t accept earlier drafts as final even when both models were telling me there was no point in continuing to refine or polish them further. They were almost always wrong about that.

I used AI coding tools to turn that routine into an app, which I called Orchestrator. It runs a round-robin among several AI models centered on one prompt or one document, which gets drafted, critiqued, and revised by a customizable panel before being handed back to me. It started in late July as a Python command-line script. Within a week it was a TypeScript web app. By the end of August it had a tournament mode (several drafters compete and a judge picks the winner), a survey mode that puts one question to a whole panel, a signed Mac app that others on our team can now run, and a server interface that lets Claude Desktop, Claude Code, and Codex all call it as a tool. Since then it has been running as a hosted instance with automatic deploys and rollback. It works well (though there are a few bugs I still need to work out, and I have some new features I want to add too), and I use it regularly for both Watkyn work and personal things where I need lots of brainstorming, careful critique, or analysis from specific perspectives.

Mandate drift: where the agents went wrong

Now the unflattering part. I’m not trying to score points against either vendor; both did work I couldn’t have done alone in the time I had. But two things went wrong that no human programmer at Watkyn would have gotten away with, and neither one was a coding error. One was a lapse in discipline, the other an error of judgment about whose project this was.

The first was a matter of discipline. Orchestrator started life as a command-line tool, and when I added a web app on top of it, I assumed the two would grow together. They didn’t. The agents kept building out the web app and let the command line fall behind, so that a feature I could click on in the browser didn’t exist at the terminal, and vice versa. Nobody decided that; it just happened, one unsynchronized feature at a time, because no one was keeping a list and I hadn’t thought I needed to ask for one.

The second was stranger, and it’s the one you really need to be careful about if you’re using AI agents for vibe coding. I call it mandate drift. Somewhere along the way the agents adopted a quasi-scientific persona. They began treating Orchestrator as if it were an experimental apparatus whose purpose was to test whether round-robin exchanges among several AI models were worth the trouble and expense. I never thought of it that way. I never said anything that suggested it. The value of the round-robin was already settled in my mind, settled by months of doing it the hard way with copy and paste; I was building the tool because the workflow worked, not to find out whether it did. I had never given any kind of mandate to build this as if it were preparatory to running an experiment. But the lab-experiment mindset, once adopted, quietly shaped design and implementation decisions made on my behalf. The clearest example: the agents introduced a restriction that if a given model had written the draft, a different instance of the same model couldn’t be used to critique the draft as part of a critical panel. This would be sensible if we were controlling variables in a formal study, but it’s obstructive if you’re a user who wants the best critique available and has found that a model can be a perfectly good critic of its own work when asked.

Imagine hiring a contractor named Herbert to build a deck because you like sitting outside in the evening, and coming home to find he’s installed a turnstile and a clipboard so he can measure whether people really do sit outside. Herbert isn’t lazy. He has simply decided what the project is about without asking the person who actually knows.

That’s the common thread. An AI agent will fill any gap in your stated intent with an intent of its own, and it will do so confidently, consistently, and without telling you. The fix isn’t more autonomy or less. It’s writing down what the project is for, in plain words, where the agent will read it every session, and checking its work against that mandate rather than against whether the code runs.

What this means if you’re hiring us

Watkyn’s senior developers use AI in their own work, to plan builds, pressure-test data models, and generate scaffolding, and it makes them faster. What my hobby projects confirmed is where the value actually sits: in the hands of an experienced person who can tell when the agent is wrong, on the way to a system a human designed, verified, and stands behind. That’s the process on our methodology page, and it’s the difference between a prototype you can demo and a system a business can run on. If you’re weighing a self-built or AI-built app against a delivered one, our custom software development practice starts with a fixed-fee assessment that will tell you which path fits, including when the answer is that you don’t need us. (If what you’re replacing is a spreadsheet, start with outgrowing spreadsheets instead.)

Tell us what you're building, or what's stuck.

A 30-minute fit call is the fastest way to find out whether we're the right firm for it.

Questions, or something specific you're weighing? We're happy to talk: 201-644-5251.