What vibe coding doesn't do for you

I vibe coded an app to charge my Tesla. It worked until the morning it didn't. Here's what I do differently now, and the Skill I open sourced to do it.

Last year I vibe coded an app to automate charging my Tesla. I wrote about it at the time, first the charging automation itself, then the two agents I now let run parts of my life. I described what I wanted, the agent built it, and it worked. It charged the car on cheap overnight rates, backed off when the Powerwall was low, the whole thing. For months it just ran.

Then one Saturday it charged for eight minutes, stopped, and told me nothing. I found out the next day, by accident. The code had over 400 passing tests. None of them was wrong. These tests just weren’t testing the thing that mattered.

When you describe what you want and let the agent build it, you get something fast, and it usually works for the case you were picturing when you asked. The agent will even write tests, because you can ask it to and it’s good at it. What it won’t do, unless you make it, is think about the things a working engineer thinks about without being told: what happens in the weird states, how this behaves six months from now, what breaks silently when a date rolls over or a service is down. It’s not trained to worry about that. It’s trained to solve the ask in front of it.

So every time I vibe coded, I got the same experience. Quick and impressive on the surface, and a pile of skeletons in the closet that only showed up later, quietly, in the one situation nobody thought to describe. The Tesla quietly stopped charging is just the cleanest example I have.

Here’s the change I made, and the strange part is it didn’t involve reading more code. I still don’t read the code the agent writes. I decided that wasn’t where my time should go. What I do now is work the process around the code instead.

Before anything gets built, I write a detailed spec. Not a list of functions, a list of real situations the thing has to work consistently. What happens when I ask for a charge on a weekday morning. What happens if I ask at ten to four, right before the peak rate starts. What happens on the day I’ve declared a road trip and the car needs to be full by 5:30 AM. Then I have the agent write test cases for each of those situations, end to end, following the actual user journey, not the little unit tests that check one function in isolation. Unit tests are what had me fooled the first time. They all passed and proved nothing about whether the car would actually charge. Then I bring in a separate adversary agent whose only job is to attack the spec, then the implementation etc, a fresh one at each phase so it isn’t grading its own work.

I have now turned this end to end working into a Skill, so I don’t have to reassemble it by hand every time. It automates the flow: write the spec, review the spec, write the user-journey test cases, do the development test-first against them, and at each phase hand the work to a new adversary that tries to break it. I’ve open sourced it, so you can read exactly what it does rather than take my word for it: https://github.com/srinatar/spec-driven-agent-build.

When I rebuilt the Tesla charger this way, the adversary found two bugs fast. The first: on a road-trip morning, a stale-session check fires at midnight and quietly caps the charge. I’d leave with a half-full car and no warning and every test would still show green. The second: the engine correctly decided to notify me in three situations, but the code that sends those messages had no template and hence it wasn’t sent. The decision was right; the message never went out. From the outside, that looks exactly like nothing running. That’s how the original bug hid for a day.

I want to be honest about the bet I’m making, because it’s the part any developer might push on - If you don’t read the code, how do you know it’s any good?

My answer is that I’m not claiming the code is clean. I’m claiming the behavior is checked from the outside, against situations I have captured in advance, by a dedicated LLM whose whole job is to find edge case scenarios. That’s a different guarantee than “I read it and it looked right,” and honestly I trust this process more, because I’ve read plenty of code that looked right.

What I have now is a way of building that keeps working, even during happy path and edge case scenarios. That’s what I was actually after.

I don’t know yet if this is the right way to build with these tools, and I’d genuinely like to hear how other people are handling it. It’s slower than the twenty-minute version. But I’ve stopped shipping things that pass every test and fail the one morning I needed them, and so far that trade has been worth it.

If you’re building this way too, or think I’ve got it wrong, tell me. I’m still figuring it out.