In Peril

A Few Thoughts on LLM-Driven Development

  • software development
  • llm

The Last Two Years

For the first half of 2025 (and earlier), I wasn’t impressed with the capabilities of LLM coding tools. I do front-end work, so lots of JS, React, CSS, and visual decisions – I’ve suspected that experiences might have differed based on task and domain. Simple asks would result in code with errors or that didn’t accomplish the goal I’d specified. There were some points where I’d ask for something that was a one-shot small task, and it would be successful, but the quality would be low. The only real plus was speed – the models could crank out code dozens of times faster than I could write.

In the second half of 2025, things improved. Models could write solid code, sometimes surprisingly good code when provided with clear instruction. At some point I tried to have ChatGPT create a small personal website and was not impressed with the code or the design. At a later point, I asked it create some Factorio blueprint strings Factorio lets players use base64 encoded JSON to import and export factory designs. This makes it an interesting test for an LLM, because its easy to generate something that looks like a valid string, but if you don’t follow the proper encoding process you’re likely to fail. for me; and while it asked relevant questions and could have a useful discussion about factory design in the game, none of the blueprint strings it generated worked.

This was consistent with the feeling I’d gotten from image generation tools over the prior years. I’d experimented with Midjourney and the like when they were in vogue, eventually coming up with a simple test: the gnome cleric. Asking for a “gnome cleric” was the hardest version of the test; when the models failed, I’d use more specificity to see if it would get closer eg. “fantasy gnome cleric” or “D&D fantasy gnome cleric character” – generally, this resulted in only modest improvement. Image generation tools from 2022 through 2024 continually failed at this, often creating an image with a garden gnome, or a vaguely fantasy gnome-ish character with a large red hat or other garden gnome attributes. In 2025, finally, I started to get decent results. But in all cases, even after getting an image close to what I was looking for, getting the model to make adjustments or move things in the direction I wanted was a constant struggle.

Code felt similar. Sure, the models could write code much faster than I could, and the quality was continuing to improve, but when the output had errors and I asked it to correct this, all feelings of productivity would vanish as wrestled the model in the right direction while trying to keep it from messing up the good aspects of its earlier work.

In late 2025/early 2026, there was a noticeable jump in quality and capability. Design capability improved drastically, improving the baseline of prompts that had zero design instruction so much that I started to use them for prototyping frontends for application features, easily ten times faster than when I’d done it by hand. Code quality also improved noticeably, to the point that I started to question my own knowledge. As we transitioned to using React hooks over the years, I found myself largely sticking to the core ones like useState and useEffect – others tended to be situational (ref), performance-focused (memo, callback) or unwieldy (context). Seeing a model use them with higher frequency and mostly reasonably use cases was a shock. I still felt it was important to write production code myself to some degree, but as time pressure increased, I started to depend more on model code for the speed gains, though I favored small changes I could read line-by-line.

In the following months, as models continued to improve in smaller steps, I experimented with building a game I’d been imagining for a long time. I decided to try going all in on model coding and focus my efforts on game design and architecture. This went well – perhaps my project was too small to run into the issues other people have encountered when leaving all code to the models, or perhaps the models have gotten better since those attempts, but there haven’t been any problems that required me to dig into the code to solve or get things back on track.

I think it’s worth noting that I did spend time on performance, architecture and refactoring as the project grew – I’ve had a feeling for a while that some of the problems people are running into with AI-heavy development workflows is the assumption that the model can do everything needed without being explicitly told. Maybe this will happen in the future, but for the time being, the human in the loop needs to serve as the proactive component in a range of cases – the model will simply do what it’s told, and if you don’t ask for an architecture evaluation or refactor, you won’t get one.

This relates to the next thing I’ve felt for some time: context management is important, and I don’t expect this to change as models get better. A year ago there was a lot of talk about “context engineering” and about how to give models enough context to do what you wanted without giving them so much you ran up against context limits. Soon after, context windows got larger, models got smarter, and much of this was set aside. And yet, it feels like models will give worse results the more context you use. I think it’s worth keeping in mind that much like humans, LLMs do better when they have to juggle less information.

The Present

LLMs are relatively smarter and much better at writing code today than just six months ago, but I think some principles in working with them are fairly durable:

← All posts