Spec Driven Design and the Devil

I tried spec driven design. I downloaded and installed speckit. I created a brand new project to work with it on. I asked it to create the boilerplate and demo level for a game written in Rust and using the Bevy game engine.

I thought that this would be the thing that sealed the gaps between what I asked AI to build for me and what I actually received. But in thinking that, I had forgotten my own history. But more of that later. So I sat down and went through the stages, answering questions, refining my request and watched documents tumble out the other side. I have to say it was not that much fun - vibe specing is not my idea of fun anyway. After what seemed like an age I was told that the specification and planning was complete... Which was good because I was starting to get bored.

So I decided I would set up 2 agents to work on this. One to write the code defined by speckit and the other to verify and do code reviews etc. I thought this approach would provide the silver bullet I was looking for.

Only I was wrong, of course I was. But I was very wrong and it led to one of the funniest moments I have ever had with AI.

At the end of the day (it was a weekend project) I had confirmation from one AI that all of the required code was written and that all tests were passing (there were a lot of those). So I then let loose the second AI to review the code and validate that all of the planned work was indeed complete. It suggested some adjustments which I manually made and then was told that it was indeed ready to go.

So I ran cargo run and got a window appear.... With absolutely nothing in it. I waited, and waited and waited. Nothing. So I inspected the code and found what I expected for a game. A main loop, a window, some rendering code, some input handling. It even had an opening screen and a demo level just like I asked for. BUT (and this was the funny bit) the graphics were not being rendered to the screen. They were moving around in memory and in fact if you hit the right keys they would move around in memory too. But you could not see them. I found it hilarious.

You see the thing I had assumed was that the AI would know that computer games require the graphics to be rendered to the screen so that the users could see them. It did not. Which brings me back to history. Many years ago towards the start of my career I was a junior business analyst working in financial services. We were building an insurance claims system which meant a boatload of meetings with stakeholders and the external team who were going to build it. After the meetings there would be documents (called feature specifications, non-functional specification etc etc) that would be emailed round for everyone to have their say. There would normally be that many comments that changes would be made to the documents and a new meeting arranged to approve them. This went on for a while until we all agreed what we had was definitive.

Only it wasn't. When they came to build that system there were holes, gaps, errors that meant some claims were never processed. Some assumptions that security would be paramount were never specified. It was only when the system was being built that these things cropped up, while users were in UAT. And you see that is when I learnt that the devil is always in the detail.