A Practical Workflow for AI Coding Assistants
Transcript
Stanford Converge 2025 · · 48 minutes
Transcribed from the recording's corrected caption track and lightly edited for readability: false starts and repeated words removed, product names corrected. Wording is otherwise unchanged.
[00:00] Jon Green: So I’m Jon Green, and I’ll be presenting a practical workflow for AI coding assistants. First of all, I want to give special thanks to Ray Saray and Ali Karim for giving me access to Claude Code, which is what I built this workflow on top of. And also Rafael Cruz who tested out all the early versions of this workflow, and was very patient about it. Also have a little bit of a headache, so I might be sitting down a little bit more in this presentation, but still I’ll be over here.
[00:46] So let’s start. Okay. So I’m sure you’ve all tried copy-pasting from ChatGPT. We’re gonna go quite a bit far beyond that. So, what we’ll be working with is an AI that is aware of the whole code base, has agents, can make changes directly to the code. So you know, there’s a bunch of them. You’ve probably tried, you know, Copilot. You’ve probably tried Gemini or Claude Code, which is what this will be on. Maybe even Kiro or Kiro CLI, that’s from AWS.
[01:44] But I’ll be focusing less on the tool, less on the AI itself and more on the workflow that I’ve developed here. So, why do we want to use these AI assistants? Basically, it allows you to work a lot faster, so, you know, these studies I cited were using Copilot, which is not at the same level as the AI I’m going to show you. It’s not the same level as Claude Code or Gemini CLI or other tools — Copilot can access the code, but not in the same way as an agent that can access the command line and run any command.
[02:48] And I’m also going to show you how to get above 46% trust in the accuracy by, you know, using tests, test-driven development with AI, and creating, you know, test document specs. But before we go into the details, I think a demo will help to show that, you know, there’s no magic here, like, I’m not a magician. I don’t wanna make it look like I’ve got a bag of tricks here. You know, I invite you to, you know, look at the code, but this QR code will take you directly to the code base on code.stanford.
[03:44] So without further ado, I’ll just go into the demo. So there’s a few things I just want to highlight here. So, you know, this is using the latest version of Claude Code. I have some commands I built into this, like this refresh, which refreshes the context. We’ll talk about that later. Basically it helps it remember what it’s doing But, you know, there’s a lot going on here, but let me just copy this first prompt here. So this first prompt is just to create a snake game, and what it’s going to use is this task document.
[04:40] You know, it’s a spec that I created beforehand. In this spec, there’s no explicit code. Sometimes there’s some paths. There might be a few tests in here, files for it to look at. But there’s no code here. A few configurations, and what the AI’s gonna do, it’s gonna read this spec, which I verified and reviewed before, and create a game. So let me paste that in here. Hello, one moment. I rebound all my keys from my keyboard at home, so I’ll just do it the old-fashioned way.
[05:51] There we go. Okay. So you see, just telling it, follow this spec. Also added a instruction for it to time itself, so we can see how long it takes. And a game like Snake isn’t so complicated, but, I don’t know if anybody could do it quite as fast as this AI is about to do it. Is there any, are there any game developers here? No? Okay. Oh, there’s one game developer. So. Right now it’s creating the tests, I’ll just show you here.
[06:47] Yeah, so it’s reading the code. It just created some tests. So you see here, it wrote all this and it’s going to use it to verify that the game is complete before we even run it. It’s created some more tests. It’s running the tests, to make sure they’re all red. They failed, which was expected. So, this is already quite a bit of work. This is 80 lines of code here. Let’s see, oops. There’s 162 lines here. So, within two minutes, it’s already created a substantial amount of code.
[07:46] And let’s see. Okay, all the tests are passing. I think we’ve got a game. So, yeah, I’ve been so this game until now didn’t exist, this Snake. It took two minutes. Well, let’s just go ahead and run it, and there we go. In two minutes we created Snake, and has collisions, works pretty fast. There’s self-collision, yeah, you can add self-collision. So this is a complete Snake game. Yes?
[08:52] Audience: Yeah, just curious did you start off by like creating a spec, a spec-driven
[08:58] Jon Green: Yeah. I started out by, yeah. Oh, sorry. So the question was did I start out by creating a spec, whether this is a spec-driven development. And yeah, that’s how I started. So I would I’ll go into more detail about that later, but basically I have a bunch of these for a bunch of games and we’re gonna build a few more. But you know, you create the spec and then it creates the code. So that was pretty fast, two minutes, one game.
[09:41] How long do you think it’d take to write, to create all these games? Have Frogger, Flappy Bird, Minesweeper, Pac-Man, Mario, a basic version. How long do you think that might take for a human developer? Probably a few hours, maybe days. Well, let me run this here. Set down the mic. Okay. So I’ll just put that there. So, I’ve written all these specs ahead of time. It’s going to start 14 agents running in parallel, and we’ll see how long it takes to do this.
[10:57] We can see it starting up, but we’re gonna go back to the presentation. Okay, so I’m not a magician and these so, I’m not going to hide any tricks from you. This workflow has basically, you know, other than the agent, has four parts. It takes notes, we have the tasks, we have guides, and we have tests. And these work together to, you know, create the software mostly automatically by itself, although there’s a lot of iteration that happens, and we’ll talk about that.
[11:48] So, the notes so, this is an instruction I’ve told the AI to do. Basically, I’m just telling it, you know, “Take notes while you’re going. Just take notes.” You know, it’s like a log, an audit log. It keeps track of the errors it runs into. It keeps track of the solutions it comes up with, and this is really important for context. So, you know, the AI will eventually run out of space in its context window, and we need to be able to refresh that reliably.
[12:34] The compressions, you know, there’s different ways to compress it. Don’t work, you know, completely well. It always forgets something important. So, that’s the main reason, but there’s also a lot of side benefits you get from notes. Oh, yeah? Yeah. Yeah. Uh, this is on Claude at Stanford. Uh, I’ll show the, the QR code later in the presentation. So let’s go here. So, I have it use these kind of triggers, habits. Basically, I give it instructions. You know, “This is when I want you to, you know, take some notes.”
[13:18] So, when it’s gonna hand off to me, I have it take notes. When it makes a commit, I have it log the commit ID so we can look it up and get history later. Errors, whenever there’s an error and just throughout the work. Okay, and the tasks is the other part of this so, these are the specs, basically. They’re to-do tasks. Started out as a little to-do list I had the AI maintain for me, and then it became you know, I add more and more detail until it became, you know, documents of their own.
[14:04] And this workflow, you know, kind of just evolved organically as I developed using these, using the system. So, you know, in the code base I’m sharing with you, it’s they’re all kept inside this to-do folder, that keeps a master list of all tasks in the to-do index. There’s a specific numbering scheme for tasks. It even keeps track of priority and the status of the tasks. I’ll show that in the code later. And, you know, it’s an iterative process, creating this, these tasks with the AI.
[14:53] So like, start out You know, the task, it what you want it to do, everything it needs to know to create this task. You start out just telling it what you want. It has instructions built into the CLAUDE.md file that allow it to understand how to create tasks, and you know, like, what to do with them, and then you review. So once it creates the initial version, you go through a review, and it’s just iterative.
[15:36] Like, look at the document. If there’s something there that you don’t want, tell it to remove it. If there’s something there it needs, you add it until, you know, you’re happy with it, and then after that is the build process where it’s also iterative. You know, with these games, you know, I’ll show you a few, it creates a lot of them in one shot pretty well. Some of them, like Mario and Pac-Man will have a little, some bugs, but you know, that’s the nature of these, you know, non-deterministic systems.
[16:12] It’s gonna create a different game every time. So I’ve run this, you know, in testing, like, half a dozen times, and each time the game is a little different. So anyways, you know, so you work with AI until the implementation is what you want, and then, you know, deploy, test, document, and eventually archive the task, which has instructions on how to do that explicitly. Guides. So, these are like documents teaching it how to do certain things. So, this could be how to debug webpage, how to patch something.
[17:03] Like, for instance, one of the websites I work on, patching is a really tedious process. I’ve created a guide, and the AI reads the guide, and then it does the patching for me. You know, there’s a few, it might not get all the patches done, but it does 90% by itself, and I’ll be demonstrating that. As well, later on. One of the issues I found with guides is security. So, I create a custom script that checks to see if there’s, you know, secrets in the guide, you know, passwords or other things.
[17:51] ’Cause it really likes to put everything into this guide. So you have to be a little diligent. You have to check, make sure it’s not doing anything, checking in stuff that shouldn’t be there. And, you know, having an automated script go through helps it do that before coming to you. So you can say, you know, run the verification script. It will go and then put placeholders itself before you even, you know, need to review it. And then testing is another part of this system.
[18:32] The testing is kind of a guardrail. So, it’s a guardrail, it also helps it during the development process. So, having the AI create these tests helps it, you know, first of all helps you be more confident, which goes back to that trust part, getting you above the 46% trust. But it also, I found, has side benefits in terms of keeping the AI focused while it’s doing the work. So you can tell it, you know, make a test for, you know, making sure the items are selectable, making sure, you know, this or that work.
[19:19] And it will put that in the code, and then it will automatically run the tests and find the issues before even coming to you. So it helps make the iteration a lot faster. The AI iterates with itself for a while, and then you can come back and do the real testing and see, you know, if it’s up to snuff. And you can even ask it to make more tests, which is also helpful to prevent regressions. ’Cause often the AI will fix one thing and then break another.
[19:55] Or it’ll fix one thing, and then you’ll add another feature, but it will put back the same broken code. And it doesn’t make any sense, I find, you know, if you’d make it, if you have it write a test. You don’t get that regression. So, let’s see how it did over here. Looks like it ran into a bug. (laughing) So this didn’t happen during testing but let’s see if it can recover. But while it’s doing that I’ll just show you, it created all this, like, all these tests just while we were talking.
[20:53] Created, you know, most of the code for these games, they might even work. Okay, context is low. /compact. Okay, so this, let’s try it. So this is what I was talking about, the compact, looks like it, the conversation’s too long. So, this is what I was talking about in terms of context. Is it not sharing my screen? Look at this. Sorry, I don’t know why it’s not showing my screen. Okay. Okay, so now you can see what I’m talking about. So, this is what I was talking about in terms of context.
[22:03] So, basically what happened is it ran out of space in the conversation, and it looks like Claude Code can’t, there’s an issue, an error with Claude Code itself, so I’m just going to exit and restart this. Okay. Let’s resume. Okay. Let’s see here. Sorry about this. Continue. Okay, now it looks like it’s gonna work. Or maybe not. Okay, I’m just gonna create a new conversation. Okay. Okay. So this is a good way to demonstrate the refresh function. So while we were talking, it was building all this code.
[23:14] It was also taking some notes, or should have been taking some notes. Yeah, so, you know, there’s some bugs. Of course, they always happen live when presenting. (laughing) But see let’s see if we have Flappy. Okay, so it did create the games. So you can see, this is Flappy Bird. Let’s see here. Let me just give it a quick instruction. Okay, so it’s going to look at the code and pick up where it left off. But pretty sure all these games are gonna work.
[24:33] Let’s try running the launcher. Okay. So here’s the launcher, here’s the games. Let’s see. Does anyone have a favorite game they want to see? Galaga? Pac-Man? Okay. Let’s do Frogger first. Okay. So not 100%, but of course, it stopped halfway, so I’ll just type it in. Frogger. Okay, so here’s Frogger. The cars, the logs, and you fall in the water, you die. Okay. So then, let’s check out, Galaga. So, here we This is pretty good. Just from a prompt. We have lives, we have waves.
[25:54] Geez. They’re not shooting, but, you know, this is pretty good. Pac-Man sometimes has trouble, but let’s see. Yeah, this is this is okay. There’s some dead ends in the maze and doesn’t fit exactly there. But yeah, if you hit a ghost, you die. So this is pretty good.
[26:27] Audience: [inaudible — a question about sound]
[26:29] Jon Green: No. It doesn’t do the song. I could didn’t want to add that, you know, too much noise during the presentation. But you’re free to, you know, copy the code and add that feature. You can use the task system that I’m going to demonstrate next. So, that’s pretty okay, so it ran the tests. All 773 tests passed, so you know, the tests, obviously, you can see how you have to iterate. You know, the Pac-Man game had dead ends, it was probably just that one level, and it didn’t have any of those power-up Those power-up balls.
[27:26] So that would be part of the iteration, but, and you can see in just a few minutes, you can create something, and then you can iterate with it and you can create tests. I guess the, the next thing I’ll show is how to use the guides. So we’ll just introduce you to this guide here. This is kind of a spec, basically, it’s just telling it how to do a process. And you can see here, it has this table of contents, which helps it find the things.
[28:07] It’s pretty interesting to watch it go through this. But you can see it has quick references. It has the list of things to do. You know, this is pretty complete. So this is also an iterative process to create. And we’re just going to, we’re gonna create a game here together. So let me paste in this prompt. So, right now, it’s reading it. I gave it instructions to read the guide and then, you know, run it. Okay. So, it’s asking what kind of game.
[29:01] I don’t know. Does anyone have a game, like a burning, like, a game they’ve always wanted, a 2D arcade-style game? Yeah. Sudoku? Okay. Let’s make a Sudoku. Yeah. Oh, maybe, does anyone have a, like, an interactive CLI game? No? Okay. Sudoku, Sudoku. And you don’t even have to spell these things right. So, that’s one thing I really enjoy, just type in, the faster you type
[29:47] Audience: Better. Okay, so now it’s asking what we want to put in the v1 of this Sudoku game. So, let’s see. One difficulty level, pre-generated puzzles. That sounds pretty good. Let’s just go with that. Let’s copy paste. Paste. Okay. So now it’s creating the task for us. So this is the idea, the spec-driven development you know, with the AI. So first we’re gonna create the task itself, then we’re gonna have it run the task to build the software. Yeah. So you can see here, it’s getting the number for the next one.
[31:10] So it’s 17, and it just wrote the whole thing. Now we can, you know, just go through and review it. I’m sure it’s probably 100%. Let’s see. And you can see how it has a testing strategy. This looks pretty good. Technical notes. Yeah, this looks, looks pretty good for a Sudoku. So it’s gonna build this in three phases. Yeah, let’s go ahead and build it.
[32:03] Jon Green: Okay. Yeah?
[32:06] Audience: So where did it come up with the list of requirements for this game? Because you wrote the guide, and the guide has enough stuff to tell it that it has to then have these sections in the list of tasks.
[32:22] Jon Green: No. So, the way it did, oh, okay. Okay. So, the question was basically, how did it know to write all the tasks for the spec? Because, you know, it’s because it, it knows how to play Sudoku already. So this AI already knows how to play Sudoku. We could create a more imaginative different game here. But that’s how it wrote the spec. So, I’ve created, you know, different games. The guide I wrote here will go through the whole process. I have it ask for specific things.
[33:19] So, it wasn’t, if it had more features, you know, it was different than Sudoku, would have asked you, you know, what the core gameplay of it is. As you can see here, it has even an example of a Zelda game built into the spec. So it, you know, it would ask you things like these, you know, depending on the game. And then, it lists out the different features it believes that are going to be needed, like either from your description of what you want or from its inference.
[34:04] You know, of course, it inferred basically all the Sudoku game. But if you wanted to build a more complex game, you could. So right now it’s working on the red phase of the tests. It’s working on the game. So, probably be done in a minute here. Okay. All the tests are passing. Okay. Let’s just type in Sudoku here. Sudoku. Okay, all tests pass, the game is there. Let’s play. So, there we have it. Sudoku. So, let’s just try putting a number down here.
[35:19] I think one might go here. Okay, cool. So, oh. Okay. So, let’s, let’s go look at you know, what, what based on, you know, this workflow I’ve created what I think is going to come down the line. So today, we have this manual workflow, which uses files checked into Git, you know, files that are version controlled. Down the line, we’re going to have, obviously, integrated tooling. We’re going to have different things built into the IDE, built into different agents that the AI can use.
[36:30] Further down the line, we might have repositories of, you know, context. So you can pick up and develop code on features that you were working on, you know, months before. You might have well, definitely have ways for non-technical users to create specs and tasks, and feed into the system. Of course, we’ll always need someone to review those with the technical knowledge, but these are just my ideas of what could be down the line. The developer role is moving towards someone who, you know, does a lot more reviewing and testing, and focuses a lot more on the system design and architecture than, you know, the nuts and bolts of the code.
[37:28] You know, I’ve, I’ve had projects where I’ve had to change the programming language because the library I was using in the, the one language wasn’t powerful enough. I just told it to change, you know. So, you know, and then, of course, all I have to do is review the code. But it’s a lot easier to review code in a different programming language than to, you know, just learn a new language. So here’s the QR code again. So, I invite you to scan the code, download the code, and you can run these if you have a Claude Code.
[38:17] You can run it just the way I was now during the presentation, and build all these games yourself. You can create a Sudoku or you can create a more imaginative or you can create a different game using this task guide here. And, yeah, that’s my presentation. So, here’s a link to my LinkedIn. You can add me and we can connect. Does anyone have any questions? Go ahead.
[39:08] Audience: Somebody was asking about the specs, right? How much time you spend on it and how do you approach
[39:16] Jon Green: Yeah. So basically, a lot of my work is now creating the specs, you know, I have to read and make sure that they have all the things that are required. So you know, it’s a lot more reading and reviewing, but it’s a lot- Faster. You can find like, you write the spec and you can find the bugs, you know, before you write the code, you can look at the architecture, can make sure it doesn’t create something, you know, wonky. You think at a higher level, you know, it’s not so much nuts and bolts of the code anymore.
[40:02] It’s more like, what do I want to create? How’s a good design for that? So it’s a lot more of the work, but it pays off in the end you know, in terms of the speed of the development. With these AI agents, you can pivot a lot quicker. You know, if you write, like my example, I wrote the code in a language that didn’t have a library that was as powerful as I needed for the features that were coming. I just changed the language.
[40:39] I wrote a spec, I told it, “You know, we need to change the language to this one using this library.” And it went through, looked at the code in the other language, translated it and put all the requirements that I asked it to put in the spec into the code. So, yeah. Yeah. So people are, you know, I oh, sorry. So the question was “Is my interaction with the code like a black box?” And I still, you know, read the code.
[41:30] I still make sure it’s good. I still write tests or check the tests, but it’s less about writing code, for sure, you know. It’s more like, “Okay, this code looks good,” you know. “This code has all the things I asked for it, and it’s passing all the tests.” So, yeah.
[42:00] Moderator: Can we jump in with an online question really quickly and then we’ll get back to you?
[42:03] Jon Green: Yeah.
[42:05] Moderator: All right. So from Reynaldo “Jon, I can see this being used when the AI knows the algorithms, such as common games, but how about custom apps that is unique, for example, to your business requirements? How will you avoid coding and rely on AI in that case?”
[42:23] Jon Green: Yeah. So that’s a great question. So that’s where the iteration in the spec becomes very important. You know, have to, you put all those business requirements into the spec. You know, you work with it as it develops. So you develop, you’re developing the spec, you’re developing the code. It’s collaborative. If it’s making, you know, a webpage that doesn’t, isn’t the way you want it, you tell it to make it a different way. If it’s trying to use the wrong JSON structure, of course, you have to step in there and tell it to correct the structure.
[43:04] So it’s not hands off. It’s still hands on, it’s just different level. You know, you’re looking, you know, making sure that it’s building the thing that we want, that the user wants less than making sure every line of code is, you know, exactly perfect.
[43:34] Audience: Thank you for the presentation, this was great. One of the questions that I have is regarding the app development and next steps. So once you develop the app locally, how do you verify that this is ready to release to the App Store, and have you experimented with the process of submitting it to different stores and is there any way to make it platform agnostic? So if I develop something that I could release to the Google or the Apple Store?
[44:04] Jon Green: Yeah. So like a mobile app?
[44:07] Audience: Right.
[44:08] Jon Green: Yeah. Making an app or a website or anything, you have to test it yourself. So, of course, then, you know, you’re not going to go and just have it generate a bunch of code and upload it. You know, you have to test it, like I have it create, you know, a test suite so that, you know, for iteration and verification purposes. You know, have to manually test it before uploading it into the, to any store. Did that answer your question? No, you upload to the store the normal way you would and, you know, you create the build file and then you upload it to the store.
[45:08] Of course, they could create the build file for you. And then of course you would test it and upload it to the store.
[45:33] Audience: I’m curious about your experience with dealing with, like, structured data, like a SQL database or any other kind of databases that you have to incorporate into an application. Do you write those specs in the spec in how it should interact with the database? Or do you have to use something like a MCP to let it know what tools to use? How would you go about it?
[45:57] Jon Green: Claude Code knows SQL very well. So what I would do is, you know, create the schema in the spec. I would say, you know, we need, you know, these fields or, you know, you could even go to a higher level and say, you know, “I’m creating a database needs to hold this data,” and then have it create a schema, and then just check to make sure that schema makes sense. So you can either be very specific and, you know, “This is exactly what I want,” or you can be more high-level and say, you know, “I want something like this.
[46:37] What would that look like?” And, you know, it depends on what you’re building. If it’s a system that’s in place, of course you need it to be specific. But if you’re creating something new, maybe you see what the AI comes up with first, and then refine it or, you know, correct it or, you know, yeah. Good.
[47:01] Audience: Like, Claude Code, would is there a better agent that works better with?
[47:08] Jon Green: I’ve only used Claude Code for databases. But all the LLMs, they’re trained on, you know, vast amounts of data. I’d be surprised if any of them didn’t know how to write SQL code. So this is, you know, the system that I have access to. Uh, I’ve tried a few others. Claude Code is the best that I’ve, I’ve tested so far, but that doesn’t mean it’s gonna be the best tomorrow. You know? You know, there’s Gemini, that’s also very good and has a free version.
[47:51] Um, there’s Kiro that Amazon was just doing a demo of, and, you know, I’ll have to revisit that one. It looks like they’ve added some features to that as well. So you just have to keep on trying different ones. You know. And see what one is good in the moment, ’cause this is just changing so fast. So all Right. I think we’ve hit our end of the time for the session. Uh, can we thank Jon one more time? (applause) If there’s any more questions, is there any way we can reach you?
[48:30] Yeah. You can reach me via email or LinkedIn here. You can add me on LinkedIn, using the QR code. So, yeah. Awesome. Thank you so much. Yeah.