If you can play Oregon Trail, you can code apps. Here’s a step-by-step guide for how artists, designers, devs, writers, and more can create software (apps, websites, tools, chrome extensions) using AI.

• Start with an idea…
• Create a plan…
• Execute in a smart way to build upon a series of micro-milestones..
• Expand, evolve, iterate and grow
• Do it all with AI.

Step 1: Start with an idea…

Record your thoughts… Talk it out. Start by talking about what you wish existed in the world. Use Apple notes, Granola, Zoom, or anything really. Just record your rambles.

Easy Trick → If you’re on a mac, open a new doc and hit F5 and it activates your mic and you can just talk and your words will appear as text in the doc. No other software needed.

What do you talk about? You don’t have to know how to actually solve any of the problems or if your idea is even possible. Pretend like you’re talking to a conversationally-permissive friend who lets you go on and on until you get to the core of your idea. Consider asking AI to rewrite and organize your ramble, or just copy and paste it. New to this? Try Granola, or even just open up ChatGPT and click on the little microphone button. Or my favorite… open up the ChatGPT mobile app, put on your headphones, and have a conversation. The idea here is to talk through what you want.

Step 2: 4 Minutes of Set-up.

Once you have a text transcript of what you are trying to do, we need to do four minutes of set-up. Go to Factory and sign up for a free account.

Don’t actually use factory on the web just yet. Just sign up. We’re going to use factory via your terminal.

Step 3: Open Terminal or Get VS Code

Open terminal on your mac, or if you do a little coding already, open VS Code or your favorite code editor. VS Code is free. Get it here https://code.visualstudio.com/download and then open a new terminal.

Step 4: Load in Factory

Type this into your terminal:

# Install droid
curl -fsSL https://app.factory.ai/cli | sh

# Navigate to your project
cd /path/to/your/project

# Start interactive session
droid

BOOOOOM !!
That’s it! Just type the curl command and then droid!! and the whole thing launches. In the future, all you’ll need to do is type droid.

From here you literally type in commands into the text box like you were playing “Oregon Trail” or some other text-adventure game.

To see some options, hit the questions mark

# hit the question mark to see options
?

The most important ones are to change the model (as of Sept 20, 2025, I recommend switching to gpt-5, but feel free to do whatever you want).

To switch the model, first type /

# to switch the model, type slash
/

And then use the arrow keys to navigate up and down to get to the model and then hit return, and then choose the model you want.

Now we’re ready to code.

ALMOST

The secret to good coding is good planning, so first we need to make a plan. But that’s hard, so to do that well, we’ll ask the droid.

We first need to shift to the Spec mode to put everything into planning mode. This sounds hard, but it’s easy once you’ve done it once or twice. So just trust me for the next 15 seconds and we’ll get you there…

Hit shift-tab and watch the “mode” cycle through from coding to “spec”

Think of it as a “keyboard short cut” to switching menu items. So hit shift and tab at the same time twice and BOOOOM we’ve cycled to the magical “spec” mode of droid. You can tell if you did it correctly if you see Spec above the command line.

Step 5 Make a Plan.

• Ask the Droid in Spec Mode to make a detailed, step-by-step plan using your transcript from your brainstorming session.
• Ask the Droid to clarify anything that is contradictory or unclear.
• Ask it for suggestions to improve on your idea.
• Ask it to write a README and NEXTSTEPS docs that you will give to the Coding Droid in a minute to follow.
• Tell it to follow the engineering best practices for AI outlined by Buckhouse in his “Coding Backwards” article.

Here’s a sample prompt for the Droid in Spec Mode.

Prompt for Droid in SPec Mode

Please take this transcript and generate a PRD, a PROJECT_PLAN, a NEXTSTEPS doc, a DESIGN_GUIDELINES doc, and a README. Put all of these docs into a folder called ./Docs Follow the engineering for AI best practices outlined by Buckhouse in his article "Coding Backwards" found here: https://buckhouse.medium.com/coding-backwards-f498805e96c1 the main point of the articles is to write the readme before you begin so that you can envision what success looks like, and then make a plan that follows a progressive build-and-test approach, so that you always have working code and make many, frequent, smart iterations to build upon code that works. Start small with something works. Choose each next steps to build this function, feature or test that will give you the most helpful information you need to be able to make progress towards the next milestone and is also easiest to debug. At every step, include robust logging for debugging. Design your logging so that the Coding Droid will get the information it needs to solve problems. At each step include a to-do to update all of the docs, especially the PROJECT_PLAN and NEXTSTEPS docs. Please note in the docs that I will occasionally go in an update the DESIGN_GUIDELINES to reflect design preferences. Please review the DESIGN_GUIDELINES when revising plans or next steps. Ask me for preferences for infrastructure, but for now, use Supabase and Vercel and GoogleAuth for all web apps. The documents you write will be read by Factory's Coding Droid, so write them so that the Coding Droid can best code the app. Think hard and write cogent plans. This product will be built by Factory.ai's Coding Droid, not by people, so make your plans in such a way that reflect how the Coding Droid best likes to work. Ask me any questions to get started. Attached is the transcript of my product brainstorm.

Step 6: Let’s Code

The Droid in Spec Mode will kick out a plan and write the docs to ./Docs. Now that we have a plan, Let’s code!

Hit shift & tab again to get into coding mode. Here I’ve selected “Medium Trust” so that it runs all the basic commands without asking me questions, but does ask before doing risky stuff.

Now we need to tell it to go do the work.

Here’s the prompt box, click on that little toolbox to get access to additional tools.

Prompt for Code Droid

Take the plan created while in Spec mode and start to code. In this repo you'll find a ./Docs folder that contains all my project planning documents written while in Spec mode. Read through all the docs and then get started coding the first step in the project plan. This project has many, frequent "micro-milestones" that we'll hit along the way. Ask me any questions you have so that you can get coding.

Along the way you can feed it screen shots from Figma, update the design docs describing what you like and don’t like, and give it aesthetic directions, where you play the role of “hovering art director” or encouraging coach.

Factory will help you connect to Vercel, Supabase Google Auth, or anything else you need. Get stuck? Ask for help. Don’t know how to get API keys or save environment variables? Just ask and ask how to do it safely.

Step 7: Security Reviews

After each micro-milestone, ask Factory for a security review to make sure you haven’t done anything dangerous like check in your passwords, private keys (secret codes used to access APIs) or accidentally opened yourself up to malicious actors.

Prompt for Security Check:

We've just We've just finished a micro-milestone in our project plan. 
Please do a quick security review to make sure we haven’t done anything 
dangerous like check in my passwords, private keys, or accidentally 
opened up the app to malicious input. If you're not sure if something 
is safe, use your web browsing tools to double check.

Step 8: Iterate

Now that you have the basic flow, keep iterating and expanding to get what you want. As you start to develop your own style, you can do things like create a stock pile of UI components that you like and instruct the Droid to use them as the basis for creating new UI components. You can also periodically ask it refactor, clean-up, to streamline your code. An important step to remember is to always ask the Droid to update the docs after they finish a task, otherwise, you’ll spend a lot of time “catching the AI up to where you are on the project” or correcting bad assumptions.

Problem Solving Guide:

1. Problem: Factory gets sluggish. Solution: start a new session. Command-C to quit. Then type droid again.
2. Problem: Factory keeps making the same mistake. Solution: Ask it to show you what files it’s stored in it’s memory. It’s likely holding on to an old idea it thinks is true. Or there’s something out of date in your ./Docs.
3. Problem: I don’t understand why it’s doing something. Or even what it’s doing. Solution: Ask the Droid to explain it to you. It’s honestly my favorite part of building with Factory, learning from the Droids how to do something interesting.

It’s a really fun way to build. Here’s an example of a finished project: the SDL Learning Library.

SDL Learning Library

SDL Learning Library—hundreds of tutorials, like Netflix for Builders…

It was all written in Factory. Supabase backend, CRUD interface for adding new videos, Vercel for hosting the code. GoogleAuth for the admin interface. Resend for auto-generated emails. Factory walked me through all of this.

Curious for more?

Subscribe to my newsletter for more on hands-on help and interesting stories as I explore the architecture of innovation and the nature of creativity.

https://jamesbuckhouse.substack.com/

❤

In 1982, at the heart of the Tron universe was the MCP, or Master Control Program. It was an old-fashioned mental-model of a computing system.

It represented an “all roads lead to Rome” mindset where all code stemmed from a single core. This was the era of linear coding: programs that ran line-by-line. No weird async insanity. No ghost-like classes and instantiations. No polymorphism. No “listeners” waiting to spring into action. No non-deterministic chaos. Certainly no cosine similarity traversing through n-dimensional latent space. The mental model of the time was that a program was a recipe or a robot that did what you told it to do (and only what you told it to do). It was a clean mindset. Deterministic. Logical. If there was a bug, it was findable (eventually) somewhere between your fingers and the metal. And even back then this was more or less Hollywood reductionism rather than how people actually coded. Coders have always found artistic, poetic, elegant ways to emulate nature’s raw glory with ingenious complications and redirections and mis-appropriations all in the service of finding a way.

Today, MCP now stands for something totally different: Model Context Protocol (MCP) and it’s a way for AI Agents to talk to other tools. The P in MCP stands for protocol. And just like there’s no actual magical power infused in the letters of the alphabet, there is also nothing special about a protocol, and yet, its mere existence means quite interesting actions can now happen. With a codified alphabet we can communicate any human thought that can be put into words, and with a codified MCP, AI Agents can now start to use the Internet, tools, and anything else we can do with our keyboards and cursor.

MCP = a new API for Agents.

MCP can help your AI Agents accomplish complex and branching workflows: so that your Agent can use tools, chain them together, and ultimately figure out what to do with the results you get back, and when to ask for an opinion or permission from a human in the loop. So unlike Tron, were all roads pointed to the central core, the agentic version of MCP is more like Ned, the “guy in the chair” in Spider-man movies… where a helpful friend is plugging into whatever systems it needs to get the information you need to get the job done.

Agents using MCP = Your Guy in the Chair

Where did this all come from?

The mindset that brought us to the agentic MCP has its origins in how devs like to code: drop into your coding environment (VSCode, Cursor, Warp, etc) and just stay there…lock in… headphones on… no need to jump out to your database or open up some other tool. Stay locked into the flow of code and send runners out to interact with the other tools and systems. Don’t break your flow state by opening up some other tool or program and have to transition to some other context… eliminated all “context-switching costs” and stay locked in and just fetch what you need.

History of MCP

The Model Context Protocol (MCP) was “developed by Anthropic to create a standard for how language models interact with external tools and services.” (so says Claude). MCP has since spread to OpenAI and Google’s Gemini.

TL;DR

• The Model Context Protocol (MCP) allows AI models to interact with external tools, fetch data, and interact with services.
• MCP is designed to support autonomous AI workflows, allowing AI agents to decide which tools to use, in what order, and how to chain them together to accomplish tasks.
• MCP can also include a process for a human-in-the-loop moment for humans to provide additional data and approve execution.
• AI Agents Using MCP = Your Guy in the Chair
• MCP = API for Agents to use tools.

AI MCP Use Cases:
Examples & Why They Matter

I asked Claude and OpenAI to collaborate on a list of resources. Here are the results.

🎨 Creative Tools

Figma MCP Server

• Link: GitHub Repository
• Why this matters: Enables AI agents to interact directly with Figma files, automating UI/UX design workflows and bridging the gap between design and development.

AI-Controlled Ableton Live Project

• Link: YouTube Demo
• Why this matters: Demonstrates real-time AI interaction in music workflows, paving the way for novel AI-assisted creative processes and compositions.

Blender MCP Integration

• Link: Blender MCP
• Why this matters: Allows intuitive, text-driven creation and modification of 3D models, drastically simplifying traditional 3D modeling processes.

🧠 AI-Orchestrated Multi-Tool Workflows

AI Control Center Concept

• Link: Cline.bot Article
• Why this matters: Highlights AI’s potential to orchestrate complex, multi-application tasks, significantly enhancing productivity and creative possibilities.

MCP-First Development Approach

• Link: Medium Article
• Why this matters: Promotes AI-ready app development, enabling seamless AI-agent integration across multiple tools from inception.

🧰 AI-Enhanced Developer Tools

Cursor IDE with MCP Support

• Link: Dev.to Article
• Why this matters: Embeds AI directly into coding environments, providing immediate access to tools like GitHub and databases through natural language.

Spring AI MCP SDK

• Link: GitHub Repository
• Why this matters: Simplifies the integration of AI functionality into Java applications, enabling standardized development of powerful AI-enhanced applications.

🏥 Healthcare Applications

MediAssist MCP Framework

• Link: Research Paper
• Why this matters: Enables AI assistants to securely access and analyze patient records, medical imaging, and lab results, aiding physicians in making informed decisions.

OpenMedicalAI Research Project

• Link: GitHub Repository
• Why this matters: Open-source initiative allowing medical researchers to build specialized diagnostic tools that coordinate between multiple hospital systems through a standardized MCP interface.

💰 Financial Services

FinMCP Open Banking Framework

• Link: Infoupdate.org Article
• Why this matters: Creates secure connections between AI financial advisors and multiple financial data sources, enabling comprehensive financial analysis through natural language.

AlgoTrader MCP Integration

• Link: Wyden.io Article
• Why this matters: Allows quantitative analysts to design, test, and deploy trading algorithms through conversational interfaces coordinating between market data feeds and execution platforms.

🔬 Scientific Research

OpenLab MCP Server

• Link: Agilent OpenLab Server
• Why this matters: Connects AI assistants directly to lab equipment and research databases, allowing researchers to design experiments and analyze results through natural language commands.

Research Data Orchestration Platform

• Link: Semantic Scholar Paper
• Why this matters: Enables interdisciplinary research teams to unify disparate datasets and analytical tools through a common MCP interface, accelerating discovery in complex scientific domains.

📚 Education

EduMCP Framework

• Link: GitHub Repository
• Why this matters: Connects AI tutors to learning management systems and assessment tools, creating personalized learning experiences that adapt to individual student needs.

Language Learning MCP Demo

• Link: YouTube Tutorial
• Why this matters: Shows how AI language tutors can coordinate between pronunciation analysis tools and grammar checkers to create immersive language learning experiences.

🛒 E-commerce and Retail

ShopOps MCP Suite

• Link: RepairDOC Overview
• Why this matters: Allows AI assistants to manage online stores by coordinating between inventory systems, customer service platforms, and analytics tools, streamlining e-commerce operations.

Smart Retail Analytics Platform

• Link: Dreamstime Illustration
• Why this matters: Connects in-store sensors, customer data, and inventory management systems through a unified MCP interface, enabling AI to provide real-time business intelligence for physical retail environments.

📘 Top Tutorials for Getting Started with MCP

1. Building an AI Agent with MCP and LangChain Adapters

• Author: Seenivasa Ramadurai
• Platform: DEV Community
• Link: Read Tutorial
• Overview: Guides you through developing a multi-functional AI agent using Anthropic’s MCP with LangChain MCP adapters.

2. Model Context Protocol (MCP): A Guide With Demo Project

• Author: Aashi Dutt
• Platform: DataCamp
• Link: Read Tutorial

Step-by-step

Recently I learned React and used it to completely rebuild my personal site. You can see the results here, if you’re curious about what a designer/artist/writer does when learning react: jamesbuckhouse.com

Learning React is, well, hard… so I developed a new method of self-guided learning for code: Coding Backwards. It’s a great way to learn about learning as you teach yourself a language by actively coding in it. It uses AI to help you take action, but also educational theory to ensure you learn as you do it. As a part of that project, I ended up coding a nice little nav bar component and thought I’d open source it.

I’ve added all the code to GitHub.

GitHub - Buckhouse/React-Nav-Bar-Tutorial: A tutorial for adding a nav bar component to your site using react

Here is a step-by-step that walks you through how to recreate it.

Step-by-Step

Follow these steps to get a responsive React navigation bar running on your machine using React, TypeScript, and Vite.

1. Clone the Repository

First, clone the provided repository to your local machine. A repo is what Github calls a collection of files. If you are totally new to Github, ask AI to explain it to you. In short, it’s a way to save versions of your code as you work on it. Later, when you work in teams, you can also use it to mix everyone’s code together. If that sounds like it might cause trouble, you’re right, but Github exists to help you wade through the trouble with as little trouble as possible. Cloning is just what Github calls making an official copy.

In your terminal…

git clone https://github.com/Buckhouse/React-Nav-Bar-Tutorial
cd react-navbar-tutorial

2. Set Up Your Development Environment

Check to see if you have Node.js installed on your computer. You can verify the installation by running:

node -v
npm -v

If Node.js isn’t installed, download it from nodejs.org.
Install all required dependencies by running:

npm install react react-dom react-router-dom
npm install -D vite @vitejs/plugin-react typescript @types/react @types/react-dom @types/react-router-dom

3. Project Structure

Now you need to make sure you have the following files in place. You should have gotten these when you cloned the repo. But just in case… verify your project structure matches this exactly:

react-navbar-tutorial/
├── index.html
├── src/
│ ├── main.tsx
│ ├── App.tsx
│ ├── components/Navbar.tsx
│ └── styles/Navbar.css
├── vite.config.ts
├── package.json
└── package-lock.json

4. Customize the Files

Open the project in your favorite code editor. VSCode is standard, so if you don’t have an opinion, try it. If you want to go more new generation… try Cursor or Warp.

The main files in the component are the navbar.tsx and Navbar.css. You can think of the tsx as the content + functionality and the css as the expression of that content in a specific style.

Navbar.tsx:
Customize the menu links by editing the <Link> components. Change the names and the destination.

Navbar.css:
Adjust the CSS variables at the top to change colors, spacing, or fonts. I’m using root variables in the css, so you set it at the top

:root {
  --navbar-height: 64px;
  --text-color: #333;
  --hover-color: #666;
  --primary-color: #000;
  --primary-hover-color: #444;
  --bg-white: #fff;
  --spacing-xl: 24px;
  --container-width: 1200px;
  --transition-fast: 0.2s ease;
  --menu-font-family: 'Inter', sans-serif;
}

And then you can use these throughout your css. This means you can make changes in a single place, and it will get sent through the rest of your css. Example: you might have a bunch of parts that use the var — text-color and so to change the color of your text, you just change it in the root section. Pretty cool!

Notice the — transition-fast: 0.2s ease variable. Make this 0.5 for a slow and gentle animation. Make it 0.1 for a faster move.

App.tsx:
This is the main content and action of the component. You can change the number of menu items, their names, and their destinations. Here, I’ve assigned a page color for each of the destinations, so you can tell it’s working, even though it’s a nav bar for a blank site.

5. Test Your Navbar Locally

Start your development server:

npm run dev

Visit http://localhost:5173/ in your browser to test your responsive navigation bar.

6. Suggested Next Steps

Integrate the navbar into a larger existing React project by adding more components to your project, or bringing the navbar to one you already have. You might want to move the root variables into a styles/Global.css file that all of your components can use.

Hope this was fun. LMK if you add it or a version to your site.

Internet juggernaut Amazon has an internal practice called “working backwards” where they write a press release before they start a project.

It takes effort. It requires difficult critical thinking, and yet it saves a huge amount of time by correcting missteps before they happen and ensures that what you get in the end is something you actually want.

What if we took this idea and applied it to coding?

As AI-Assisted coding becomes the norm for all types of software developers, how can we get the best results from our AI helpers and learn new systems as they appear? The answer is Coding Backwards.

Trads and Vibes unite! Seasoned programmers and AI-noobs can use this technique to get what you want faster. Your code will be cleaner, more elegant, more robust, easier to expand, and easier to debug.

You can learn a new language while coding in it (?!?!?!?!) and solve problems as you encounter them — but it takes discipline. And the technique requires flipping your mind inside-out. Here’s the recipe.

The Coding Backwards Technique

• Write the Readme ← Start as if you’re already finished .
• Make Skeleton files ← Start with empty files and empty functions
• Progressive Build-n-Test ← Always have code that works
• Explain it to Me ← this one is important, this is how you learn.

Step-by-step

Write the Readme

Don’t just jump into code. Start by writing your Readme. A Readme file is usually done at the end, to tell other people how your code works. Instead of waiting to the end, start with a Readme for your project and use AI to help you write it. Tell OpenAI or Claude or any other model what you are trying to do and ask it for structured assistance. Here’s a sample prompt for building your own Readme.

Prompt:

I'm planning to build a [your project here] app using [React, Python, etc], and I want to start by writing the README to clarify my thinking 
before writing any code. Please help me craft a detailed README that covers:

- Project overview and purpose: what problem does this project solve?

- Project outcomes: what does the final result look like? 

- Core features and functionality my app needs to have

- Technical architecture - components, state management, routing

- Technology stack recommendations (what software or services?)

- Data structure and database

- API design if needed

- Deployment and hosting considerations

- Performance optimizations for my content

- Accessibility considerations

- Future expansion possibilities

- File structure 

For context, here is how I see the app working: 
 
- The user first [does this]
- Then the user [does this]
- The system checks to see [how to handle this]
- And then responds by [doing what I want]
- and then the user [does this]
- etc.

And I need help to think through how to structure this app both 
technically and visually and experientially. 

Please ask me any questions you need to know as a part of your pre-planning. Once you feel like you've got all the info you need, then start writing the Readme. 

Let's begin!

A prompt like this will get you started. Answer all the questions the AI asks in the pre-planning stage and work with the AI to iterate until you’re happy with the Readme.

After you have a pretty good version, give it back to the AI a few more times and ask it to improve it. Ask it to figure out possible security problems, design flaws, usability flaws, etc. Ask it to double-check your file structure for the type of project you are building. Do this a few times to iterate and revise. Once you have a readme in hand, you’re ready to get started.

Make Skeleton Files

Once you have a readme, what’s next? The next step is to build out semi-blank files that follow the plan of the readme and the suggested file structure. Start by creating near-empty placeholder files that follow the file structure listed in the readme. Add a comment at the top that lists the name of the file and the structure. Save each file with the correct file extension.

Name all the files correctly, move them into the right folders, and then one-by-one ask the AI to help you set up empty placeholder functions inside the files.

With all your skeleton files and placeholder functions in place, you can then either start coding (if you’re old school) or ask the AI to help you code, or if you’re like most people, use both traditional and AI-assisted coding at the same time.

Progressive Build-n-Test

With your skeleton files in place, now start writing the actual files. Ask the AI (or yourself) to write the file in a way where you can progressively test it as you go. It’s easier to debug an isolated problem than to try to debug many interconnected, untested, interrelated possible problems.

Ask yourself (or the AI): What’s the simplest thing I can test to make sure everything is wired-up correctly? What the first step I can take? How can I know it’s working?

Once that first step is working, then take the next step. And the next.

Along the way, figure out how you can check yourself before you wreck your code.

Ask yourself (or the AI) how can I add in print statements or temporary output tests to prove that each next step is working? For instance, last night I was building a react app that assembled itself by reading a local json file. The whole thing was complicated, so I first had it just retrieve and display only one element to make sure the pipes were all connected. Once this was working, it was easier to add in everything else.

As you build, you can replace your skeleton functions and tests with more and more complicated and complete functions. Progressive build-n-test means you always know you are building on a base that actually works.

This is different than the yolo-coder’s approach of describing what you want, crossing your fingers, and hoping the AI will build everything for you. Honestly, prompt-n-pray yolo-coding sometimes works, and sometimes is fun, but almost always there’s something wrong or something that nearly works (but doesn’t actually work).

And even if your yolo-code does work perfectly, if you later want to expand or change it, you won’t know how.

Instead, code backwards with the progressive build-n-test approach, so that you’re never lost — because you’re always taking it one step at a time and scaffolding your knowledge as you go.

You can ask better questions, debug faster, and build in an expandable way from the beginning.

Also — and importantly — you get the satisfaction of many micro-victories along the way. We all need encouragement when we build. And progressive build-n-test offers the chance to feel a sense of accomplishment every few minutes, instead of every few days.

Explain it to me

As you work with an AI progressively writing one file at a time, ask it to explain each section of each file. Do this at every step so that you are learning while you are building, rather than just blindly prompting and hoping.

As the AI explains the code, rewrite the comments by hand in your own voice and style in your files to tell yourself what’s happening. Don’t just copy and paste the AI comments. Write them yourself, as a way to double-check yourself on whether you actually understand something. If you run across a line of code or a function you don’t understand, ask the AI what it does.

Asking the AI to explain it to you sounds like it will slow you down, and you’ll be tempted to skip this step, but the opposite is true. If you know what’s happening in your code (even if the AI originally wrote it for you), then you can change it, fix it, expanding it and build what you want. You’ll also have a better chance at debugging when the AI gets it wrong. You’ll build “error intuition” around what’s likely wrong.

Asking “Explain it to me” is the step that most people skip. Your code finally works, so you just want to move on. But this is the key moment that takes you from low-agency passivity to becoming a high-agency builder. This is the moment you learn, grow, and succeed.

The big idea is to take things one step at a time. Plan, execute, test, and learn. Them expand on what works. Save out versions every step of the way. And design projects that walk before they run so that you always have something that’s working, that you understand, and that is able to handle the next layer.

Does this work?

Yes. The process works. It requires a combination of imagination and analysis. It requires dreaming big and then breaking large projects into smaller tasks in a smart and humble way. It requires a love of learning.

And it’s a lot of fun.

Summary: The Coding Backwards Technique

• Write the Readme ← Start as if you’re already finished .
• Make Skeleton files ← Start with empty files and empty functions
• Progressive Build-n-Test ← Always have code that works
• Explain it to Me ← this one is important, this is how you learn.

You can do it!

If you’re a trad, then all you needed from this article was the idea of writing the readme first. If you are new to coding and trying to learn python, React, node, deno, html, css, js, etc for the first time and your only teacher is AI, then I hope this approach gives you a way to get started.

If you do make a project, let me know! I’d love to give you a digital high-five and shout “keep going!” 👋 “I believe in you” 👋 “you can do it!”

Going Further…

This idea of outline, skeleton, build-n-test, learn, and expand feels applicable to other topics outside of coding. How would you use it for learning a language, or learning a sport, or learning to be more kind, or a more thoughtful conversationalist? I’m interested in the idea that you could learn nearly anything if you find the right approach. For instance, I’m at work on a new painting. Here are three recent views. Looking at it, it feels a little like the Coding Backwards idea, these three images would be the skeleton files → build-n-test part of the process. Fascinating!

Going Further…

This idea of outline, skeleton, build-n-test, learn, and expand feels applicable to other topics outside of coding. How would you use it for learning a language, or learning a sport, or learning to be more kind, or a more thoughtful conversationalist? I’m interested in the idea that you could learn nearly anything if you find the right approach. For instance, I’m at work on a new painting. Here are three recent views. Looking at it, it feels a little like the Coding Backwards idea, these three images would be the skeleton files → build-n-test part of the process. Fascinating!

Interested in reading more? Subscribe to my free newsletter.

This guide presents hands-on, practical advice to use story to help you get the job you want. Think of your story as a machine that helps you understand yourself (and get hired doing work you want to do).

Your goal during an interview is to position yourself as a 1-of-1 candidate who posses a unique combination of skills, traits, and experiences. You are not one of many, you are 1-of-1.

When you present yourself as a 1-of-1, then no one else can compete, because there’s only you. For this to work, however, your 1-of-1 combination must solve their problem and give them something new and show you are a special person that they will want on the team. You must do all three.

Examples:

• You posses both the skills of a designer and a the product sense of a PM and you have a personal story of resilience that is inspiring, formidable, and impressive.
• You can code and are a charismatic sales person and you are a beloved leader of a large and powerful open-source software community group.
• You can write copy and illustrate and are loved by a large and respected creative community.
• You are a lawyer and you fluently speak the native language an important sales region and you have a personal reason for working on this problem.

In each example, you accomplish three goals:

1. Solve their problem
2. Give them something new
3. Show you are a special person.

Your 1-of-1 story comes from the choices you’ve made in life, your grit, your resilience, and your attitude. Attitude? Really? Yes. Your attitude matters. No one wants to work with someone who acts like they don’t want to be there, only helps when they are asked, or seems untrustworthy, selfish, or condescending.

1. Tell me about yourself…

Don’t just list how long you worked at your last job… Instead, give your interviewer a reason to care. If you are unsure how, then try this template based on the series This I believe that aired on NPR from 2005–2009.

Here’s my version of the template:

• I believe…
• So I…
• You can see this in how…

Example… Here’s my own “I believe” statement that attempts to make sense of my varied career. Instead of apologizing for being involved in many things, I offer the listener a through-line that gets to the heart of who I am and why I care.

I believe that story, art, and design can bend the arc of humanity’s progress, if you do it right, and so I try to bring this idea into everything I do: from movies to startups to paintings to books and to ballets. You can see this in how I work with founders to design their products and how I at every job I’ve ever had, I’ve always help create something of tremendous value that didn’t exist before. I do this through a unique approach I call Story-Driven Design.

Show how your experiences prepare you to solve the hiring manager’s specific problems. Show that you have a unique combination that they can’t get anywhere else. Don’t try to be the same as everyone else. Instead show that you are a 1-of-1 unique combination that they need to solve their problem. It could be your unique combination of past experiences. It could be your unique perspective on hard problems. It could be a unique combination of skills. Find a combination that no one else has. Win a race that no one else is running.

2. Tell me about a project you’ve worked on…

Don’t just say “we shipped a thing, it was cool.” Instead, tell a story that has the interviewer actually rooting for you. Here’s the template for your story

• We had a problem. It was bigger than we thought. It was critical to the business.
• Initial Failure. We thought we knew how to fix it. Easy, right? But our first attempt was a total failure.
• Realized we were wrong. We were thinking about it incorrectly. What we thought was the problem wasn’t the real problem. We made a second attempt. There was signs of progress, but it wasn’t enough.
• Solved the deeper problem. And then we saw a way to solve the bigger/deeper/real problem. We finally got to the root. We kept asking why, kept digging into what customers were saying, and kept re-examining the data until we found out what was really happening. Every problem is a search for truth. Once you find out the truth, the real truth, then you have a shot at success.
• Transformative Results: big impact, transformational for the business. Changed how we go forward.

3. What else should we know about you…

Screenwriters have a way to create compelling characters. You put the character into a nearly impossible situation and then watch what they do. Then ask: “what choice does the character make with it really matters?”

The choices that matter are the big ones. Everyone has a few of these stories. So before the interview, review these for yourself what choices have you made when it really mattered? Reveal your true character and we will root for you to win. The interviewer won’t pull stories out of you. You need to provide them in the right places when it matters most.

Final Note: The Gift of Practice

Interviewing for a job is hard. Telling a compelling story is hard. It takes practice. And what makes the problem even worse is that you can’t sound so rehearsed that you sound insincere. You have to deeply know your stories, so that you can deliver them with ease. Spontaneous telling.

Not memorization — but embodied, in the moment telling.

If you want to workshop your stories, call this number. It’s free.

(650) 675–5734

It’s free. Open 24–7 365 day-n-night. And is programmed to be supportive and helpful. This is my Delphi. It’s an AI-version of all of my writing and posts you can use to talk through your story, art, and design problems.

Give it a try. Call or text this number: +1 (650) 675–5734.

Thanks for reading. Best of luck on your job search. I believe in you.

Interested in more?
Subscribe to my free newsletter here: https://jamesbuckhouse.substack.com/

The departed return through our stories.

The word eulogy means “good word” in latin. It is a chance to say a good word about the deceased and help those who have gathered to remember a life well lived.

Your role is to conjure a moment, a story, or a feeling that we can hold onto during our grief. It is a chance to sustain the memory of the departed. The dead are alive again through our stories. It is a chance to bring them into the room for another moment with us.

The best stories help us remember what they cared about, what they loved, and what set them apart. The story you choose might show them at the peak of their powers, or help us remember one of their most deeply felt joys. It could be a quiet moment, a funny moment, or an example of their generosity to others. It could be a story of how they changed someone else’s life or a touching moment when they showed kindness or strength. It could be a telling triumph or a moment when they handled difficulty with grace. It might be happy and sad at the same time. It could be nearly any story, as long as it’s true, vivid, and helps us keep them alive in our hearts.

How to Begin

If you’re not sure where to start, ask yourself:

• What did this person love?
• How did they make people feel?
• What moments defined them?
• What choices did they make in life when it really mattered?
• How did they help others?
• How did they show their love?
• How did they act when they were at their best?

A Simple Structure to Follow

A eulogy can be three acts, like a short story:

1. Introduction — Who was this person? What made them special?
2. A Story or Theme — Pick one or two moments that capture their essence. Give us a moment that we can feel.
3. Closing — What will we carry forward from them? What would they want us to remember? How can we make a change in our own lives to honor theirs?

How to Find the Right Tone

You don’t have to tell every detail. A eulogy is not a résumé — it’s a portrait. Give us a sense of who they were when they were living to their fullest. If you cry, it’s okay. If you don’t, it’s also okay. There is no single correct way to deliver a eulogy. What matters is that you deliver to us a moment we can feel. Help us feel what we need to feel.

What to avoid

Do not talk too much about yourself; a eulogy is not about you. Avoid nervous jokes. Avoid revenge. Avoid belittling yourself, the audience, or the deceased. Turn off your phone. Speak from the heart.

Eudaemonia

Aristotle’s idea of Eudaimonia (a flourishing life) reminds us: The measure of a life isn’t just in accomplishments, but in how it was lived. A good eulogy helps us see what mattered most in a person’s life and, in doing so, reminds us what matters most in our own.

Alternative—Give a (meaningful) toast

if writing a eulogy is too much or too emotional or somehow doesn’t feel right… Instead try imagining you’re giving a toast. Here is a guide to doing exactly that (and how to do it well).

This weekend I took every public blog post, Medium post, Tweet, Insta, Substack, etc. I’ve ever written and used it to create an Intelligence Archive of my Digital Mind using Delphi.

This creates a totally new way to interact with a body of text. Instead of reading everything I’ve written, you can just ask me about it. You can call me or text me and we can have a whole conversation about story, art, design — except, of course, I’m not actually there, you’re talking to my second self.

The digital voice sounds a lot like me (it’s not perfect, but honestly it’s not bad). And the answers it gives are based on what I’ve written.

Try it: +1 (650) 675–5734

(you can call or text, it works like a normal USA phone number.)

Might I recommend a voice call?

Ask my about the movie you are writing or the product you’re designing. So far I’ve helped people talk to their bosses about getting a raise, I’ve helped people write website copy, I’ve helped brainstorm art projects. I’ve helped plan parties, write toasts, and write headlines…. All by giving people a radically new interface to traverse, explore, and engage with my past writing.

And if you do call me, reply here and let me know how it went… I’m looking forward to hearing what you think.

The most interesting class on AI being taught today isn’t for engineers at Stanford, MIT, or Harvard, but for art students at RISD, the famed institution that produced both the Talking Heads and Airbnb.

Taught by Kelin Carolyn Zhang, best known for being one half of the design duo behind Poetry Camera — the camera that uses AI to print out poems of what it sees instead of photos — the course takes 3rd year Industrial Design students and challenges them to create real products and apps worthy of a professional designer or a VC-backed startup.

How are the students doing it?

The students are encouraged to make deliberate and abundant use of AI. Zhang has turned the traditional process upside down. The students protype, test, and iterate, using AI. And when they’ve got something they want to take further, they then just jump into coding the apps and solving problems — even if they have no idea how — by asking AI for help every step of the way.

She has publicly posted her syllabus. You can read it here

RISD AI Design Studio | Fall 2024

Notice it’s a Notion doc, not a PDF. That’s because she changes and rearranges as she goes — adding in new guest speakers and adjusting the course based on the accelerating progress of the students. For instance, the students quickly grew out of basic artifacts and wanted to code real projects using APIs, so she’s moving that up so the students can learn it sooner.

She is also tweeting about the experience as it goes. You can follow along the Tweet thread here

https://twitter.com/kelin_online/status/1843731509246865606

Zhang has also stacked the class with designers and engineers from her previous life in the tech world: MidJourney, Figma, Humane, OpenAI, Apple and more. This past week I was one of her guest lecturers. I spoke on the Architecture of Story and Agentic Design.

Lecturing in her class felt a little like a homecoming for me. I had studied painting at RISD in the 90s in a precursor to the dual degree program with Brown. I did it by getting into both schools and then petitioning both administrations to let me take more than the normal four crossover classes. I moved fluidly between both schools. It was wonderful. It also put me in a position to operate at the intersection of different disciplines.

(side note: have you seen 90s Art School? an instagram account of scanned photos from that era at RISD). (Read more bout it here)

Now back to the AI design class…

A few months ago, I hosted a workshop on “Agentic Design” called Augmented Imagination (after the article I wrote in 2023), where I posited that the future belonged to artists and designers.

The idea is that when AI can code anything instantly, then the only thing that will matter is what we design in the first place. And that artists and designers are best positioned to do this well. We then walked through the emerging practices that used AI to augment the imagination of the creators (and help them build it). Zhang presented her Poetry Camera at the event. It was a huge hit. She and her art partner Ryan Mather, were one of the best examples of people who were living up to the claims I had made that AI can help you build seemingly impossible projects.

Now she’s making this happen for a whole generation of students. Midterms are next week, I can’t wait to see what the students create. And by the time the students do their final projects, each will have transformed into an artist/designer who builds.

I’ve lectured at many schools, and the students in all of them are universally smart and engaged, but this class, in this moment, felt different. It felt like watching people discover the world could be recreated, shaped, formed, adjusted, improved — and that all we had to do was try.

But unlike other eras, where human effort, no matter how industriously applied, was never quite enough, this generation has AI on their side. AI holds within it the sum of all human knowledge. It contains all the ideas, research, thoughts and effort of everyone who came before, all bundled into a curiosity-machine that then waits for us to ask it what we need to know. Suddenly the idea that we might make the word we wish to inhabit didn’t seem so far-fetched. It seems possible. Or at the very least, it seems exciting to try.

Originally published August 2024
updated March 2025

An ever-updating collection of new approaches to UI/UX in the era of AI. If you see something in the wild… please lmk and I’ll add it to this doc and credit you (of course!) for mentioning it!

Categories:

• Dials, Knob, and Sliders
• Direct Manipulation
• Node-n-Edge Graph
• g(x) Generated Experience
• The Document in the age of Contextual Intelligence
• The Infinite Canvas
• LOD Summarization via zoom
• Voice Input
• Visual Interface
• The Side-by-Side
• Whole Body Interface
• AI Chat Comments

Dials, Knobs, and Sliders

Physical knobs, digital sliders, and quadrant “dials” can be used to adjust the tone of the response or other input variables for AI interactions.

This first novel UX idea comes from the generative text feature in Figma’s new Figma Slides product. You can slide the orange indicator around to blend the tone from casual to professional and from concise to expanded.

Here’s an example where I’ve added in Tone-Sliders to adjust the vibe of text mid-document.

This example is from a post by Twitter user Johannes Stelzer. I do not know the details, but loved this demo.

Johannes Stelzer on X (formerly Twitter): "Happy mixing of decoder embeddings in real-time! Base prompt is 'photo of a room, sofa, decor' and the two knobs are 'industrial' and 'rococo'. If you are wondering what is running there in the background... pic.twitter.com/5svyDy5C4e / X"

Happy mixing of decoder embeddings in real-time! Base prompt is 'photo of a room, sofa, decor' and the two knobs are 'industrial' and 'rococo'. If you are wondering what is running there in the background... pic.twitter.com/5svyDy5C4e

Here is an example of “DJ Knobs” to adjust the direction of a story.

Here’s my own little AI project… a Chrome extension that summarizes any web page. You can adjust the output with sliders. Want a four-word story? Easy: Adjust it to just 4 words. Want an essay? Go long.

Direct Manipulation

Here’s an example of direct manipulation of a text field with a slider-like bar that regenerates the text.

And another with pinch to expand & contract to rengenerate.

The Node-n-Edge Graph

LangGraph Studio uses a node and edge graph system to visualize the logic and flow of the LangChain Agents. Inside each Agent is a collection of nodes and edges. Each node is a “micro-agent” that does a task and connects to other nodes to make a system or a flow. In this way, you can build complicated agents from basic building blocks.

g(x) Generated Experience

Decart has a model that is so fast and so deep that you can play it like a video game. But there is no game engine, just the model. Decart envisions a future were every experience is generated.

The Document in the age of Contextual Intelligence

Eddie Jiao at Common Knowledge is looking at documents in the AI era. Here’s an Tweet of a demo.

The Infinite Canvas

An infinite canvas give you an open space to build and create as you go. Figma, FigJam, TLDRAW, and Visual Electric are great examples. Here is a look at Visual Electric’s canvas.

Figma and FigJam also have “infinite” canvases. The working environment lends itself to expansive creativity. Here is Julie W. Design jamming on the infinite canvas with FigJam AI.

Here is an example from Cove that uses an infinite canvas to create workflows.

Here is an example of Claude’s Artifacts and TLDRAW’s infinite canvas uniting to make a near instant site builder.

LOD Summarization via zoom

Here Eddie Jiao of commonknowled.ge shows us a Level-of-Detail (LoD) approach to summarization. I love this.

Voice Input

Our digital experiences have had voice input before, but AI’s mental model of a “chat interface” makes voice input feel natural. This is not the only mental model we can have for AI, but it is the one that gained traction with OpenAI’s ChatGPT. (h/t David Lam)

OpenAI on X (formerly Twitter): "We're starting to roll out advanced Voice Mode to a small group of ChatGPT Plus users. Advanced Voice Mode offers more natural, real-time conversations, allows you to interrupt anytime, and senses and responds to your emotions. pic.twitter.com/64O94EhhXK / X"

We're starting to roll out advanced Voice Mode to a small group of ChatGPT Plus users. Advanced Voice Mode offers more natural, real-time conversations, allows you to interrupt anytime, and senses and responds to your emotions. pic.twitter.com/64O94EhhXK

And here is Dot by New.Computer. In addition to typing in your chats… there is an elegant voice input option. If you look in the back left of this image… you can just see a screen with a card view. It’s not an ordinary card view, however, you get there by “pinching out” from the main chat. This is a wild, very very cool and fluid experience that proves why Jason Yuan is one of the best of the best.

Visual Interface

AI-tools are also able to process visual input. Here is OpenInterpreter looking through a webcam to see Killian holding up a sticky note, and then proceeding to connect to wifi.

killian on X (formerly Twitter): "i showed a fully local, computer-controlling AI a sticky note with my wifi password. it got online. pic.twitter.com/pFcUhP1jUP / X"

i showed a fully local, computer-controlling AI a sticky note with my wifi password. it got online. pic.twitter.com/pFcUhP1jUP

The side-by-side

In this approach, you chat on the left and get results on the right. This pattern works on desktop, but requires tabs or swipes or something else for mobile. Here are a few examples.

Here’s Layla’s travel planning AI. You chat away about your hopes and dreams for a vacation in the left window. Results, links, videos and offers appear on the right, along with your itinerary.

Here’s Claude’s side-by-side UX for when you are asking Claude to help you code something. There’s a standard chat window on the left and a results page on the right. In this case, the results have a toggle at the top so we can see the code and the results of the code.

Whole Body Interface

This video shows media pipe to track your gestures and then create results. Is this AI? Well… yes… there is ML in there. Mostly, though, how cool!

AI Chat Comments

What if the comment rail in a doc was a chat input for AI? Or Arbel shows us how here:

Stay tuned! I’ll add more examples as they come in… and if you see something interesting, please lmk so I can add it to the list.

OVER FIFTY YEARS AGO, design legend Dieter Rams penned his legendary 10 points for good design.

Below are 10 additional suggestions for good design that attempt to reflect today’s state of perpetual (and accelerating) change.

Good design tells a story.

Good design solves a problem.

Good design extends our understanding.

Good design improves the human condition.

Good design transforms its audience.

Good design leverages constraints.

Good design complements its context.

Good design enlists us as its collaborator.

Good design welcomes inclusion.

Good design is never finished.

❤