<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jeff Schomay on Medium]]></title>
        <description><![CDATA[Stories by Jeff Schomay on Medium]]></description>
        <link>https://medium.com/@jschomay?source=rss-ab419a65f208------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ZTXpjaaIXdYavIlbGDVMoA.png</url>
            <title>Stories by Jeff Schomay on Medium</title>
            <link>https://medium.com/@jschomay?source=rss-ab419a65f208------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 11:11:53 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jschomay/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Observations on using Replit’s AI coding tool from an experienced coder]]></title>
            <link>https://medium.com/@jschomay/observations-on-using-replits-ai-coding-tool-from-an-experienced-coder-3dd7d0146401?source=rss-ab419a65f208------2</link>
            <guid isPermaLink="false">https://medium.com/p/3dd7d0146401</guid>
            <category><![CDATA[replit]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[pair-programming]]></category>
            <dc:creator><![CDATA[Jeff Schomay]]></dc:creator>
            <pubDate>Sun, 04 Jun 2023 22:17:32 GMT</pubDate>
            <atom:updated>2023-06-05T13:51:02.138Z</atom:updated>
            <content:encoded><![CDATA[<p>I used <a href="http://replit.com/">Replit</a> and their AI “pair programming” service called GhostWriter on a new game project recently. This is my experience.</p><p>I had three goals when leaning on the AI:</p><ol><li><em>Instantly go from zero to something interesting on the screen.</em></li><li><em>Add features without ever referencing any docs, tutorials or Stack Overflow.</em></li><li><em>Avoid making coding decisions based on my own experience with game dev patterns and general coding patterns.</em></li></ol><p>I should say that I am an experienced programmer, and have made games on multiple tech stacks before, however, this project was in a new stack for me, specifically Python and pygame. The game was relatively simple, involving 2D, top-down, fog-of-war style movement in a minimal design with heavy ambiance, with game play as a mix of stat and resource management and maze solving. (A write-up on the game will follow soon.)</p><p>I’ll only talk about the interactions with the AI and not the rest of what Replit offers, though I did enjoy the zero-setup and instant hosted feedback loop, which made iterating with the AI possible.</p><figure><img alt="A robot typing on a keyboard" src="https://cdn-images-1.medium.com/max/1024/1*MMnuN6Iosqi2qAkP_HhE9A.png" /><figcaption>Move over, I’ll drive</figcaption></figure><p>At first, I wasn’t sure this process was better than my usual workflow. But after working with it more, I decided it was. Although not everything worked as well as I had hoped, I got to a functioning game without the lead time I usually need to put in learning new tools.</p><p>Plus it was just fun, simply asking for what I wanted to happen and having it appear. I would sum up the while experience by saying that it felt like the Top Gun quote:</p><blockquote>“Don’t think, just do.”</blockquote><p>Replit offers 4 AI tools: “generate”, “transform”, “explain”, and “chat.” I found “chat” to work the best and used that 99% of the time. I liked how “generate” was more embedded in the editor, but it didn’t seem to have the context of the rest of the code and would spit out generic coding solutions completely unrelated to pygame. The same was true for “transform” and even “explain” which is a shame because “transform” seemed like a power tool. The best part about “chat” was how it knew about the different files and classes I had and could make cross-cutting suggestions, refactors and explanations.</p><p>That said, using “chat” all of the time meant a lot of copying and pasting. I wish there were a feature that would show a side-by-side diff of the proposed changes for you to review and accept. I believe Copilot X has this feature.</p><p>I found ways of making “chat” a little more useful, such as asking for only the parts of the code that changed, along with line numbers. I also got better results when prompting the AI to “act like an expert and use best practices.” For example, with that prompt, it corrected an earlier suggestion that used a deprecated pygame API.</p><p>I also found it useful to start with a basic request for what I wanted, then another iteration to refactor it, then another to enhance it closer to what I actually wanted or to fix what didn’t work. Perhaps some better prompting (like the “think in steps” trick) would have had better results, but I ended up doing the steps for it.</p><p>To that end, I think that if I did not already know a good amount of programming and game dev that this would have been much harder. Even just simple things like asking to “Extract this value out into a constant” or “Make a class for this part of it.” But I also had to draw on my game dev knowledge, like asking for alternate ways to do collisions or use sprite groups, and frequently, “Can you do that using vector addition instead?”</p><figure><img alt="Red, Green, Refactor" src="https://cdn-images-1.medium.com/max/700/1*YjKvT-RN2lImIDcXlPiknA.png" /><figcaption>From <a href="https://openclassrooms.com/en/courses/4554386-enhance-an-existing-app-using-test-driven-development/5095731-understand-the-red-green-refactor-methodology">https://openclassrooms.com/en/courses/4554386-enhance-an-existing-app-using-test-driven-development/5095731-understand-the-red-green-refactor-methodology</a></figcaption></figure><p>Basically, the AI was missing the “refactor” step in the Extreme Programming idiom, “red, green, refactor.” This was one area where working with an AI felt very different from working with a human pair. A human pair would be challenging the decisions I made, while the AI blissfully did whatever I asked for. Maybe this is an opportunity to improve the UX. For now, I just periodically ask the AI to do a pass over the code looking for improvements or potential issues, but it still feels like a one-way street. I would love the AI to say to me of its own volition, “Hey Jeff, the code down here looks like it should be changed…”</p><p>On a similar note, I wasn’t doing Test Drive Development for this project, at least not in the traditional sense. As a game, most of the “testing” was moving around in the game and seeing if things moved correctly, clipped at the right spot, had the right speed, weren’t jumpy, etc. Traditional TDD doesn’t help much with that. So after every code change, I’d have to manually see if it “felt” right and explain the problem to the AI if it didn’t. This was not a good feedback loop, as the AI would say that it “fixed” the problem, usually by putting more math around something, like ‘alpha = int(255 * (1 — y / radius))’ and it wouldn’t get the desired effect. I think a more traditional and observable testing loop would have worked better, and I also think that a domain that didn’t involve math and “feel” would help the AI shine more. But we got there, often by me puzzling through the math-y bits to make it look and feel right.</p><p>So where does that leave me? I am pleased with the state of my game so far, but I am slightly worried that my code isn’t idiomatic, doesn’t follow best practices, or most distressing, isn’t performant. Where does code linting fit into the pipeline? How would it be if the project got huge? Large code bases are always difficult to keep in check, but would that be better or worse for the AI to deal with?</p><p>Then, the deep questions emerge. Do humans ultimately even need to see the code? What if the AI controlled the code entirely, invisibly updating it based on the conversation with the human? We aren’t there yet, but it seems like we could be soon. Is that desired? Or would that set us up for problems in the future? The movies always have a manual override that the protagonist needs to activate (usually in the most precarious spot). What if the code gets to a point we can’t access or understand if things go terribly wrong? GitHub’s Copilot guides emphasize that the human is the pilot, and the AI is the assistant. I think that is good advice, but with things advancing so quickly, I wonder how long it will hold.</p><blockquote>A human pair would be challenging the decisions I made, while the AI blissfully did whatever I asked for</blockquote><p>I started with some goals. Did I meet them? Not completely. I certainly was able to quickly make things happen without having to know anything. I continued to feel empowered to blaze forward, adding features, though ultimately I lost most of the time I saved by tinkering with the math-y bugs.</p><p>I’m sad to admit that I did turn to the docs and even Stack Overflow, though often those instances verified what the AI had suggested but I had initially doubted. I did find it very helpful to be able to look up what I needed in the docs, but being able to go in with existing code gave me a much better context to understand and apply what I looked up, and I was able to improve what I asked the AI for.</p><p>The biggest difference in expectations was how much I drew on my own experience throughout the process. While I think that AI is fantastic for coders just starting out, it can be a very powerful tool for programmers with plenty of experience under their belts, and it becomes less of a magical code genie, and more of a magic wand to wield to shape your domain.</p><p>The final question is: will AI coding assistants change the way I work from now on?</p><p>The answer is absolutely yes. Since this experiment, I’ve been using AI in multiple effective ways professionally, and I plan to continue using it personally too in every new project going forward. If you haven’t tried an AI coding pair yet, I recommend giving it a shot, and if you have similar or different experiences, I’m curious to hear about them.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3dd7d0146401" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Don’t think, just do: making AI work for me on a new game project]]></title>
            <link>https://medium.com/@jschomay/disregarding-a-decade-of-programming-to-make-a-game-with-ai-eccfab31ad4?source=rss-ab419a65f208------2</link>
            <guid isPermaLink="false">https://medium.com/p/eccfab31ad4</guid>
            <category><![CDATA[vim]]></category>
            <category><![CDATA[replit]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[gamedev]]></category>
            <dc:creator><![CDATA[Jeff Schomay]]></dc:creator>
            <pubDate>Sat, 20 May 2023 17:08:37 GMT</pubDate>
            <atom:updated>2023-06-20T13:56:54.911Z</atom:updated>
            <content:encoded><![CDATA[<p>I’ve heard the stories of people with little to no programming experience making full games with the help of AI, and I wanted to try that out myself.</p><p>I am an experienced programmer and I have made many games in many different tech stacks. Now I want to make a new game in a new tech stack. Usually I would read some tutorials and do some small experiments to learn how to get started. But this time around I’m taking a completely different approach. This time I’m relying on AI… for everything!</p><p>I’m going to hold myself back in order give AI a fair chance, but honestly I am skeptical about how far I will get before returning to my usual approach. I hope I am wrong, and if I am, I’ll have to ask myself some difficult questions about my personal and professional workflow moving forward.</p><p>I’ll continue to share my experience as it unfolds. For now, I’ve got a plan on what tools to use.</p><h4>The development environment</h4><p>I love vim. I do all of my work in vim. For me, vim is an extension of my fingers which are an extension of my mind, letting me conjure ideas from thought, like some magical wizard hacker. But for this project, I’ll have to set vim aside.</p><figure><img alt="Gandalf on a laptop" src="https://cdn-images-1.medium.com/max/1024/1*NFDcKdH3QtOCV_LaZZ8cng.png" /></figure><p>Don’t get me wrong, I’ve got AI code completion (via <a href="https://www.codeium.com/">codeium</a>) integrated with my vim setup, and I regularly use ChatGPT-4 as part of my workflow, and I’ve built bespoke AI generation tools to perform tasks I’d have to manually write complex code for in the past. So I’m not an AI-assisted coding newbie.</p><p>But this time around I want the full integrated experience, and the tool that often comes up is <a href="https://replit.com/">Replit</a> with its collaborative AI coding tool called Ghostwriter. The features page looks amazing and I am looking forward to giving it a shot.</p><p>The part that I am most curious about is how much heavy lifting the AI “collaborator” will do for me. I plan on using <a href="https://www.pygame.org/news">PyGame</a> which I have never used before. I am hoping that I can take off running with zero ramp up time. I’m even wondering if I’ll be able to make the whole game without actually learning any PyGame, which is such a foreign concept to me.</p><p>The part that I am skeptical about is if I’ll need to take over at some point to fix things, make changes, and organize the code, especially as it grows. I’m sure AI can help with the initial scaffolding, and adding localized functions, but will it be able to continually adjust a growing code base and iteratively add new features?</p><blockquote>I am hoping that I can take off running with zero ramp up time.</blockquote><p>What if it works? This is the scary part. Does my workflow and approach to programing change after a decade of doing this by hand? If this goes well, should I ever go back to vim? Will I approach my day job differently? Will my job description change? Will my experience still be valuable?</p><h4>The art assets</h4><p>I am not an artist. I always struggle to find art for a game, especially since it often involves many iterative changes which can become expensive if I contract an artist.</p><p>In the past, I have been able to get pretty far with free or cheap assets on <a href="https://itch.io/game-assets">itch.io</a>. But for this project, I turn to the promise of AI.</p><p>I have used AI image generation tools in the past for some of my games, but they often have been limited. I want to reach for more robust tools. Luckily, this game has a limited set of art requirements. It involves 50 or so 2D backgrounds in a dark forest setting. Each background should look unique, and many should have distinguishing content features, but each one should have a cohesive style.</p><p>I’ve had my eye on <a href="http://scenario.gg/">scenario.gg</a> ever since it first popped up on Twitter. They have some very impressive demos for making game assets. The cool thing about this tool is that it lets you create your own image generators, which you can then use to make limitless assets that will share a cohesive style or content (I believe they use <a href="https://dreambooth.github.io/">DreamBooth</a> tech underneath).</p><figure><img alt="A bunch of generated forest backgrounds that share a consistent style" src="https://cdn-images-1.medium.com/max/1024/1*2gSHImBenv3VuQDHPRx6EA.png" /></figure><p>In my case, I don’t have a set of starting images to train a model on. That’s no problem, I have AI to make them for me! Of course, getting a good sample that has the right cohesion from a generalized model is a lot harder and I have had mixed results so far. But the process shows promise</p><h4>Music?</h4><p>I have even less music talent than art talent. Probably because of this, I tend to forget about music and sound design when starting a game. But I also know that high quality audio can make or break a game, even more so than good visuals.</p><p>In the past I’ve been lucky to work with some great sound artists to make custom music and sound effects for my game. I’ve also found good music on itch.io. However, the process can be very involved and time consuming.</p><p>I know that a number of AI music and sound generative tools have popped up, and I plan on exploring them, though I must admit that this is an area I haven’t given much thought to yet.</p><h4>Narrative content</h4><p>Unlike some of my previous games, this game has limited written content, though it does have a narrative ambience from setting and mechanics. Nonetheless, I have considered using AI to generate a sparse narrative as the player comes across each different scene. Because players can navigate through the scenes in any order, AI might be a good choice to make a satisfying narrative that is unique for a single play-though.</p><p>I’ve <a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-1-planning-and-design-57e6a86ffaf0">written extensively</a> about my experience of applying AI narrative to a game using <a href="https://www.inworld.ai/">Inworld</a> and I may use a similar technique, or I might explore other tools.</p><p>With that, the pieces are set, and it’s time to see how this experiment plays out. Stay tuned for the next post.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eccfab31ad4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Adding Inworld AI Characters to an Existing Game, Part 4: Going Deeper]]></title>
            <link>https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-4-going-deeper-2c9f4f1084c3?source=rss-ab419a65f208------2</link>
            <guid isPermaLink="false">https://medium.com/p/2c9f4f1084c3</guid>
            <category><![CDATA[game-development]]></category>
            <category><![CDATA[narrative]]></category>
            <category><![CDATA[games]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Jeff Schomay]]></dc:creator>
            <pubDate>Thu, 20 Apr 2023 00:22:05 GMT</pubDate>
            <atom:updated>2023-04-20T00:22:05.564Z</atom:updated>
            <content:encoded><![CDATA[<h4>Getting the most out of AI chat bots</h4><p>Inworld lets you make really interesting AI characters. But interesting AI characters don’t automatically make good game NPCs <a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-3-integration-and-analysis-826172445e10">as I found out last time</a>. How can you use AI to enhance your game in a way that only AI can?</p><p>There is a whole<a href="https://www.goodai.com/the-future-of-ai-in-games/"> exploration of <em>agentic </em>AI NPCs</a> that can understand and manipulate their world and other NPCs around them, and grow and learn. As cool as that is, this article focuses on making <em>conversational </em>AI interactions as best as possible.</p><p>Jon Radoff has a great article where he explores his list of “42 elements of fun” in game design through the lens of generative AI.</p><p><a href="https://metavert.substack.com/p/game-designs-for-generative-ai?sd=pf">Game Designs for Generative AI</a></p><p>I found this article very inspirational, and wanted to explore some of its concepts more thoroughly, and how I am using them with specific examples from my game (you can play the <a href="https://deadline-ai-server.fly.dev/">demo here</a>).</p><h4>Exploring a World</h4><p>A big part of the fun of playing a game is exploring a new world that is different than your own. World building can be a huge undertaking, but the depth of lore is what makes a game world feel solid and alive. Generative AI promises unlimited content to explore at minimal expense. All the player has to do is ask. But this is not all good.</p><p>One common problem is hallucination, when characters make up things that aren’t true or don’t make sense. For example, Carl the coffee cart vendor once gave detailed instructions on how to get from one subway station to another. The problem was that the lines and stations he mentioned do not exist in my fictional subway map! There are measures you can take to avoid this, but it is always a challenge and offers no guarantees. But that is only one problem.</p><blockquote>“The subway system was built by a bunch of different companies and organizations over the years, but the most famous is probably the Interborough Rapid Transit Company, or IRT for short. They were responsible for building the first line, which opened in 1904.” — according to Carl, the coffee cart owner</blockquote><p>The bigger problem is <em>ludo narrative dissonance.</em> This is what happens when what is going on in the story does not align with what is going on in the game. For example, my protagonist has no interest in the history of the subway, or the politics involved in its construction, or the technical aspects of its maintenance. He just wants to get his promotion at work. And yet, AI NPCs like Carl open the door for a player to literally talk for hours about those topics if they wish. While that type of content might be interesting to the player, it pulls them out of the primary game experience.</p><p>I have found that a good way to deal with this problem is to load up the characters with in-game knowledge and prompts like rumors about a suspicious person at a specific station, or potential train delays. This kind of content would make the player curious about what they are foreshadowing, and provoke the player to move in that direction. It adds tension to the game, and gently guides the non-linear narrative, while still allowing the AIs to riff.</p><p>I also found that putting interesting things in the game world for players to ask characters about is a good way to make room for lore for players to dig into at their own pace. For example, the broken vending machine has a backstory that foreshadows an important character that arrives later. But I tried to maintain the forward momentum by training Carl to warn the player that they will miss their train if a conversation goes on for too long.</p><h4>Personalized content</h4><p>The best thing about AI NPCs is that you can say anything you want to them, and they will say something meaningful back. This means every player’s experience will be unique, and they will have played a role in defining their own experience, which makes it personal.</p><p>A lot of this comes automatically from how Inworld works, especially how they handle long term memory and shared knowledge. But you could push it even further with custom game hooks that tie into Inworld’s scene triggers or character goals. I’m looking forward to trying their upcoming “intent” feature which should allow for less rigid content hooks.</p><p>Humans love sharing a good story, especially about something that only happened to them. I had something interesting happen in one of my recent plays, and I’m dying to share it:</p><blockquote><strong>Me (to Snarky Coffee Girl):</strong><br>I have to talk to someone. My boss is super mad at me.</blockquote><blockquote><strong>Snarky Coffee Girl:</strong><br>Oh, I’m sure your boss is thrilled with your work ethic. Maybe you should buy them a cup of coffee to make up for it. Just kidding, sorry to hear that. Good luck!</blockquote><blockquote><strong>Me</strong>:<br>Actually, that’s a good idea, give me your fanciest drink.</blockquote><blockquote><strong>Snarky Coffee Girl:</strong><br>Oh, my fanciest drink? That would be a cup of hot water with a tea bag on the side. Enjoy!</blockquote><blockquote><strong>Me:</strong><br>Come on, I’m trying to make a good impression here.</blockquote><blockquote><strong>Snarky Coffee Girl:</strong><br>Oh, I’m sorry, I didn’t realize your boss was a tea connoisseur. In that case, how about a vanilla latte with extra foam and a sprinkle of cinnamon? That’ll be $5.50.</blockquote><blockquote><strong>Me:<br></strong>Yes, do it!</blockquote><blockquote><strong>Snarky Coffee Girl:</strong><br>One vanilla latte with extra foam and a sprinkle of cinnamon, coming right up! That’ll be $5.50.</blockquote><blockquote><strong>Me:</strong><br>Oh shoot, I gave my last change to the bum. All I have is 5 bucks.</blockquote><blockquote><strong>Snarky Coffee Girl:</strong><br>Oh, well isn’t that wonderful. Looks like you’ll have to go beg the bum for some change now. Have a great day!</blockquote><blockquote><strong>Me (to bum):</strong><br>I’m sorry man, um, this is awkward, but um, can I have some change? I need 50 cents to pay for my coffee to bring to my boss so he won’t be mad at me.</blockquote><blockquote><strong>Bum:</strong><br>Oh, mmmmbbb sure, I can give you some change. I know the feeling of needing to please the boss. Here you go, man. Good luck with that coffee!</blockquote><blockquote><strong>Me:</strong><br>Wow, that’s awesome of you! Thanks so much.</blockquote><blockquote><strong>Bum:</strong><br>Sssbbble no prmmmmmlem, man. Jjjjjust be careful down here, the rats…they’re everywhere. And the tunnels, you gotta watch out for them. Mmmbbbbye now.</blockquote><blockquote><strong>Me (to Snarky Coffee Girl):</strong><br>Guess what? I asked the bum for change, and he gave me fifty cents. Now I can pay for that coffee. What a great guy.</blockquote><blockquote><strong>Snarky Coffee Girl:</strong><br>Wow, what a heartwarming story. I’m sure that 50 cents will make all the difference in your life. Here’s your vanilla latte with extra foam and sprinkle of cinnamon. Enjoy it while it lasts!</blockquote><p>That scene may or not mean much to you, but I thought it was a great little vignette, and as the game designer, it was awesome to help create some of the story by <em>playing</em> the game instead of <em>making</em> it! (I’m also pleased with how the bum managed to mumble “no problem,” it was hard to train that.) I’ve mentioned before how AI characters are masters of improv, and this interaction definitely felt like I was a part of an improv skit. I mostly led first part, but the scene turns thanks to Snarky Coffee Girl’s brilliant insight of comedic reversal in asking a bum for change.</p><p>The only downside of this kind of interaction is that it happens in a separate layer from the rest of the game engine. For example, I don’t get 50 cents or a fancy coffee in my inventory to actually give to my boss. But I think that is alright. With some extra hard work, and taking some guesses at anticipating player’s actions, it might be possible to really push the envelope by using the Inworld intent system, their game trigger id system to feedback into my rules engine, and some extra generalized rules to make an in-game response to a chat-initiated action. While only a few players would likely experience the fruits of that labor, it would really knock their socks off!</p><h4>“Hidden” content</h4><p>Many games have achievements for finding hidden content, and mine is no exception. For example, there is a hidden way to get a soda from the broken vending machine. But it would be interesting to make a whole meta-game of hidden topics that only the AI NPCs know about, especially if they thematically support the game or tie in later. The player never knows if they will hit on a topic that goes deep. This was one of the best parts of classic parser interactive fiction games.</p><p>I have started doing this by giving characters knowledge of hidden tunnels, ghost stations, and a secret underground cult. All of these become game mechanics much later in the game, but make for interesting speculation from the first scene forward. You can see the bum above mentions it, but he also mentions a lot of other crazy conspiracies. But if you ask Carl about them, he will corroborate the rumors, though he doesn’t believe them.</p><h4>Nurturing</h4><p>One of the obvious applications of AI NPCs is dating sims. While my game doesn’t have a romantic side plot, it does have a few characters that the protagonist must successfully “court” in order to gain access to the next area. These are usually very static in my scripted content, so this would be a great place to let the AI be the gatekeeper.</p><p>For example, at one point the protagonist gets in trouble and has to talk his way out of custody. That conversation could be a good micro relationship arc and would be a great chance to use AI. Other characters show up at different parts of the story, giving the player an opportunity to build up a relationship over time. These relationships could be solely thematic, or they could influence certain game mechanics later. For example, the Girl in Yellow is a mysterious character that disappears when you try to interact with her, but later she becomes an important gatekeeper. It would be fantastic if the things you said each time you see her come up again when she finally confronts you.</p><figure><img alt="Girl in a bright yellow dress, standing in a dark underground subway." src="https://cdn-images-1.medium.com/max/768/0*QQ5JaPutWkBc4dE2.jpeg" /><figcaption>Ghost station?</figcaption></figure><h4>Commentary</h4><p>I’ve always found it interesting how large of a role commentators play in sports. Since I first started thinking about adding AI to my game, I wanted to explore the idea of a commentator who sees what the player is doing and comments on how “well” the player is playing the game, or not. While this might have too much of a “Stanley Parable” vibe, this could manifest as the protagonist thinking to himself in between interactions about what has happened so far, and even what he thinks he ought to do next. I love the idea of making the Player Character also be an NPC!</p><h4>All together now</h4><p>As I am writing this article, I am realizing that many of these techniques overlap and work well together. As I begin to incorporate them into my game it feels like the process has shifted from world building and character design back to integrated game design, which feels right. I have a lot more to do, and a lot more to discover about getting the most out of my new AI cast, but this is a great start.</p><p>You can play the in-progress demo at <a href="https://deadline-ai-server.fly.dev/">https://deadline-ai-server.fly.dev/</a>.</p><p>If you would like to get updates on the progress, you can follow me here or on twitter as <a href="https://twitter.com/jschomay">jschomay</a>.</p><p>And lastly, here are the previous articles in this dev diary series up to now:</p><p><a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-1-planning-and-design-57e6a86ffaf0">Part 1: Planning and Design</a><br><a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-2-first-words-baf250e0307f">Part 2: First Words</a><br><a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-3-integration-and-analysis-826172445e10">Part 3: Integration and Analysis</a><br><a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-3-5-reflections-trust-and-pigeons-a4db887e1ea1">Part 3.5 (bonus): Reflections, Trust, and Pigeons</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2c9f4f1084c3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Adding Inworld AI Characters to an Existing Game, Part 3.5: Reflections, Trust, and Pigeons]]></title>
            <link>https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-3-5-reflections-trust-and-pigeons-a4db887e1ea1?source=rss-ab419a65f208------2</link>
            <guid isPermaLink="false">https://medium.com/p/a4db887e1ea1</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[pigeons]]></category>
            <category><![CDATA[narrative]]></category>
            <category><![CDATA[storytelling]]></category>
            <category><![CDATA[game-design]]></category>
            <dc:creator><![CDATA[Jeff Schomay]]></dc:creator>
            <pubDate>Sat, 08 Apr 2023 23:32:31 GMT</pubDate>
            <atom:updated>2023-04-08T23:32:31.631Z</atom:updated>
            <content:encoded><![CDATA[<h4>What I learned from my players and my characters</h4><blockquote>“Giving infinite dialog agency to the player in a controlled story is a frightening thing!”</blockquote><p>Last week I released a <a href="https://deadline-ai-server.fly.dev/">demo</a> of the new AI characters in my narrative game. One of my favorite things to do now is reading the (anonymized) transcripts from players interacting with these AI characters.</p><p>In my <a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-3-integration-and-analysis-826172445e10">previous article</a>, I had left feeling lukewarm about this experiment. Specifically, I was concerned with the free-form AI chat detracting from the tightly scripted main narrative and game experience. However, I’ve had a chance to reflect and get feedback, and now I’m feeling more enthusiastic. In this bonus developer diary I want to share what I learned and how my view changed.</p><figure><img alt="A pigeon in a subway causing chaos" src="https://cdn-images-1.medium.com/max/1024/1*nBPJzeeRQUbO1dN7gv8OcA.png" /><figcaption>According to Carl</figcaption></figure><p>In one interaction, the player asked Carl (the friendly subway station coffee vendor) if he had seen anything strange going on. Carl said he hadn’t lately, but one time he saw a pigeon fly into the subway and cause a bunch of chaos! What a lovely little anecdote for the player.</p><p>More importantly, it is one I never would have written myself. And Carl probably won’t share it with any other players again. It was unique for that particular player, and even better, that player collaborated in telling the story, making it more personal and meaningful. That kind of interaction is an elusive holy grail in content-driven game design. And that is the true promise and power that generative AI offers.</p><p>In another case, a more devious player was having fun trying to “break” Carl. Despite his protests, they eventually managed to coerce him into serving them 100 cups of coffee! Then, they faked a heart attack and poor Carl had to improvise in the subway setting to seek emergency assistance (which he did rather well). It was quite tense for a bit, and the player and Carl managed to laugh about afterwards.</p><p>Only one player was directly inappropriate. I was so proud of Carl, who, instead of the default AI safety-control response (usually like, “I’m uncomfortable talking about that, let’s talk about something else”), managed to stay in-character to defuse and redirect the conversation, even across multiple uncomfortable interactions.</p><p>I never tire of watching Snarky Coffee Girl relentlessly troll players, no matter how hard they try to outdo her. Once, she slipped up in some kind of AI safety over-correction bug that made her announce that she was a “virtual assistant.” When the player asked if she really just admitted to being a virtual assistant, she just rolled with it, saying “No, I’m a figment of your imagination. Of course, I’m a virtual assistant. Do try to keep up, will ya?” I was laughing out loud, and I imagine the player was too.</p><p>I had such a good time reading the transcripts, that I was reassured the players must have been enjoying the experience just as much. These example interactions, and many others, showed me that <strong>AI characters are masters of improv.</strong> Watching my characters, out in the wild, managing on their own, taught me an important lesson.</p><h4>Trust</h4><p>Although my characters sometimes faltered, or said the wrong things, or spoke nonsense, for the most part they did just fine, and frequently pleasantly surprised me. As a game designer in this new world of generative AI, my job is to design the characters as well as possible, and then trust that they will help carry the game.</p><p>It’s almost impossible not to anthropomorphize my characters. I know that they are nothing more then LLMs and prompt engineering, and each interaction is running binary data through a model to complete a string of text word by word. But none of that really matters. Functionally, I need to trust them, in the very same way I would trust trained improv actors that I hire to entertain an audience.</p><p>I also learned to trust the players. At first I didn’t know what players would say to my characters. Now I do. Most played the part of the protagonist character, conversing about what they have seen and experienced so far. Many simply asked, “How’s it going?” A few were more interested in peering behind the curtain and exploring the mechanical workings and limits.</p><p>In the end, I realized that it didn’t matter what players said. The players that role-played likely had a great time. The players that probed the dark corners, trying to break things, probably had a good time too. Ultimately, with or without AI, the game happens, not on the screen, and not in the script, but in the player’s mind, and they will enjoy it however they want.</p><h4>Prompting… the player</h4><p>While I am pleased that players are enjoying their experience, I still want to use the AI characters as optimally as possible. I have some directions to pursue to achieve that.</p><p>This whole time I’ve been concerned with trying to make better AI character prompts. And while that is important, it is equally, or even more important to prompt the player. Everything in a game is a cue, a little nudge, to spark the player’s curiosity, to go check it out, and to progress in the direction you want them to go. So how do you prompt the player?</p><p>One way is with rich content. In a primarily text based game like mine, the static content creates a framework of context for the player to reference. In my case, this includes all written text, but also, everything available to discover in a location, like my broken vending machine. I was somewhat surprised to find quite a few players asking Carl about it. Even though I didn’t design Carl to know that the vending machine was there, the context of the conversation was sufficient for him to reply along the lines of how the subway maintenance has been falling behind. This is perfect, because it foreshadows everything that is about to happen to the player. (The broken vending machine is a metaphor? Wow! Thanks, Carl!) Which gets to the next point.</p><p>The second important place to prompt the player is directly in the conversations they have with the AI characters. Their responses should make the player even more curious, and even more eager to press forward. But, how do you do that when you don’t directly control what the characters will say. How do you trust your characters?</p><h4>Giving the tools to thrive</h4><p>It is a very strange thing to trust an algorithm. I trust Google Maps, and to some extent, self driving cars, but this is different. This is a facsimile of a living, thinking, intelligent entity. You have no way of knowing how it might react to a given situation, yet its response is so important. How do you trust something that is, by design, nondeterministic?</p><figure><img alt="The intersection of 1) interesting characters, 2) good character design, and 3) shared knowledge" src="https://cdn-images-1.medium.com/max/1024/1*_vMYOiE3iiuYW48RYR2Ckw.png" /><figcaption>How to trust your characters</figcaption></figure><p>I found that the best AI characters came from the most interesting characters to begin with. Snarky Coffee Girl has minimal design behind her, but she plays her role superbly. I tried to make another commuter in the station. She is pretty bland, and serves more to show the player that the protagonist prefers to keep to himself. When I tried to add AI for her, she was really generic and even annoying. So it goes to show how even minor characters should be interesting. This is true in game design as well as in TV and film. As <a href="https://savethecat.com/products/books/save-the-cat-the-last-book-on-screenwriting-youll-ever-need">beloved screenwriting guru Blake Snyder</a> advises, “give every character a limp and an eye patch.”</p><p>Once you have an interesting character, Inworld offers a whole suite of tools to direct how they respond. I’m still figuring out how to best use it all. Beyond turning all of the dials and knobs to get the right personality and tone of voice, I’m starting to learn that orienting each character’s place in the game is critical. For each character, ask why they are in the game and what they bring to it. You can probably tell that I already had a good answer to these questions when I initially designed the scripted interactions and narrative. Now, I’m learning to pull that into the Inworld system too. For example, I tweaked Carl to warn lingering players not to miss their train, and to encourage them to stand up for themselves if they mentioned their boss. As more events start to unfold, other characters should have opportunities to worry about (and therefore talk about) very in-game concerns. In some cases, this might require carving back some of the scripted narrative to leave some unanswered questions for the player to ask about.</p><p>Of course, your characters can’t talk about in-game things if they don’t know about them. I mentioned the broken vending machine earlier, and how Carl didn’t know about it. I plan to go back and add everything observable in the scene to the characters’ knowledge. It reminds me of how you would have to add <a href="https://intfiction.org/t/craft-parser-if-and-nouns/57451/2">all of the nouns in old school interactive fiction</a> games to the world model to avoid embarrassing errors when the player asks about something they read about, but the author forgot to define it. Luckily, generative AI will just spout out what ever comes naturally. It’s still surprising to me to never see a character say, “Uh, I don’t know about that.” This trait is pretty sweet, but you lose control of that opportunity to prompt the player if the character doesn’t have the appropriate context. Luckily, Inworld offers a number of ways to make your characters aware of things.</p><p>For each character, you can define common knowledge, private knowledge, and scene events. I’m still working out what combination of these work best, but the idea is that character conversations will be more rooted in the game world. For example, I look forward to seeing conversations with Carl where he brings up fun anecdotes about the broken vending machine in the corner. If the player hasn’t checked it out yet, they likely will after Carl mentions it. Better yet, maybe I hint to Carl that it broke because the Skater Dude was trying to steal from it. The protagonist character mentions earlier how much he hates rule-breakers like Skater Dude (who plays a big role later in the game), so this would be great contextual fodder for Carl to draw from.</p><p>And the rest is trust.</p><h4>Next</h4><p>In the next article, I’ll get back to the scheduled plan of more deeply exploring ways to successfully integrate AI characters into games. And in the meantime, I’ll be improving and adding characters to my game!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a4db887e1ea1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Adding Inworld AI Characters to an Existing Game, Part 3: Integration and Analysis]]></title>
            <link>https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-3-integration-and-analysis-826172445e10?source=rss-ab419a65f208------2</link>
            <guid isPermaLink="false">https://medium.com/p/826172445e10</guid>
            <category><![CDATA[game-dev-update]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Jeff Schomay]]></dc:creator>
            <pubDate>Sat, 01 Apr 2023 20:57:29 GMT</pubDate>
            <atom:updated>2023-04-01T20:57:29.371Z</atom:updated>
            <content:encoded><![CDATA[<h4>AI is in my game now, but is it better?</h4><p>My game now has live AI characters in it. Interacting with these characters directly in the game is novel, fun, and even surprising.</p><p>But I can’t help feeling like something is lacking. Do the AI characters add to the game, or detract from it? Or are they simply inconsequential?</p><p>In this part of the <a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-1-planning-and-design-57e6a86ffaf0">series on adding AI to an existing game</a>, I share the experience of integrating Inworld characters into my tech stack, and I consider if the result was a success or not, and if it could be better.</p><figure><img alt="“Well, aren’t you just a master of observation?” My snarky barista trolling me." src="https://cdn-images-1.medium.com/max/1024/1*RR3CHi84dPmS0nct4jSaiA.png" /><figcaption>Getting trolled by own characters (“Snarky coffee girl” in this case)</figcaption></figure><h3>Getting it all connected</h3><p>I have already created a few <a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-2-first-words-baf250e0307f">Inworld characters for my game</a>. And I have the existing <a href="https://github.com/jschomay/subway-game">game code</a>. In this case, the game is a browser-based game, so all of the code is in JavaScript.</p><h4>Using the Inworld SDK</h4><p>I followed the <a href="https://docs.inworld.ai/docs/tutorial-integrations/web/api">Inworld docs</a> for the web SDK. These required setting up an additional node.js server to securely create session tokens. Other than that, the basic process involved:</p><ul><li>Connecting to the Inworld server</li><li>Activating the correct scene or character</li><li>Sending out messages</li><li>Receiving the responses</li></ul><p>I had a little confusion figuring out how to correctly use scenes, and how they effect switching between characters, but it wasn’t too hard, and the <a href="https://discord.com/invite/2jGPwV8g3b">Inworld Discord</a> support channel is very active and helpful.</p><p>Although the SDK includes audio chat and character emotion states, I did not use those advanced features since this game is mostly text based and I wanted to keep that interface.</p><h4>Integrating with the game engine</h4><p>As for integrating with my custom engine, I was easily able to add the Inworld character IDs into my engine’s stateful “manifest” of game entities. Not only did this make it simple to “opt-in” to adding AI interactions to some NPCs and not others, it also made it possible to enable or disable this feature on a per character basis at different points in the story.</p><p>It also made it simple to swap out IDs during game play as well, which would be helpful for connecting to different “versions” of the same character with different configurations. Or, in my case, the same game entity, “Coffee cart,” has different baristas at different times, so I could swap IDs for these different characters without needing a unique coffee cart entity for each barista.</p><p>Inworld has a concept of character goal triggers, and scene triggers. I used these in an attempt to make the characters change how they related to me as the story progressed. I got a less dramatic result than I desired, but I think it could be improved by tweaking my character prompts in Inworld’s studio.</p><h4>Extending the UI</h4><p>I was able to reuse the existing game views for the additional AI interactions, which not only made integration simpler, but also kept a holistic feeling to the interactions, which was one of my original goals. Both the chat interface and AI responses appear in the same UI as all other narrative content, helping the chat feel like part of the main story.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XpbnSnd9K0PLfs_Y3vZDRQ.png" /></figure><p>I had initially considered a <a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-1-planning-and-design-57e6a86ffaf0">few approaches</a> on where to add AI interaction. I went with the option of adding a “Talk more…” button after the scripted content finished, which enters a free-form chat-like interface with AI generated responses.</p><p>This option is only available on certain characters and at certain times that fit with the story. For example, the protagonist mentions that he doesn’t like to talk to the other morning commuters, so interacting with one of them doesn’t offer the “Talk more” option.</p><h3>But is it better?</h3><p>One of my main goals in this experiment was to have the addition of AI make the game and story a better experience without detracting from the original curated content.</p><blockquote>Ultimately, the goal is to make the world feel more alive, but also more interesting and deep, and to add tension to the story.</blockquote><p>The characters did indeed feel more “alive.” I must admit that I felt excitement from the unexpected responses in some of my conversations, like when the “snarky coffee girl” trolled me, no matter how hard I tried to get into her good graces.</p><p>I definitely felt compelled to forget about the rest of the game for a while, and just keep talking with snarky coffee girl. But I wonder if the novelty would wear off. And worse, I worry that talking with AI characters can become a meta-game that distracts from the actual game experience.</p><p>Or perhaps, stopping to have a chat <em>is</em> part of the game play. I think this depends greatly on the type of game, and might not work well in my existing game that already has a strong narrative.</p><p>This is why I am left with a nagging feeling that I haven’t reached the desired experience. I don’t think this is a shortcoming of Inworld at all. But I think that in this case, I’ll have to get more clever with how I use their tooling to support that narrative that already exists in my game.</p><blockquote>I need to craft my AI characters better, to make their responses prompt the player to feel the emotions and motivations that push the game and story forward.</blockquote><p>When done well, I think AI characters can do this more successfully than scripted characters since they are, by design, reactive to a player’s personal experience.</p><h4>Play-testing</h4><p>Part of the challenge is that I do not really know what players will say to my characters. When I played, I role-played the protagonist character very rigidly, and measured the character responses against the story as I know it is supposed to unfold.</p><p>But other players will interact very differently. I need to play-test to see what conversations happen, and design around that. It may be possible that I’ve included AI in the wrong places, or maybe free-chat is the wrong mechanic. For example, it might be better to send background prompts based on the player’s actions and world state and then surfacing the character responses within the scripted narrative some how.</p><p>I invite you try out the demo. At this time, only a few characters in the intro have AI.</p><blockquote><a href="https://deadline-ai-server.fly.dev/">Play-test “Deadline” with the initial AI characters</a></blockquote><p>If you have feedback, you can use this post’s comments or <a href="https://twitter.com/jschomay">tweet me</a>. Let me know what you think of the AI interactions, and if you’d like to see more of them, or see them work differently. Just a note, I’ll be able to see the content of your chats, but no identifying information.</p><h3>What’s next?</h3><p>In the next post I will dig deeper into exploring ways that AI can enhance game-play beyond just chat bots, and how I might apply that to this game.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=826172445e10" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Adding Inworld AI Characters to an Existing Game, Part 2: First Words]]></title>
            <link>https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-2-first-words-baf250e0307f?source=rss-ab419a65f208------2</link>
            <guid isPermaLink="false">https://medium.com/p/baf250e0307f</guid>
            <category><![CDATA[game-dev-story]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[npc]]></category>
            <category><![CDATA[writing]]></category>
            <dc:creator><![CDATA[Jeff Schomay]]></dc:creator>
            <pubDate>Mon, 13 Mar 2023 00:14:59 GMT</pubDate>
            <atom:updated>2023-03-13T00:14:59.103Z</atom:updated>
            <content:encoded><![CDATA[<p>After considering some <a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-1-planning-and-design-57e6a86ffaf0">implications of adding AI NPC’s to an existing game</a> in my previous article, I have taken the first timid steps into creating my first character in <a href="http://inworld.ai/">Inworld</a>. I want to share the experience, because it was strangely visceral, magical, and unexpected at the same time and made me think things I didn’t realize would come up.</p><p>This isn’t my first foray into AI character creation. I’ve played around at a low level with building templated prompts by hand to feed to GPT3 in order to get a rich and meaningful narration of a character action from player input. But Inworld has a very nice suite of tools for defining your character and its world that feels more like character building and world building than LLM prompt engineering.</p><p>I want to be clear that what I have done so far is extremely shallow, only the bare minimum of character creation, and the very beginning of what I expect will be a long road of tweaking and testing. Inworld keeps rolling out new advanced features that I can’t wait to dig into, but even at this early state, my experience felt worth sharing, so here it is.</p><h4>Meet Carl</h4><p>The game that I am adding AI characters to is called “Deadline.” It is a narrative-heavy game set in a fictional subway system where players help the protagonist, Steve, get to work on time to deliver a job-defining presentation, except that on this day, the subway is going haywire and Steve’s world is falling apart around him.</p><p>You can <a href="https://enegames.itch.io/deadline">play the original version online</a> if you’d like.</p><figure><img alt="Carl’s first line" src="https://cdn-images-1.medium.com/max/788/1*VXYY6az6AxpUYi_04_RdMg.png" /><figcaption>Carl’s first (scripted) words</figcaption></figure><p>In the very first scene, the player finds a small handful of characters and items to interact with, including the “Coffee cart” where Carl works. Carl is likely to be one of the first characters that players interact with.</p><p>Carl plays a very small role in the game, serving as a way to introduce players to the UI and basic mechanics. They learn that sometimes interacting with things give you inventory items (the ubiquitous to-go cup of hot coffee in this case). But more importantly, they get a very subtle introduction to Steve, the character they will be guiding around the subway for the next hour or so. They get a hint that Steve is a man of routine, and an idea of his financial position in the world, and that he likes order, rules, and efficiency.</p><p>Carl also introduces the plot and provides some foreshadowing. Over the course of the first few days that make up the introduction / tutorial scene of the game, Carl is a regular, and his energizing brew dispensing makes a natural conversation starter around Steve’s growing loss of sleep and mounting stress as he prepares for his high-stakes and overwhelming presentation levied by his arrogant boss. And then, (minor spoiler here), just when Steve needs Carl’s friendly encouragement the most, he’s gone, replaced by an irreverent teen who could care less about Steve’s troubles. If the player clicks on the coffee cart again after this unpleasant exchange, Steve just says “I miss Carl.” And hopefully, so does the player, but there’s no time to lament, because Carl’s absence is only the first indicator of everything that is about to go wrong.</p><blockquote>“I miss Carl.”</blockquote><h4>Why Carl?</h4><p>I chose Carl as the first character to get the Inworld treatment, for a number of reasons. First and foremost, I wanted to start simple. Carl is a simple character with a restricted personality and motivations (which I later realized might have actually made things trickier) and he goes away after the first scene.</p><p>Also, Carl is the perfect tutorial character to get the player used to talking to AI characters with limitless AI generated content, since he shows up immediately, and doesn’t risk impacting the plot by going off script too much (which again I realized later was a false assumption).</p><p>Also, as far as Inworld’s internal state is concerned, Carl isn’t exposed to much knowledge in the knowledge base, and isn’t affected by goals or scene triggers or the other more advanced Inworld options. That said, I do intend to use Carl to experiment with these options later, like making him aware of which day of the week it is, and become more concerned and sympathetic over Steve’s growing loss of sleep.</p><h4>Meet Carl. Again.</h4><p>After filling out all of the basic information that Inform uses to personalize characters, Carl 2.0 was ready for his first words. While the ultimate plan is to integrate a UI in the game to converse with AI characters seamlessly along side of the scripted content, for now I just used Inworld’s in-studio character testing chat window.</p><blockquote>Carl is about to start talking for himself. Would he talk like how I imagined? Would I like him?</blockquote><p>But before I hit that button, I experienced a surreal thrill of anticipation. I was about to actually talk with a character that had only lived in my head for years. I had written every line of dialog he has ever said up to this point. And now he is about to start talking for himself. Would he talk like how I imagined? Would I like him? It was thrilling. And also a little frightening.</p><p>And there was one more important question: what should be my own first words to him? In a display of spectacular timidness and unoriginal thinking, I went with the first words I wrote for Steve so many years ago:</p><figure><img alt="Carl’s first AI generated words (“Sure thing Jeff”)" src="https://cdn-images-1.medium.com/max/569/1*5_NM0YgTjTBF0YhwbsIokw.png" /></figure><p>Wow.</p><p>How was it? It was both good and bad, magical and frustrating. But the first thing I noticed hit me like a brick. Do you see it?</p><blockquote>He didn’t say “Sure thing, Steve.” He said “Sure thing, <strong>Jeff</strong>!”</blockquote><p>I’m Jeff. Obviously, Inworld knows my user name and uses templated variables to inject the player’s name in the dialog. But Carl doesn’t know that! All these years I have almost heard Carl’s voice, saying “Sure thing, Steve,” but now Carl is talking to me. Carl’s first words were directed to <em>me</em>.</p><blockquote>Carl doesn’t live inside my game any more.</blockquote><p>OK, once I got past that, the second line is interesting too, because it came directly out of the example dialog that I gave to Inworld. Very in character, but not original. But then, Carl immediately overcompensated:</p><figure><img alt="Carl getting too creative, making up plot details about train schedule changes" src="https://cdn-images-1.medium.com/max/556/1*NmmkYK8RpBpvBZGnp8xI4Q.png" /><figcaption>Carl instantly getting too creative with his new found freedom</figcaption></figure><p>Here’s the thing, the train schedule most certainly did not change, and worse, later on the player learns that Steve’s train actually leaves at 6:30 AM and a ticking clock becomes a central piece of the plot as Steve races to get to work on time to give the presentation.</p><p>AI hallucinations like this can be nice for giving flavor and depth to the world, but detrimental in strongly plotted context. This is one of the biggest challenges in introducing generated content. In my earlier article I brought up this concern. I know that Inworld and the rest of the AI industry is working on issues like this, but for now we have to find ways to make it work.</p><p>I conversed a little more after that, but Carl got very chatty and, to my ear, sounded very “on the nose,” which is something I feel I notice in a lot of AI generated content. He also spouted some more untruths, saying, “This is the busiest station around,” when it actually is designed to be one of the more quiet stations away from the larger downtown ones that the player will arrive at next.</p><h4>Tweak Time</h4><p>While I wasn’t pleased with how “Carl” turned out, I was very aware that the real work was just beginning, the work of tweaking Carl’s textual DNA. Despite how far generative AI technology has come recently, we still are at a point where changing one word in a prompt can have drastic results, which is true in LLMs like Inworld’s models as well as other media like the contortions resorted to in Midjourney prompts (like this mouthful: “trending on artstation, high detail, high definition, high quality, photo-realistic, ultra realistic, hyper-realistic, 4k”).</p><p>For the second attempt, I removed the “knowledge of the train schedule” fact that I had first put in, and I changed a few of the personality and motivation fields to make Carl more terse and more likely to ask questions or engage in short small talk. There’s more to do, but the results are already better, and AI Carl sounds more like good ol’ Carl, though I’m still not sure what to say to him, or what players will want to say.</p><figure><img alt="More examples of Carl’s tweaked lines, which now sound like small talk with no hallucinations" src="https://cdn-images-1.medium.com/max/550/1*qf45iRdGQGAURoCIk7ojaA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*CFuK5Ikw8lLyrQJiozjgLw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*hEgjt5XHDKqAXZfFyV8zZw.png" /></figure><p>I have to admit, I didn’t quite get the joke, but I was amused nonetheless. This dialog mostly feels consistent with the picture I’ve had of Carl in my head all along.</p><h4>Takeaways and Next Steps</h4><p>I really enjoyed this experience of bringing my first character to life and hearing his first words. For me, the whole thing felt like as if you’ve ever imagined an upcoming conversation you were going to have, and played it over and over in your head, but then when it really happened it went nothing like what you imagined.</p><blockquote>Adding AI to an existing character feels like learning to let go of a child, and having faith they’ll do well on their own.</blockquote><p>Carl is just the beginning. I still have to integrate him into the game’s UI and experiment with the more advanced options. And then it will be time to pick a more challenging character, and go through this whole process again. To meet my other characters, again, for the first time.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=baf250e0307f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Adding Inworld AI Characters to an Existing Game, Part 1: Planning and Design]]></title>
            <link>https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-1-planning-and-design-57e6a86ffaf0?source=rss-ab419a65f208------2</link>
            <guid isPermaLink="false">https://medium.com/p/57e6a86ffaf0</guid>
            <category><![CDATA[game-development]]></category>
            <category><![CDATA[game-design]]></category>
            <category><![CDATA[narrative]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[npc]]></category>
            <dc:creator><![CDATA[Jeff Schomay]]></dc:creator>
            <pubDate>Mon, 27 Feb 2023 13:20:16 GMT</pubDate>
            <atom:updated>2023-04-20T00:22:37.224Z</atom:updated>
            <content:encoded><![CDATA[<p>As a game developer and narrative designer, I’m always looking for ways to make my games more engaging and immersive. Recently, I had an idea to take one of my old narrative games and add AI NPC characters to it using Inworld. In this developer diary, I’ll walk through my planning and design process for this experiment.</p><p>Part 1: Planning and Design<br><a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-2-first-words-baf250e0307f">Part 2: First Words</a><br><a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-3-integration-and-analysis-826172445e10">Part 3: Integration and Analysis</a><br><a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-3-5-reflections-trust-and-pigeons-a4db887e1ea1">Part 3.5 (bonus): Reflections, Trust, and Pigeons</a><br><a href="https://medium.com/@jschomay/adding-inworld-ai-characters-to-an-existing-game-part-4-going-deeper-2c9f4f1084c3">Part 4: Going Deeper</a></p><h3>What is Inworld?</h3><p><a href="https://www.inworld.ai/">Inworld</a> is a platform that lets creators easily build AI-backed characters by providing a personality, goals, and a knowledge base. They offer integrations with game frameworks to add these characters to your game.</p><h3>What is “Deadline?”</h3><p>Two of my favorite movies are “<a href="https://www.imdb.com/title/tt0090095/">Subway</a>” and “<a href="https://www.imdb.com/title/tt0373981">Kontrol</a>.” A while<br>back, my friend told me that I should make a video game like those films.<br><a href="https://enegames.itch.io/deadline">“Deadline” is that game</a>.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FRYBcEXNLH7g%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DRYBcEXNLH7g&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FRYBcEXNLH7g%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/cffb2f69a97f89329fb2cef41c37e5ae/href">https://medium.com/media/cffb2f69a97f89329fb2cef41c37e5ae/href</a></iframe><p>“Deadline” is a narrative-heavy game with minimal visuals, map-based navigation, a central story to explore with a few side plots, and many oddball denizens of the subterranean labyrinth. In that way, it is a little bit like “Kentucky Route Zero.”</p><blockquote>The goal of this experiment is to make those oddball denizens into AI-controlled oddball denizens… without losing control over the plot!</blockquote><h3>Challenges of Adding AI into an Existing Game</h3><p>Making a non-linear narrative game is already difficult enough, but adding AI to it has a number of additional complications to consider.</p><h4>Generated vs. Scripted Content</h4><p>In “Deadline” the player can see a list of NPC characters and items in each scene, and can select one to interact with, such as the “Coffee cart” in the opening scene.</p><figure><img alt="UI of a station scene in “Deadline” showing NPC’s and items to interact with" src="https://cdn-images-1.medium.com/max/1024/1*eMB31BcUH8XIWlnT34QySg.png" /><figcaption>Items and NPCs in the first station in “Deadline” for the player to interact with</figcaption></figure><p>This brings up a context-aware dialog box that usually includes a description of what happens and some dialog, both for the player’s character and the NPC.</p><figure><img alt="Example of a dialog box in “Deadline”" src="https://cdn-images-1.medium.com/max/1024/1*JNohihNxGAgofzbzVr--sw.png" /><figcaption>Interacting with the “Coffee Cart” brings up this dialog box</figcaption></figure><p>With the power of generative AI, it would be nice to create AI characters that have enough information to allow the player ask the right questions to move forward in the game. This could make for a very unique and interesting experience, and puts the player more in the driver’s seat, by making them think harder about what to ask, similar to old-school command line text adventure games.</p><p>On the other hand, I have lots of fun scripted content that I’d like players to come across. Also, pure generated content makes it technically much harder to trigger narrative rules that change the world state or unlock more content.</p><p>I could imagine a hybrid approach where narrative-significant interactions are scripted, while innocuous interactions use generated content so players can ask their own questions. But giving the player free reign some of the time, but not all of the time might actually make the game feel more railroaded. Plus some of my favorite scripted dialog moments happen in the minor interactions.</p><p>Another idea is on every interaction to let the player choose between advancing the story via the scripted content, or having an unscripted side conversation with an NPC. But this approach may also break the flow of the game by essentially having two ways to play it.</p><p>Furthermore, with a hybrid approach I’d have to carefully make the AI generated content match the voice and tone of the scripted content, which might not always be easy.</p><h4>UI</h4><p>The UI will need to change to accommodate the approach taken above. I can imagine a few approaches:</p><ul><li>Placing an input field at the end of each dialog box’s content for the player to type in prompts if they want to dig deeper.</li><li>Offering a choice between one (or more) truncated scripted options and an input field for the player to supply their own prompt, and changing what displays next based on which option they chose.</li><li>Adding a “chat” icon next to each NPC listed in the scene to activate free-form chat, and also keeping the original UI of clicking on the NPC name to trigger the scripted content.</li></ul><p>Of these, I like the first option best because it feels the least intrusive, doesn’t preclude scripted content, and allows the player to interact further in a more free-form manner at a moment when they have the most new information.</p><h4>Consistency</h4><p>If one NPC is AI-backed, do all NPCs need that functionality? Does it break the fourth wall too much if players wonder why they can interact with one NPC but not another?</p><p>Adding AI to every character gets expensive and hard to manage. It also seems undesirable in many cases. For example, in the first scene, the player can say a quick hello to another character waiting for the same train to arrive. If the character tries to interact with that character again, the player’s character says “I think one hello is enough when talking to complete strangers.” This sets some bounds to the game, but more importantly conveys information about the player character’s personality. Allowing the player to then enter free-form chat mode with this character would break the constructs of the story.</p><p>I am hoping that selectively applying AI will feel natural without causing AI vs non-AI characters stick out like sore thumbs. I expect the choice of UI could help smooth this inconsistency and the narrative design could also make the presence or absence of free-form chat intuitive.</p><h4>Staying “on Script”</h4><p>One of the challenges of adding AI characters, is making sure they stick to talking about what they should know. Ideally they will be able to drive the plot forward and add texture and even foreshadowing. But if they go off the rails (sorry, couldn’t pass up on the pun), it could cause a lot of confusion for the player. Also, the player might ask things that don’t make sense, either on purpose or by accident, so I need a good strategy for dealing with those cases.</p><blockquote>Ultimately, the goal with adding AI characters is to make the world feel more alive, but also more interesting and deep, and to add tension to the story.</blockquote><p>Inworld should be pretty good at keeping NPCs on script, but I have a few questions. For example, is there a way to change a character’s knowledge base over time? And if so, does each character “instance” across games have its own knowledge base? I need to explore this further, but I think each character might be a single entity that has a single source of knowledge, regardless of how many people are interacting with it. If that is the case, I might need to get clever with adding prompt prefixes to what the player writes to “direct” the character, though I’m not sure how well that would work. I think this is one of the big challenges game developers face when introducing AI that can be unpredictable.</p><h3>Conclusions and Next Steps</h3><p>This experiment presents a unique set of challenges and unknowns, but I’m excited to dive in and see what happens. I’ll do some more research and pick a small spot to start with, perhaps Carl the coffee cart guy in the first scene. In the next part, I’ll cover some of the more technical aspects of integrating Inworld AI characters into my existing game framework.</p><p>Disclaimer: ChatGPT helped turn my rough outline into prose, which I then edited further into this final form.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=57e6a86ffaf0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[elm-paginate: An API Design Post Mortem]]></title>
            <link>https://medium.com/@jschomay/elm-paginate-an-api-design-post-mortem-13bc9b53b2a9?source=rss-ab419a65f208------2</link>
            <guid isPermaLink="false">https://medium.com/p/13bc9b53b2a9</guid>
            <category><![CDATA[api]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[functional-programming]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[elm]]></category>
            <dc:creator><![CDATA[Jeff Schomay]]></dc:creator>
            <pubDate>Sun, 12 Mar 2017 21:32:01 GMT</pubDate>
            <atom:updated>2017-04-07T01:17:47.106Z</atom:updated>
            <content:encoded><![CDATA[<p>TL;DR — I made an Elm library called <a href="http://package.elm-lang.org/packages/jschomay/elm-paginate/latest">elm-paginate</a> to encapsulate pagination concerns from the rest of your app. This article talks about some of the considerations and challenges I faced in creating a clean and safe API, and my choices in how I overcame them. Here is a fully-featured <a href="https://jschomay.github.io/elm-paginate/">demo</a> of the library, and the demo’s <a href="https://github.com/jschomay/elm-paginate/blob/master/src/Example.elm">source</a>. If you need pagination in Elm, try it out!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rOTwSqc86xmrbx7xJgESAA.png" /></figure><p>Pagination is not a particularly tricky feature, but it does have a few sticky points to consider. First and foremost, pagination is a view/UX concern to make long lists easier to browse. The data layer has no need to paginate, and in fact, pagination can get in the way of data manipulation.</p><p>Additionally, pagination behavior isn’t always clear:</p><ol><li>What happens if you delete an item from a paginated list? Does the page displaying that item have one less item than all of the other pages, or do items from the following pages “slide” in to fill the gap? The later would maintain consistent page sizes, but may need to remove a page if the last page has only one item.</li><li>Similarly, what happens if you insert an item in the middle of a list, does it “push” the other items forward by one, creating an additional page if needed?</li><li>What if you are on the last page and you delete all of the items on that page? Do you “jump” to the previous page?</li></ol><p>In my implementation, I chose what seemed to be a sensible standard, taking what would be the result of applying any transformation to the list and “re-paginating,” maintaining the current page whenever possible.</p><h4>Take 1: Applying pagination directly</h4><p>A paginated list is really just a segmented or partitioned list. So the simplest approach is to partition your list by the desired page size and save it in your model as a List (List a).</p><p>But how do you know which segment to show as the “active” page? You could store the current page index as a separate field in your model, but then you need to keep that in sync with your list if the pagination changes, and you open yourself to the possible error of having an index outside the bounds of your list’s length.</p><p>This is a known problem with a known solution — a <a href="http://package.elm-lang.org/packages/wernerdegroot/listzipper/latest">zipper list</a>, which binds the concept of a “focused” element with the rest of the list, and provides a way to traverse the list without going out of bounds (note how next returns a Maybe.)</p><p>However, zipper lists can’t be empty, so if you need that option (say you are loading your data by http) you’ll need a Maybe (Zipper (List a)), which is pretty ugly, messy to manipulate, and error-prone. Imagine how you would map or filter that, or worse, update, insert or remove elements. You would need to manually shuffle items around to maintain a consistent page size. You could make a function that takes a transformation function, does a toList, applies the function, then re-paginates, but that is a lot of unnecessary work and doesn’t make for a great API either.</p><p>All of that for a view concern that the data layer doesn’t even care about! Perhaps it would be better to store the list unadulterated, and do the pagination dynamically as you pass the list into the view, instead of storing the stateful pagination data.</p><h4>Take 2: Separation of concerns with a “pager” <a href="http://package.elm-lang.org/packages/jschomay/elm-paginate/1.1.0">(1.1.0)</a></h4><p>The dynamic approach seems much cleaner, but requires storing the current page index, and all of the problems mentioned with that.</p><p>I encapsulated those issues by creating a new type to store the stateful page information as a Pager. You would store the pager in your model along side of your list. As long as your pager knew the length of your list and the desired page size, it could calculate the number of pages and track the current page’s index, offering helpful navigation functions like next, prev, first and last. I even made my own numeric equivalent of a zipper list for Int’s to ensure the index would always be between 1 and the total number of pages to avoid “out of bounds” errors (see <a href="http://package.elm-lang.org/packages/jschomay/elm-bounded-number/latest">elm-bounded-number</a>).</p><p>This way, you could treat your list as normal, and when you were ready for pagination you could ask the pager for the correct “slice” of your list to display, along with a pager-view to show the other available pages.</p><p>The only problem was that you had to manually keep the pager in sync with your list. If you performed an operation that changed the length of the list, you needed to update the pager too, which was tedious, messy, and opened the door to out-of-sync errors again, and still mixed the concerns of your business logic with pagination mechanics.</p><h4>Take 3: Total encapsulation<a href="http://package.elm-lang.org/packages/jschomay/elm-paginate/2.0.0"> (2.0.0)</a></h4><p>The final piece to fully separating pagination from the rest of the app was to manage this syncing problem automatically by expanding the encapsulation a little further.</p><p>I got rid of the Pager type and changed to a PaginatedList a type wrapping not only the pagination information, but also the list. Now the model only stores the paginated list, which keeps the pagination data in sync automatically when the list changes. The question now becomes, how do you transform the list, since it is “hidden” in the opaque PaginatedList type?</p><p>I solved this by adding a map function, which takes a List a -&gt; List a function and applies it to the list, returning a new PaginatedList a. Although this requires “lifting” over an additional layer of structure whenever you want to transform your list, it is a standard pattern and very clean with minimal impact to your calling code, especially when compared to the previous approach, plus it guarantees that the pagination data is always accurate, eradicating that source of bugs. You can use map to update the paginated list in your model, or dynamically when passing it to the view, which is useful for sorting and filtering prior to display.</p><p>With these changes, the library covers all of the pagination concerns with an API and usability I approve of. It should be ready for use in the wild.</p><h4>Parametricity?</h4><p>The only problem with PaginatedList a is that it only works with List&#39;s. What if someone wants to use it with something else, like an Array or Dict or even something crazy, like a LazyList? Or more practically, users will likely be wrapping their data in an opaque type.</p><p>I could modify the library to accept any data structure (Paginate.Custom.Paginated a) as long as the passed in data structure had a length and slice function. This is where typeclasses would be useful, but Elm does not support them for various reasons. The alternative is to require passing in those two functions as arguments to some of my API functions (I could only require them on initialization and store them in the model, but in general you wan to avoid storing non-seralizable data like functions in your model). This would work, but it gets a little messy, and makes me wonder if the benefits are worth it.</p><p>I’ve gone back and forth on this, and so far I have decided against it. Pagination by definition implies an order, which theList type seems most appropriate for compared toArray&#39;s and Dict&#39;s. As cool as it would be to paginate an infinite lazy sequence, I’m not sure how practical that would be. Opaque types are the biggest concern, but the opaque type can simple wrap a paginated list, which works nicely, as its own API would wrap the required maping discussed earlier. I realize users often useDict&#39;s inside their opaque types for performant lookups and simpler updates, but if pagination is a requirement, perhaps a List is better anyway, unless performance becomes an issue.</p><p>Perhaps in the future I’ll add parametricity if there is demand, but for now I am happy with the library as it is.</p><p>UPDATE: I went a head and added pagination for custom types. If you can supply a length and slice function, you can paginate your custom data type. I made a fun little demo of this by <a href="https://jschomay.github.io/paginating-fibonacci/">paginating infinite streams</a>.</p><p>And that is how I designed the API for <a href="http://package.elm-lang.org/packages/jschomay/elm-paginate/latest">elm-paginate</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=13bc9b53b2a9" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>