Back in 2016 when I was a sophomore in college I attended a local hackathon and had a funny idea

Editor’s note: Besides being an excellent article on the acceleration of innovation using AI, this is also a reminder to all of us that developers love to explore and learn. In AI Enterprise Engineering, our hope is that you see how to excite your developers while creating the guardrails that Steven highlights: proper architectural thinking, proper security, focusing on the best user experience while adhering to enterprise policy. Enjoy!!

What if there were an app like Tinder where the user was looking for chicken tenders instead of their next close friend? The idea was fun, but wasn’t quite compelling enough to devote a lot of attention to. Our team (shout-out to Wade and Matt) built out a barely functioning prototype, drove back to our home college and didn’t think about it again for a decade.

Flash forward to January 2026, I’m in the middle of the holiday season and looking for something to think about between late night bottles with my few-months-old daughter when I see people posting about the power of the latest AI dev tools. I realized this was the perfect opportunity to get hands-on experience in some of the AI tools and take advantage of my skillsets, a dev background who’s relatively fresh into product management. Then came the hard part, what is the right thing to build? Well, Tendrrr was the perfect fit! I had the stub of the idea and knew it was feasible, so I whipped open the Macbook and got to building.

What I came out with on the other side was not just a published app in the app store in less than a month, but a realization of just how much the world of software development has changed since my last role in the job in 2024. I also have come to realize the incredibly valuable experience these tools have given me in my role as PM in the AI space: this stuff is really impressive! Come along with me as I break down what I built, what I learned, and how even as a PM I learned valuable skills on how AI is building the future (and where there’s still a ways to go).

The Idea (5 Minutes)

The concept was simple: a Tinder-style swipe interface for discovering nearby chicken tender restaurants. Swipe right to save a spot to your favorites. Swipe left to skip. That’s it.

Part parody, part genuinely useful. The “hot tenders in your area” energy was intentional — I wanted something that would make people laugh but also actually could be used.

PM Insight #1: Constraints make decisions easier. By picking a single, hyper-specific use case (chicken tenders, not “food”), I eliminated hundreds of feature debates before they started. Every design choice had a clear filter: does this help someone find chicken tenders faster?

The Architecture Phase (Days 1–4)

My first instinct was to just start coding. But I’d been a PM long enough to know that’s how you end up with a 600-line Object that does everything poorly.

Instead, I started with a Product Decision Record: a lightweight doc capturing:

• Core user flow (discover → swipe → save → navigate)
• Technical decisions (SwiftUI, local-first data, Yelp API)
• What’s in scope vs. out of scope

Then I sketched an architecture before writing any code. The initial codebase had everything crammed into one file — business logic mixed with UI, no separation of concerns. I refactored into a clean MVVM pattern with distinct layers: Models, Services, Repositories, and Features.

The result of this step was a simple markdown PDR that drove development for Tendrrrr via Claude’s project feature to give context. This was relatively bare-bones, but allowed for me to get prompt creation tips directly from the system on how to best get what I wanted from the AI at each step.

PM Insight #2: Technical debt is a product decision. My “quick prototype” would have worked fine for a demo. But I wanted to ship a real product and learn what that takes. The extra day spent on architecture saved me at least a week of debugging later.

The API Decision (Day 5)

I needed restaurant data. The options:

• MapKit (Apple): Free but limited — no photos, no reviews, basic metadata
• Yelp Fusion: Rich data, great images, 5,000 free API calls per day
• Google Places: Most comprehensive, but expensive and overkill

I went with Yelp. The free tier was more than enough for launch, and the image quality made the swipe experience feel polished. MapKit stayed as a fallback.

PM Insight #3: Free tiers exist for a reason. For a side project validating a concept, I didn’t need the enterprise plan. Yelp’s 5,000 daily calls would support thousands of users before I’d need to worry about scaling. Build for where you are, not where you hope to be.

The Security Oops (Day 10)

I pushed my code to GitHub. Public repo, nice README, felt good about sharing my work.

Then I realized: my Yelp API key was hardcoded in Config.swift and visible to anyone who looked.

The fix:

1. Immediately rotated the API key (the old one was compromised)
2. Added Config.swift to .gitignore
3. Created a Config.swift.example template for anyone cloning the repo
4. Made the main repo private

Since GitHub’s free tier doesn’t support Pages on private repos, I created a separate public repo just for the support site and privacy policy. More work, but no monthly cost.

PM Insight #4: Security isn’t a feature — it’s a baseline. In a professional setting, there’d be a security review. Solo, I had to catch my own mistakes. Build security into your workflow: never commit credentials, always use environment variables or config files that stay local.

The App Store Gauntlet (Days 12–18)

Submitting to the App Store for the first time felt like filling out tax forms in a foreign language. Here’s what actually mattered:

Required before you can submit:

• Apple Developer account ($99/year — no way around it)
• Privacy policy URL (I used a free generator and hosted on GitHub Pages)
• App icon (1024×1024, no transparency)
• Screenshots for multiple device sizes

Things that seemed easy but weren’t automatic:

• Age rating questionnaire (took 5 minutes but was mandatory)
• Support URL (pointed to my GitHub Pages support site)
• Clear description of what data you collect

Things I worried about that didn’t matter:

• Fancy app preview video (didn’t need one)
• Elaborate onboarding (kept it simple)
• Perfect copy (they approved my first draft)

I was able to leverage AI to build the required pages for the app quickly without needing to start what felt like a full project within a project:

PM Insight #5: The App Store review process is predictable if you follow the rules. Apple publishes their guidelines. Read them. Most rejections happen because people skip steps, not because Apple is capricious. My submission was approved in under 48 hours.

The Monetization Question (Ongoing)

For a novelty app with maybe a few hundred users, ads would generate about $3–12/month after integrating an SDK, handling consent flows, and dealing with review headaches. Not worth it.

Instead, I added a simple tip jar — three tiers ($0.99, $4.99, $9.99) using StoreKit 2. Non-invasive, tucked away in Settings, no guilt-tripping copy. If people love the app, they can support it. If not, no friction.

PM Insight #6: Monetization should match your goals. My goal wasn’t revenue — it was learning. A tip jar lets fans contribute without compromising the experience. If the app takes off, I can revisit monetization with real data.

What I’d Do Differently

Draft first: In my other projects I’m working on now, I’ve realized a paradigm shift for how to manage the product. I’d been focused on getting user interviews and product concept feedback early, but in this era of AI development speed, why not build out the prototype and get feedback on that early on. It’s so much more powerful to have something tangible to discuss, even if it’s ugly.

Build the support infrastructure earlier. Privacy policy, support page, and App Store metadata felt like afterthoughts, but they’re required work that is less fun late in the game. Front-loading this work would have made submission smoother.

Test on real devices sooner. The simulator is great for iteration, but location services behave differently on actual hardware. I found bugs in the last 48 hours that should have surfaced earlier.

The Product Manager Perspective

Building Tendrrr reminded me why I moved from engineering to product in the first place: the interesting problems aren’t the technical ones for me — they’re about understanding what matters.

Should I add filters? Probably not — that’s scope creep for a novelty app. Should I support delivery ordering? Maybe — but not until I validate that people actually use the app. Should I spend a week on animations? No — ship the core experience first.

Every feature request I generated for myself went through the same filter: does this help someone find chicken tenders faster? If not, it went on the backlog.

The hardest part wasn’t code. It was deciding what not to build.

What’s Next

Tendrrr is live. I’m tracking basic metrics (downloads, daily active users) to see if anyone besides me wants this to exist. If there’s genuine interest, I’ll add features based on actual feedback rather than my assumptions.

If not? I shipped an app to the App Store. I learned SwiftUI, wrestled with App Store Connect, made trademark mistakes, fixed security holes, and remembered what it feels like to build something from scratch.

That was the point.

Tendrrr is free on the iOS App Store. It finds chicken tender restaurants near you.

Download now!

Key Takeaways for PMs Building Side Projects

1. Constraints are your friend. A hyper-specific use case eliminates decisions.
2. Architecture isn’t overhead. A little structure saves a lot of debugging.
3. Free tiers exist. Use them until you have a problem worth paying to solve.
4. Security is baseline. Never commit credentials. Rotate keys if you do.
5. App Store review is predictable. Follow the guidelines, and you’ll be fine.
6. Ship the core experience first. Polish is earned, not assumed.

Most importantly, I think every product manager should try to build out at least one idea with tools like IBM Bob, Claude code, or another similar tool. This technology is hard to believe until you try it yourself. Make something small, improve your workflow, build an app that doesn’t exist and only you would use, because now you can.

Have questions about the process? Find me on LinkedIn or check out the webpage for the app

Lessons in Vibe-Coding as a PM was originally published in AI Enterprise Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

While I’ve used many coding assistants, it’s been mostly for small things (generate code, create terraform to provision, etc.), I’ve not used a fully functioning agentic development tool. Well, I just finished my first pass at using IBM Bob, and I have to say, I am blown away!! In 20 minutes, I had a fully functioning font-end and back-end application that called APIs to fetch dynamic data and was given architectures, runtimes, and recommendations on provisioning it OnPrem or in cloud.

Here’s the video where I walk through it all, and then below I provide some highlights.

What I Created

I thought about what my first creation should be, and all kinds of IBM topics came to mind (and those will be coming in future articles, including agents, MCP, automation, etc.), but I really wanted to try something that would resonate with a broad audience, so I turned to my biggest passion: Music! I lead an acoustic trio, where we play around southern Minnesota. We have well over 100 songs and we have always thought it would be cool if we had an interactive “Song Request” app.

In all of 20 minutes, I had it working (PS: If you want to see the app in action, go to 6:15 in the video.)

Bob Highlights

Here’s what IBM Bob looks like (It’s basically VS Code)

The Magic happens on the right side where you “Start New Task”, give it an initial prompt, and it will create a To-Do list for you to approve (and edit if you want), and once you approve it will start performing the tasks.

Importantly there is an “Auto-Approve” section where you can approve levels of actions: Read files, edit and save files, run commands, and more. For my work I allowed it to read, edit, and save files, but I did not give it approval to automatically run commands. You can choose the level of “Human-in-the-loop” you want.

First prompt

It all starts with a prompt.

I want to create a new web app. It will input a spreadsheet of 
songs and band that is playing them. It will then allow the 
user to request any of those songs by tapping them. 
The app should have a list of songs and the requested songs. 
Further, the app should list "similar artists" by accessing 
music libraries that compare styles and genres

To be honest. I didn’t think this would be enough, but it created the basics and a fully running app on the first try. In its preparation (think Reasoning Agent), it created a to-do list, then marched through each one.

To-Do List:

• Create project structure and setup
• Design data model for songs and bands
• Implement spreadsheet import functionality
• Create UI for song list display
• Implement song request functionality
• Add requested songs list view
• Integrate music API for similar artists feature
• Add styling and responsive design
• Test the application
• Update app name to "Orchid Jane - Song Request"
• Redesign with purple theme and orchid graphics
• Fix Similar Artists API and provide registration instructions
• Add word cloud of artists after spreadsheet upload
• Remove word cloud section
• Add Song List / Artist List toggle view
• Remove toggle view buttons and simplify interface
• Add "Look Up" feature for Last.fm track search when no results found
• Add "Similar Artists" button to requested artists in Requested Songs section
• Auto-load Orchid-Jane-Songs.xlsx on server startup
• Increase scrollable area height by 30% for 
  Available Songs and Requested Songs
• Fix filter persistence - maintain search filter after 
  requesting song with "Back to Song List" button
• Add song count next to "Available Songs" heading
• Move "Top 5 Songs" button next to each similar artist with 
  Request Song functionality
• Create comprehensive architecture documentation with Mermaid diagrams

Note that this is the “final” To-Do list that summarized everything I did throughout the 2-hour session. The first 9 tasks were the “first pass”, and the rest was me interacting with IBM Bob to make things better, add enhancements, etc.

Various “Enhancements”

That’s the thing: It’s so easy to make enhancements I quickly became eager to make the app awesome (and I became picky). Some requests it made perfectly the first time: all defects I found it fixed the first time, but other things it just didn’t get, and I either had to compromise or abandon the idea.

For Example, I wanted a “Word Cloud” of artists that showed all the artists we play and the and the more songs we play of that artist the bigger the name of the artist. It seemed like it understood, and the code looked good, and it even added links to each artist name to filter the songs, but it never looked good.

Interestingly IBM Bob’s responses and resulting code showed it was doing what I wanted it to, but across multiple browsers it never displayed properly. I eventually said, “Just remove the word cloud”. Which it did, but it only removed the functionality from the running app, not the code. Which bothered me a bit. Minimally it should have commented out the code, but with all the code active for functionality that is not used, I could see over time this trial-to-succeed way of “vibe programming” could result in a big wasteland of unused code (hacker’s delight?).

Thought: maybe I just need to learn IBM Bob’s quirks and explicitly ask Bob to clean up/comment out code that is unused.

The following are examples of what I prompted to change

Enhancement list #1:

Make the following changes:
- Change the name of the app in the web UI to "Orchid Jane - Song Request"
- Update the style to be more modern with colors of purple and 
   add some orchid graphics
- The "Similar Artists" button doesn't work. can you figure out why 
   and if I need ot register an API key, tell me where to get it

The last bullet is a big deal here: Bob first suggested I use the Spotify API for “Similar Artists”, but when I went to get the API, Spotify is on “hold” accepting new apps to use the API. I told that to IBM Bob, and it recommended the free last.fm site, and even created a Markdown to instruct me how to get the API.

Amazing. I really liked that it thought of the human’s productivity for times it couldn’t / wouldn’t do an action for me.

Thought: Could I have asked it to create an agent to sign into a developer site as me and grab API key for me? Maybe. But then credentials representing me are known to the agent and that’s bad form, so we would need to add in identity and permissions.

Enhancement list #2:

In the Similar Artists list, add a "Request Artist" button that 
shows up in the Requested Songs list

This added the capability to add an artist from the “Similar Artists” Popup.

When the app starts, automatically load the file "Orchid-Jane-Songs.xlsx" 
located in the song-request-app folder.

This sped up my testing since I didn’t need to load the data every time the app started.

make the "available songs" and "requested songs" visible 
scrollable area 30% longer

This altered the UI panels (I did ask where it was set and it showed me, but it was so fast to just have IBM Bob do the work.)

Defect Fixing

This was really fun (almost more fun than adding features). In my past when I led design teams, I would work really well with the development team and we would work on fixing issues, altering the designs (I would call those defects as well), but it took time. With IBM Bob, I just type it and it was fixed in 45 seconds.

There's a defect: When I search and get a subset of songs, 
then click "Request Song", the list of songs returns to the 
full list. the filtered list should stay with the subset of 
songs and a "Back to Song List" button at the top

This is Bob’s response:

I'll fix this defect by maintaining the search filter state 
and adding a "Back to Song List" button when viewing filtered results. 
Let me update the code:

Realtime Enhancements in Video

In the video I wanted to show some real-time action, and IBM Bob did not disappoint. It’s worth looking at the video at 8:43.

In Similar Artists popup, add a button called "Top 5 Songs" that 
will show the top 5 songs of that artist using the last.fm API. 
Put a "Request Song" button by each of those songs so it is 
added to the Requested Songs list. Then close the popup and 
return to the home screen.

IBM Bob’s Response:

I'll add a "Top 5 Songs" button to the Similar Artists modal that 
fetches and displays the top songs for the selected artist. 
Let me start by updating the backend to add the new API endpoint:

However, the result wasn’t exactly what I wanted, so I asked differently:

that was close. I want the "Top 5 Songs" button to be next to each 
of the similar artists so I can see similar artists then see the 
top 5 songs of that similar artist, with a Request Song button 
next to each of the songs.

Can you fix that?

IBM Bob’s Response:

I understand! You want a "Top 5 Songs" button next to each similar 
artist (alongside the "Request Artist" button), so you can see the 
top songs for that specific similar artist. Let me fix that:

That was it!

Thought: This is more of “how to train Greg to talk to IBM Bob” than it is to request IBM Bob enhancements. It also reinforces that a holistic, design-driven human that can communicate to IBM Bob what an application should do is more important than ever. IBM Bob is just responding to the prompts, so precision in how you ask it, detailed examples, and even listing security best practices, etc., is essential.

Final Documentation and Architecture

In the end, I asked it to create a comprehensive architecture spec, and it created several diagrams including the main architecture:

This was a fantastic first experience with IBM Bob, and I urge you to take it for a spin.

Have you tried IBM Bob? How about Claude Code? Others? I’d love to hear your experiences below.

IBM Bob — HOLY BUCKETS This is Amazing! was originally published in AI Enterprise Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

With Nanochat released from Andrej Karpathy we have a great training ground to explore the architecture of Large Language Models. It is compelling since a working ChatGPT-like model can be built and trained on GPU hardware in 4 hours.

Cost? About $100.

I wanted to explore what it takes to put Nanochat LLM into the enterprise ready Red Hat OpenShift AI environment.

This Git repository steps through the exploration.

• Train the Nanochat Model on Lambda Cloud
• Use the Nanochat model on RHEL AI
• Convert Nanochat to ONNX and use in Red Hat OpenShift AI

The code and walkthrough is here.

Step 1: Train the Nanochat Model on Lambda Cloud

Karpathy makes the training phase very straightforward. Lambda Cloud offers 8 x H100 NVIDIA GPUs for approximately USD $25 per hour and the training run takes 4 hours.

Although not outrageous in cost (compared to the major cloud providers GPUaaS) we don’t need H100 GPU environment hanging around. Since I needed to understand how the trained model and its outputs worked I had to capture them into an S3 bucket.

On my Lambda GPU instance I created an S3 filesystem and cloned the Nanochat repository. A simple script kicks off the training process. But before you start you configure the model output directory to be on the S3 filesystem and not use up the GPU instance disk.

export NANOCHAT_BASE_DIR="$HOME/caine-fs/.cache/nanochat"

Once the training was done I copied all the files cloned and generated to another S3 bucket in IBM Cloud where I have an account. Then I could de-provision the Lambda environment.

Nanochat’s training happens in three stages:

1. Base training (3.5 hours) — The model learns language

2. Mid-training (15 minutes) — It learns chat format and reasoning

3. Supervised fine-tuning (10 minutes) — Final polish for assistant behavior

I kicked off the training in a screen session and walked away. Four hours later, I had a 480-million parameter language model.

With training complete, I used rclone to copy everything across to IBM Cloud Object Storage into a bucket called nanochat-llm.

Step 2: Use the Nanochat model on RHEL AI

With our enterprise architecture thinking hats on the ‘rent vs buy’ for both training and inference relates the model in question and your use cases for it. This informs the GPU investment and consumption strategy.

I provisioned an IBM Cloud Virtual Server with an NVIDIA L4 GPU. The L4 is designed for inference (not training) and has 24GB of RAM right sized for a 20 layer, 480M parameter model.

The Virtual Server Instance (VSI) used an AI-enabled operating system image — Red Hat Enterprise Linux AI, NVIDIA bundle. Red Hat Enterprise Linux AI (RHEL AI) comes pre-bundled with vLLM and InstructLab container images.

I could then use rclone to mount the S3 bucket directly as a filesystem.

rclone mount ibmcloud:nanochat-llm /var/mnt/caine-data/s3-mount \
 - vfs-cache-mode full - allow-other - daemon

We want to spin up and use the LLM in this VSI. Nanochat expects its files in ~/.cache/nanochat/ . Rather than moving files around, I created symbolic links pointing to the S3 mount:

mkdir -p ~/.cache/nanochat
ln -s /var/mnt/caine-data/s3-mount/nanochat-complete-run-20251115/chatsft_checkpoints ~/.cache/nanochat/chatsft_checkpoints
ln -s /var/mnt/caine-data/s3-mount/nanochat-complete-run-20251115/tokenizer ~/.cache/nanochat/tokenizer

Out of the box in the Nanochat repo is a working chat web app. With port 8000 exposed, and the model started I could get to the chat web app from my laptop and interact with the LLM and ask questions about the sky!

RHEL AI is a single-server solution for AI operations and has many possibilities for integrating in different platform and deployment architectures. But I am initially interested in how a custom LLM could be brought to an enterprise ready hybrid AI platform like Red Hat OpenShift AI (RHOAI).

Step 3: Convert Nanochat to ONNX and use in Red Hat OpenShift AI

Red Hat OpenShift AI offers a number of ways to build and train data science projects and AI models. These environments are managed on the platform running on OpenShift Kubernetes clusters. A data science project is a specialised OpenShift project (namespace). In here we can create model servers.

With the model server instance created you can then deploy different types of AI model:

• openvino_ir opsetl (OpenVINO Intermediate Representation native model format, optimised for speed)
• onnx-1 (Open Neural Network Exchange models, best for framework-agnostic, models from PyTorch, scikit-learn etc)
• tensorflow-2 (native Tensorflow and Keras models)

The ONNX Conversion

Nanochat has a custom architecture that doesn’t map directly to standard frameworks like HuggingFace. With some vibe assistance I created a conversion script that exported the PyTorch model to ONNX:

…
torch.onnx.export(
model,
dummy_input,
"model.onnx",
input_names=['input_ids'],
output_names=['logits'],
dynamic_axes={'input_ids': {0: 'batch_size', 1: 'sequence_length'}},
opset_version=14
)
…

The resulting 1.8GB ONNX file went back into the S3 bucket.

The Red Hat Developer Sandbox is a great way to get started with understanding OpenShift AI. You create a model server and then deploy a model to it:

1. Create an OpenVINO Model Server instance (Medium: 4 CPU, 8GB RAM)
2. Deploy the ONNX model from S3
3. Get an inference endpoint URL

Basic curl testing could send a text input to the endpoint. Those input tokens generated raw logits (prediction scores) for every possible next token. We know the model is now serving but of course its not very user-friendly.

The Chat App API

Using the Nanochat web app as a guide, I could create a simple FastAPI app that would be a more user-friendly chat API endpoint.

The app

• takes natural language input
• tokenises it into token IDs
• calls the RHOAI inference endpoint
• samples the next token from the logits
• repeats until the response is complete
• decodes back to text

The hardest learning part of this exercise was the tokeniser. Nanochat’s tokenizer uses tiktoken, which has Python wrappers around compiled C++ code. The model build code pickled the tokeniser in the Lambda environment, which then doesn’t work when moved to the new VSI environment on IBM Cloud.

AttributeError: 'Encoding' object has no attribute '_core_bpe'

Pickle files save Python object states, including references to C++ extensions. But they don’t save the actual compiled code. When you unpickle on a different machine with different binaries, those references point to nothing. One quick and dirty solution to this is to build a fresh tokeniser at chat API runtime. It extracts the vocabulary data and regex pattern to create a new encoding.

# Load broken pickle
enc = pickle.load(f)

# Extract working parts
mergeable_ranks = enc.mergeable_ranks 
pattern = enc.pat_str 

# Rebuild fresh encoding
new_enc = tiktoken.Encoding(
  name="nanochat",
  pat_str=pattern,
  mergeable_ranks=mergeable_ranks,
  special_tokens={…} # Hardcoded
  )

(Meanwhile, many talented people are already beginning to industrialise Nanochat — example here)

This experience puts your enterprise thinking hat on again. Enterprises don’t build critical systems in one repo, or one notebook. You have to think about the physical separation of different software components and how each manages scaling. You can’t just copy virtual environments between machines, and packages must be reinstalled to get properly compiled extensions in your target environment.

The Final Result

With the new chat API app in a GitHub repo that was deployed to in the same project namespace of the OpenShift cluster where RHOAI was installed and configured to talk to the Nanochat LLM.

curl -X POST https://nanochat-app-jeremycaine-dev.apps.rm2.thpm.p1.openshiftapps.com/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
  "messages": [{"role": "user", "content": "Hello, who are you?"}],
  "max_tokens": 50
 }'

# Response:
# "I am nanochat, a Large Language Model.
# I was built by King Andrej Karpathy in 2025…"

Conclusion

Training is the easy part (when you are reusing 😂). Four hours and $100 got me a working model. The rest of the time went to deployment, debugging environment issues, and understanding production serving.

Pickle files are not portable. Save data, not objects. Export vocabularies and configurations as JSON. Rebuild objects in the target environment.

The GPU matters for the use case. Training GPUs (H100) are expensive powerhouses. Inference GPUs (L4) are cheaper and optimised for serving. The L4’s 24GB was the right size for a 20 layer, 480M parameter model.

Cloud storage and S3. Mounting S3 as a filesystem eliminated file copying. Just remember some tools (like Python’s uv) don’t play nicely with network filesystems.

Read more about AI models and Red Hat OpenShift AI here.

Views are my own.

Putting Karpathy’s $100 LLM to work on Red Hat OpenShift AI was originally published in AI Enterprise Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Train Once, Deploy Responsibly, and Optimise Inference

A while back I took a notebook based machine learning tutorial and built my own AI platform. Well, not really.

What I did was break up and organise the tutorial code so I could learn about the data manipulation during training, how to build a model and package up for use. I wanted to see what the runtime application flow and model serving needed to be. To that end I created an API endpoint that loaded the model into its memory — it was in essence a single task, custom model server. Then a web app would send data to the API endpoint and make an inference.

Of course in the real world leading companies consume, and do not build, state of the art platforms that perform and scale achieving balanced cost, agility, and risk capabilities.

Red Hat OpenShift AI (RHOAI)

This data science platform has evolved a lot over the last few years extended from machine learning and deep learning to generative AI. Its foundation is a multi-tenant, multi-modal, secure enterprise software platform.

Porting MNIST to RHOAI

The original tutorial app is a MNIST image recognition app. It takes a hand-drawn image of a numeric digit 0–9 and uses the model to infer what number has been drawn. That exercise is captured here, and the Git repo is here.

I have now ported the code to the OpenShift AI platform (repo). It’s not a production system but through the exercise you can see how the components interact and how models and apps need to scale securely for real enterprise scenarios.

The Red Hat Developer Sandbox

With you Red Hat account login you can spin up a basic OpenShift cluster in the AWS cloud for 30 days free. From here you can get an OpenShift AI service, where a default data science project is created for you so that you can train and deploy models. Since it is an OpenShift cluster you can also deploy a Kubernetes web app that will call the inference service.

Navigating the OpenShift AI Environment

In a Data Science project you can create a number of things including:

• multiple workbenches — which launch as Jupyter Hub development environments
• connections to object storage — for your model data files and final models for serving
• model servers — in the sandbox the single type allowed is an OpenVINO Model server with multi-model serving enabled
• model deployments — different model typeslike Keras TensorFlow and ONNX can be deployed onto the model server.

Before I had created my own model server via a Flask API app wrapped around the model.

Now in a real platform you simply take the model and deploy it — meaning you configure the server to connect to the model file so it can load it into memory and provide an inference endpoint URI for an app to call.

There wasn’t a great deal of change to be done. The calling app is still the same web app taking a mouse-drawn image on HTML canvas sent as payload data to the model inference.

Walkthrough using OpenShift AI

I have a new repo for this exercise here.

There are many useful tutorials from Red Hat across its products. I ended up cutting across a few of these to figure out what component is doing what, how environment variables are turning up, and entries into the model server and deployment configuration.

For example, it wasn’t obvious at first but when you deploy a TensorFlow model to a server it needs to be a single model file (.keras) and it cannot be the SavedModel format and its multiple files.

The steps of the exercise:

1. Source the MNIST digit image dataset from Kaggle.
2. Create your Red Hat account and spin up a Sandbox.
3. Create a workbench inside your data science project.
4. Create and connect object store and buckets to the project.
5. Upload the MNIST data files.
6. Use a notebook to build and train a model.
7. Convert the model to formats like Keras and ONNX.
8. Export the model to the object store for serving.
9. Create an OpenVINO Model Server instance.
10. Deploy your models to it and generate the inference endpoint URIs.
11. Deploy a Kubernetes app to consume your model.

The Future is Hybrid AI

This has been a simple exercise that quickly familiarises yourself with many of the features and architecture of the Red Hat OpenShift AI platform.

Red Hat OpenShift AI is a significant, state of the art, enterprise ready AI platform. And because it is built on top of OpenShift Container Platform it gives enterprises total flexibility with its cost, agility and risk drivers.

The stages of the AI development and delivery lifecycle need not be constrained. The flexibility to host models in multiple infrastructures is possible and the GPU cost of inference can be optimised. The locations of data for model training, fine tuning and generative retrieval cases can exhibit different security, isolation and performance characteristics.

• Train Once — create or fine-tune machine learning, deep learning and generative AI models once and host in multiple destinations.
• Deploy Responsibly — choose a deployment location for the model and its data that right for their sensitivity.
• Optimise Inference — since inference is estimated at 80% of the overall cost [1], run the model on software and hardware that favours price-performance optimisation.

The future of AI is hybrid.

Sources:

[1] Inference cost can account for 80% of the total operating expenses for AI-first companies. 2023 analysis by Andreessen Horowitz.

Create a leading Hybrid AI capability with the Red Hat OpenShift AI platform was originally published in AI Enterprise Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Have you felt the visceral feeling the moment you hear a synthetic AI voice delivering your creative content as a new, generated, piece of work?

Jeremy and I have. (Watch the video below!!) ꜜꜜꜜꜜꜜꜜꜜꜜꜜꜜꜜꜜꜜꜜꜜꜜ

We are both deep in AI technology, and yet wanted to experiment with NotebookLM, not just as a useful AI tool to summarize lengthy documents, but as creators of that content, and create a “reaction video” as we listen back to the results.

Uniquely, we wanted to focus on the explosion of creative AI: Music, articles, books, images, video that is generated based on data it was trained upon: created, curated, and many times copyrighted data originally created by humans (I know, synthetic data…but even that is generated from original human creation).

How would we feel? Would it be an accurate representation of our work? Could we call it our own work? Would we use it again?

Further, what do other creators think when they see (or hear) generated content that is created in seconds but is a result of being trained on content that took years of craft-building to create?

Watch the video, and comment if you’ve experienced AI using your creative material as its own. I would love to hear your reaction!

For reference, here’s the original article that was used called “How to Become an AI+ Enterprise”

When AI Uses Your Creative Content was originally published in AI Enterprise Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Using Bee Stack, Ollama, Podman and the IBM Granite models

We can already see that 2025 is lining up to be a big year for AI agents. Towards the end of 2024 IBM open sourced the Bee Agent Framework and platform.

Bee Agent Platform lets developers create, customise, and deploy AI agents. It is a collection of components that enable the build of production-ready AI agents.

In this short introduction I explore the Bee Stack and how it uses models.

The key outcomes are:

• run everything locally on macOS
• setup and start the lo-code/no-code Bee Stack
• run the Bee Stack with Podman
• use Ollama as the model runtime
• configure an assistant to use an IBM Granite model instead of the default llama 3.1

What is Bee Stack?

The Bee Agent Framework, its components and the “all in one” Bee Stack are in early stages of development. At the time of writing there are plenty of code samples, but not so much in the way of walkthroughs and tutorials.

Start here for Bee Stack

Bee Stack is an application system. It has everything that is needed to run the complete production stack for integration into application solutions. The system runs using Docker Compose.

The Bee Stack is:

• bee-agent-framework gives the foundation to build LLM Agents.
• bee-code-interpreter runs a user or generated Python code in a sandboxed environment.
• bee-api exposes agents via OpenAPI compatible Rest API.
• bee-ui allows you to create agents within your web browser.
• bee-observe and bee-observe-connector help you to trace what you are agents are doing.

Using Podman

I am using Podman instead of Docker. Podman is ideal for designing and building production ready, secure container based applications. This include running containers as rootless, and sets a path to deploying into enterprise ready container platforms like Red Hat OpenShift.

The Bee Stack guidance is to run its containerised components in a VM that has at least an 8GB RAM VM.

Ensure Podman is running.

$ podman machine start
Starting machine "podman-machine-default"
API forwarding listening on: /var/run/docker.sock
Docker API clients default to this address. You do not need to set DOCKER_HOST.
Machine "podman-machine-default" started successfully

Bee Stack uses your container runtime and Compose to run its components. For a Podman installation you also need to install Compose. This brings in Docker Compose under the hood that podman-compose executes.

$ podman compose -v
>>>> Executing external compose provider "/usr/local/bin/docker-compose". Please see podman-compose(1) for how to disable this message. <<<<
Docker Compose version v2.30.3-desktop.1

Bee Stack also needs the container runtime to run as root. You can check that if Rootless is false then Podman is running as root.

$ podman info - format "{{.Host.Security.Rootless}}"
false

To setup Podman to run in its VM with root privileges:

podman machine stop
podman machine set - rootful=false
podman machine start

Using Ollama to serve the models for the agent

In order to run locally we need a model runtime and some models. Start by installing Ollama.

Instead of using the default llama3.1 we are going to use the IBM Granite model and the team recommend granite3.1-dense:8b. Get the model locally to use with Ollama.

ollama pull granite3.1-dense:8b

The assistants will use this model once Bee Stack is configured.

You can see the list of models pulled down to your local machine.

$ ollama list
NAME ID SIZE MODIFIED
granite3.1-dense:8b   86ac4cf0cb84   4.9 GB   19 minutes ago

Configure the Bee Stack

First, we clone the Bee Stack repo.

git clone https://github.com/i-am-bee/bee-stack.git
cd bee-stack

Next we run setup to use Bee Stack with Ollama.

# Run setup script to configure LLM provider and start the stack

./bee-stack.sh setup
Choose LLM provider:
[1]: watsonx
[2]: ollama
[3]: openai

Pick 2 for Ollama, and when asked for the OLLAMA_URL hit enter and accept the default http://host.docker.internal:11434.

Checking Ollama connection
Ollama is running

If you have enough RAM for your containers you can enable Docling text extraction.

Do you want to enable docling text extraction? 
⚠️ Requires >= 15GB of RAM **CONFIGURED** for the container runtime ⚠️ (Y/n):

For an initial setup we can say no to this. We can re-run setup any time.

What the setup script is doing is creating a .env file which should look something like this.

AI_BACKEND=ollama
EMBEDDING_BACKEND=ollama
OLLAMA_URL=http://host.docker.internal:11434
FEATURE_FLAGS='{"Knowledge":false,"Files":true,"TextExtraction":false,"FunctionTools":true,"Observe":true,"Projects":true}'
EXTRACTION_BACKEND=wdu

The next question in the setup:

Do you want to start bee-stack now? (Y/n):

If you do NOT have llama downloaded as an Ollama model AND you start the Bee Stack, then when using the default Bee Assistant it will fail because it cannot find the model.

If you want to see that error then answer Y to start bee-stack. Now Bee Stack pulls down the its container images and starts them up. Once everything is up you go to the Bee Stack UI.

http://localhost:3000

After a fresh setup, you are greeted with:

From top left, go back to the app library.

Now try the assistant and type something like “hello”.

Each agent is configured to use a model. If that model is not available from the model runtime provider (Ollama in our case) then you get this error.

The setup has created this default Bee Assistant and set the model to llama which haven’t pulled down (yet).

General Bee Stack operation

The control script is used to setup, stop, start and clean the Bee Stack system.

# setup the stack - configure it to use a model provider (e.g. watsonx, Ollam) runtime and create .env file
./bee-stack.sh setup

# start the stack - pulls any containers it needs, and runs
./bee-stack.sh start

# stop the stack
./bee-stack.sh stop

# clean up the stack - removes the container images and data
# the team advise it is good practice to run this to remove the stack between upgrades
./bee-stack.sh clean

Configure Bee Stack to use IBM Granite

Start your Bee Stack ./bee-stack start. Now we need to set the model that an assistant will use. Currently we have to apply this configuration through calls to the Bee API server.

We tell the Bee API server to create a new assistant, and use the Granite model where the model name is the same as the one returned from ollama list.

curl -X POST \
  "${BEE_API:-localhost:4000}/v1/assistants" \
  -H "Authorization: Bearer ${BEE_API_KEY:-sk-proj-testkey}" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "IBM Granite assistant",
  "model": "granite3.1-dense:8b"
  }'
  
Returns:
  {"id":"asst_6789549fad9c88b02e17f96f","object":"assistant","tools":[],"tool_resources":null,"instructions":null,"name":"IBM Granite assistant","description":null,"metadata":{},"created_at":1737053343,"model":"granite3.1-dense:8b","agent":"bee"}

Use Bee Stack with IBM Granite

Now if you go to localhost:3000 and the assistants list in the left navigation you will see “IBM Granite assistant” and chat to it.

Discover the assistants

There are multiple assistants pre-configured by Bee Stack which you see with the GET assistants call.

curl -X 'GET' 'http://localhost:4000/v1/assistants?limit=20&order=desc&order_by=created_at&public=false' \
  -H "Authorization: Bearer ${BEE_API_KEY:-sk-proj-testkey}" \
  -H 'accept: application/json' | jq - color-output

This returns JSON data and you can see the assistant ids for each of them.

{
  "data": [
    {
    "id": "asst_6789….",
…

Bring in other models

Through the API you can do all sorts of things including update the model and tools being used by an existing agent.

You can get the default model Bee Stack expects ollama pull llama3.1 and now the Bee Assistant will work.

Cleanup

You can see the full Bee API at http://localhost:4000/docs (4000 is the port the bee-api listens on).

e.g. to delete an assistant.

curl -X 'DELETE' \
'http://localhost:4000/v1/assistants/asst_6789549fad9c88b02e17f96f' \
  -H "Authorization: Bearer ${BEE_API_KEY:-sk-proj-testkey}" \
  -H 'accept: application/json'

{"id":"asst_6789549fad9c88b02e17f96f","object":"assistant.deleted","deleted":true}

Discord channel

For support you can join the Discord channel. The invite to join the channel is here. The Bee Open Source community support channel is https://discord.com/channels/1309202615556378705/1313222938714963979.

Conclusion

The Bee Stack is a great way to get up and running with all the components needed and create a production like agent and assistant system “in one box”.

Next to explore is the Bee Agent framework and Bee Agent Python SDK and custom build and integration of AI agents into applications.

Happy 2025!

Getting Started with a local setup of the Bee Agent Platform was originally published in AI Enterprise Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Started with a Question

LinkedIn’s Generative AI group posted a comment with a request to share thoughts on the following quote from Jensen Huang:

“In a lot of ways, the IT department of every company is going to be the HR department of AI agents in the future.

Today, they manage and maintain a bunch of software from the IT industry; in the future, they will maintain, nurture, onboard, and improve a whole bunch of digital agents and provision them to the companies to use.”

- Jensen Huang, CEO of NVIDIA, CES 2025

TL/DR

Here’s what I think:

• This is a limiting view. Harnessing AI tech of any kind involves the whole business, not just IT.
• Experts in the use case that the AI Agent is delivering needs to be the team that acts as HR to onboard, manage, and improve that AI agent. IT will be HR for their AI Agents, and lines of business will be HR for their AI agents.
• IT plays an essential role: IT should deliver centralized method for the technology required to onboard and improve their AI agents.
• IT needs to apply its expertise in running data, apps, hybrid cloud platforms, pipelines, day-2 operations, and security so that AI agents don’t break the business.
• SLAs, RACIs, NFRs, and basic laws of physics need to be applied to AI agents…and IT is best suited for that.
• HR should create a “Business Conduct Guidelines for AI Talent” for AI agents built from IT and lines of business expertise

Read on for more insight.

Limiting View

Look at this graphic:

I’ve worked with enough enterprise clients to confidently say that 1) AI impacts the entire business and 2) AI can only be successful across the business if the entire business embraces AI in its work and its culture.

Most of an enterprises business value comes from the use cases delivered by the lines of business. The lines of business have spent years creating value for their customers, and lines of business know where AI can exponentially deliver value for their customers. IT is similar in that IT wants to deliver greater value to its customers, but in this case IT’s customers are the lines of business themselves.

Everything is interconnected.

Additionally, lines of business and IT both want to optimize and reduce costs. This increases ROI. AI is already optimizing internal processes very effectively and will continue to do so with AI agents.

In the end both lines of business and IT will be responsible for managing AI agents that deliver greater value and optimize costs based on the use case the AI agent is trained for.

HR is for managing Talent (Agents and Humans)

HR is responsible for the well-being of current talent, the attraction of new talent, and the handling of talent that is not quite in line with what the business wants and needs. According to Jensen, AI agents will need to be managed just like humans, so a new mission may be “HR is responsible for managing talent instances: Human and AI”.

HR knows that every talent instance is unique has different strengths and weaknesses. HR runs programs to celebrate those strengths and handle weaknesses either through improving that individual talent instance, or through creating a team where the strengths across talent instances complement each other (and ideally one’s strengths support the another's weakness).

Since an AI Agent is based on an LLM (or a group of LLMs), each agent can be very unique in its strengths and capabilities that contributes to achieving the use case at hand. It is not in the interest of the enterprise to have IT manage all AI Agents because IT does not know where the strengths and weaknesses are for a specific agent because it depends on the use case being delivered by the line of business. For example: An AI agent may be working in the trade finance industry to detect compliance and regulatory fraud in shipping documentation. The trade finance lines of business knows very well how that agent is performing, the risk involved in the work, and if there needs to be a human in the loop. Further, there are very technical people in lines of business that have the capability of choosing AI agents based on their strengths, combining agents based on strengths and weaknesses just like there are very technical people today writing code, architecting, and managing cloud instances of their business applications.

I assert that the agent is only as good the value it delivers, and the lines of business know that value.

IT Plays an Essential Role

All that said, IT is essential in delivering value with AI agents. The AI agent itself is just a small part of the enterprise environment. IT has key roles in creating a centralized and scalable environment to build, train, run, update the AI agents, as well as create the hybrid cloud environment all of this runs on, to enable pipelines to provision and deliver, update with changes, to run day-2 operations that can sustain this over time, and to secure the entire enterprise with AI agents much like it does today with governance, risk, compliance, and security practices.

I recently listened to a podcast about everything we’re experiencing, and relating it to software architecture evolution…which generally creates a new abstraction layer on top of what already exists. If I extrapolate what I heard, then AI agents are just another level of abstraction on top of software and data. When properly trained, the agent has a long list of functions that can call and instructions on what those functions can provide. It is kind of like an API Gateway, but better because the agent knows when to call certain functions.

All that is fantastic. However, it doesn’t remove what was abstracted: Proper software architecture, enterprise architecture, cloud architecture, data architecture, and so on.

Here’s my view (reinforced from the graphic above):

• The AI agent is composed of AI technology (LLMs, frameworks, runtimes) that needs to be trusted and secured.
• The LLMs are trained on enterprise data.
• The Data needs to be secured, protected, accessed, and curated properly to provide the highest quality data.
• The AI agent calls functions to get complex tasks accomplished, and those functions include existing software endpoints (APIs) that return content based on what is being asked for.
• The endpoints depend on the underlying application. If it is a dated application, it may have trouble scaling to the number of requests, which make the agent useless. Proper cloud-scaling application architecture is required for these functions to respond with speed so the AI agents can return value with speed.
• Certainly over time AI agents will go deeper and deeper into the application stack, but years of business logic will not be removed quickly (are we ready for an agent to create an internal API that replaces regulator-approved business logic?).
• The AI agent (and functions) need a hybrid cloud platform to run on. Enterprises I work with have regulatory controls dictating where the data has to be. If the agent is composed of data, there are strong rules about where can run and where it can store data.
• AI, data, apps, platforms need to be deployed via automation. AI technology (especially agentic technology) changes weekly so it’s essential for an to update which agent is used based on its quality and performance based on the use case. The line of business should do the deciding, and IT should provide the automation.
• Day-2 operations need to sustain this environment and make it thrive. How do we know in four months everything is performing as it should? Who is the decider? Both lines of business and IT.
• Vulnerabilities that show up in AI agents, the functions it calls, the platforms they are running on, all need to be discovered, remediated, and managed.
• Security, governance, risk, and compliance envelops everything a business does today. This will not change with agentic technology running around in the business (even that phrase ‘running around’ is unsettling).
• As business needs grow, where will these AI agents (and the functions they call) need to be deployed? Is that automated with an AI agent from IT, or do we need a human-in-the-loop governance and operations team that approves what the agent is doing just like HR set in place controls and guard rails for human.

IT Enterprise Architecture and AI Agents

There is an area that IT does need to lead and get right with agentic technologies: Making the AI agents understand the laws of physics, SLAs, RACIs, and NFRs (Service Level Agreements; who is Responsible, Accountable, Consulted, and Informed; and Non-Functional Requirements).

Clients I work with are experts at these: They know what is required to run a multi-billion-dollar business. It is not just about calling the right functions, organizing information, and returning the results. To achieve success with AI agents, enterprises need to ensure the following is done (and IT is the team to do it…along with their AI agents):

• How software calls databases.
• Where databases are running throughout the country / region.
• Business logic dictating software-defined network paths
• Ensuring latency is at a minimum and response times are their fastest
• Data needs to be located where regulators demand
• Application should be located wherever SLAs (response time, latency) can be met. If apps can run on cloud while data is OnPrem, and SLAs be met, this may be best of all worlds. AI agents should know that.
• Decisions on app location should be laws of physics and SLA driven, not expert opinion. It doesn’t really matter what conclave the data is in, where the applications are, as long as those SLAs are met.

With agentic technology, it is critical for the IT team to add guardrails around the agent controlling what it calls in which endpoints so that those SLAs are met. For example: An AI agent is solving a problem to approve a loan for a customer for a world-wide bank. The AI agent “knows” functions to call, but does it know appove.in() and approve.us() and approve.sg() are for different regions of the world and have very different regulations governing that business logic? Does the AI agent know if data from India is stored in US data centers it may be an illegal activity?

Further, SLAs are a key component of a user’s experience. If we deliver value to customers, but the experience is slow, or data leaks where it shouldn’t, the customer won’t care if it’s agent technology or hamsters running a treadmill: they will not use it and they will tell others.

As we give more responsibility to AI agents, it is essential we provide the guardrails for success with this higher level systems-thinking.

Summary — Business Conduct Guidelines

AI agents will be a great way to provide value for enterprises and customers and optimize and save on costs. IT plays an essential role, as does lines of business.

Everything is interconnected.

Don’t think for a minute that either IT or lines of business on their own can 100% control agentic technologies. AI requires the entire business to be engaged to embrace AI technology. It’s a “we’re all in this together” kind of answer.

In fact, just like HR creates “Business Conduct Guidelines” today for humans, the future HR team (abstracted another level higher just like Grady Booch says) can pull insight from IT and lines of business so that any talent instance, human or AI agent, can use these business conduct guidelines as guardrails when acting to deliver value for the business. The benefit for AI agents, is we can generate these guidelines with precise measurements, compliance controls, code, and measurable values to help it determine action or inaction.

What are your thoughts?

AI Agents: How to Manage AI Talent was originally published in AI Enterprise Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

My work at IBM focuses on 2 things: 1) Industrializing AI for enterprises and 2) enabling hybrid multi-cloud environments for those same enterprises.

Part of keeping technically current is to experiment with anything I can find to make myself better at what I do.

Oh, and I am a gigging musician as well.

Today I got a slack “Hey, you should try v0 by VerCel”, so I did…and thus begins another Visceral moment in AI.

Maybe it’s because I just got off 2 weeks of vacation where we gigged a LOT, or because in 2025 I am striving to be continually grateful for all the things in my life and want to blend it all together, so when v0 asked me what I’d like to build, I instantly thought of a guy at our gig walking up asking if we did song requests. “Sure, if we know it!” was our response. He asked and we knew a different song from the same artist, so thanks to his bold move, we played it and everybody was happy. But, what about the timid person who wants to hear a song? Maybe they could just open an app to do the same?

Here’s what I want:

• Mobile app that allows listeners to submit song requests
• The app “knows” who is playing based on proximity so it’s a simple “open app, select/type song, hit enter”
• The app knows all songs the band plays based on scanning the interwebs or pulls from the band’s submitted song list
• The listener can submit “any” song as well, since maybe they can wing it, or play a different song by the same artist
• Once submitted, the band can see the request from their “artist view” in the app
• The app shows the music/lyric charts for the song if in the song list
• The app shows Ultimate Guitar chords for songs requested that are not in the band list (so the musician doesn’t need to go hunting for the chords/lyrics)

Now I’m excited!! I typed in,

can you create a mobile front end for an app that requests a song for the musician you are listening to now in the club?

And it generated code, then showed a preview!!

Wow. I quickly told it a few more things:

add a bar scene in the background graphic

It did OK with that but I was envisioning a really cool blurred shot of a band on stage, not a generic rectangle (I think the AI thought “a bar of gold”, not “a bar with awesome music”)

I then typed in:

add a button called “Browse Song List” then when selected, show a list of 100 songs I can select and submit

v0 went to work, and generated the code, and below is the result!!

This happened (including the inspiration for the app) in the span of 4 minutes. Amazing.

Now, is it perfect and ready to rock? No.

Is it interesting enough to connect a database with song lists and see if this sucker would fly? A big, huge YES!

Amazing.

This is the interface where I can type my requests, see the preview or view the code to make changes (and console).

One more thing

Remember, this is generative AI. If you don’t like what it generated, try again!

Back in Figure 1, if I clicked “Browse Song List”, it showed “Song 1…Song100”, and when I clicked it, nothing happened. I closed the list, then realized the selected song was put in the box.

So I was able to revert back to “previous versions”, and requested again

add a button called “Browse Song List” then when selected, show a list of 100 songs I can select and submit

…and look at the improvements!

• It showed in responsive UI (but notice the buttons are in a different order)
• It showed real songs (then some generic)
• OnClick, it closed the list and showed the main view with the selected song

Of course, if I wanted small tweaks, I’d have to manually do it not knowing if it would re-generate everything and de-generate the item I really liked.

Summary

This is exactly the kind of use case that puts excitement, and to be honest, fear, into the hearts of people around the world.

The excitement is for anyone with a great idea (I think this would KILL in the music world) who can type the idea into existence, and with GenAI, create an interactive prototype to see if the idea can fly. With this tech, great ideas will not be limited to those with deep coding skills. On the flip side, anyone with marginal coding skills may think they may not find jobs as easily. But I think those with marginal coding skills will THRIVE since they can use tools like v0 as a way to deliver great outcomes by augmenting their skills with GenAI.

Finally, deep development skills to connect to back-end databases, access control systems, multiple views (artist/listener views in my example above), integrating 3rd party apps (Ultimate Guitar chords) are still needed. GenAI won’t get all of it working without a smart human orchestrating all of this.

At least for now. (after all, I did ask it for a ‘front-end’, not a full app connected to a database at a specific location…maybe all I had to do was ask :-).

Until then (or because it is coming), I urge every single human to experiment with AI technology now. Go right now to v0 by VerCel or any other GenAI site, and see what you can do.

Why? Because of something my very smart son said. “Magic is just technology we don’t yet understand”. If we understand it, we can wield its power…we can become better humans, more creative humans, more productive humans, all by understanding the kind of tool Generative AI can be for us. If we don’t, then we are in trouble…maybe not for our existence, but quite possible for the dream job we want that may be filled by the human that can wield this tool.

Go and experiment!

Now!

When you come back, tell me below what you experimented with and created in the comments below.

Happy Experimenting!!!

Visceral Moments in AI: Generating my Song Request app was originally published in AI Enterprise Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

We have all been witnessing the transformative power of generative artificial intelligence (AI), with the promise to reshape all aspects of human society and commerce while companies simultaneously grapple with acute business imperatives. In 2024, companies confront significant disruption, requiring them to redefine labor productivity to prevent unrealized revenue, safeguard the software supply chain from attacks, and embed sustainability into operations to maintain competitiveness.

AI is at a turning point, driving exponential advancements in an organization’s prosperity and growth. Generative AI (gen AI) introduces transformative innovation to all aspects of a business; from the front to the back office, through ongoing technology modernization, and into new product and service development.

While many organizations have implemented AI, the need to keep a competitive edge and foster business growth demands new approaches: simultaneously evolving AI strategies, showcasing their value, enhancing risk postures and adopting new engineering capabilities. This requires a holistic enterprise transformation. We refer to this transformation as becoming an AI+ enterprise.

Become an AI+ Enterprise

An AI+ enterprise innovates with AI as the primary focus, understands that AI is fundamental to the entire business, and recognizes that AI impacts all aspects of the business: product innovation, business operations, technical operations, as well as people and culture.

An AI+ enterprise integrates AI as a first-class function across the business. They understand that if one area of the business adopts AI while others lag or resist it (due to valid concerns), this exacerbates issues like Shadow AI, making it challenging to implement a holistic strategy.

Benefits of being an AI+ enterprise

The vast business opportunity with AI, forecasted by Gartner to bring USD 3 to 4 trillion in economic benefits to the global economy across industries, prompts companies to recognize the investment required to use AI effectively, and are demanding a dramatic return on investment (ROI) before investing in an AI use case.

By becoming an AI+ enterprise, clients can realize the ROI not only for the AI use case but also for improving the related business and technical capabilities required to deliver AI use cases into production at scale.

Organizations with high data maturity that embed an AI+ transformation model into the enterprise fabric and culture can generate up to 2.6 times higher ROI.

IBM has developed AI+ Enterprise Transformation to equip clients with the business and technical strategy, architectures, roadmaps and hands-on experience to become an AI+ enterprise.

AI+ Enterprise Transformation

With IBM’s depth in AI and hybrid cloud, we have discovered that companies becoming an AI+ enterprise leads to faster realization of business results. What’s exciting is that many clients we work with are already excelling in AI, and by adopting AI+ Enterprise Transformation, they uncover activities that accelerate their business growth through running AI in production at scale.

Figure 3 summarizes AI+ Enterprise Transformation, highlighting the multiple domains across the organization that an AI+ enterprise needs to address to bring AI to production at scale:

• Key use cases that enhance business performance
• Responsible AI technology to implement these use cases
• A well-designed data foundation to fuel AI initiatives
• Application innovation to deliver AI experiences, and application modernization to handle AI requests
• Hybrid cloud platform, including integrations, to run AI, data and applications as required
• Building pipelines for continuous updates, enhancements and fixes of apps, data and AI, with deployment protection through scanning and guardrails
• Day-2 operations using AI to predict (and repair) failures before they happen, fostering a culture where employees embrace AI’s value instead of fearing replacement.
• Security, governance, risk and compliance mechanisms are essential not only for governing AI but also for managing the IT estate running AI, providing evidence for regulatory compliance.

Start with the use cases

The most important step for an AI+ enterprise is identifying transformative use cases. After experimenting with various options, the enterprise selects high-value use cases that show faster ROI. It then delivers them into production across the IT landscape, laying the groundwork for additional use cases and fostering ongoing innovation.

Figure 4 illustrates the AI+ use case funnel that an AI+ enterprise adopts to systematically and rigorously transform use cases into broad-reaching AI enterprise solutions that deliver high ROI, aligned across delivery, operations, security and governance.

Harness the right AI technology

After identifying use cases, the next step for an AI+ enterprise is choosing the appropriate AI technology and architecture. Often, this decision is made too quickly. It should be approached thoughtfully to help ensure suitability.

Consider the following:

• Do you need a public foundation model?
• Should you build your own? If so, where will it run?
• Should you use a retrieval augmented generation (RAG) model by pairing your data with a public foundation model?
• Do you use gen AI out of the box? How can you master prompt engineering? When should you prompt-tune or fine-tune?
• Which approach requires on-premises GPUs?
• Where do you harness gen AI vs. predictive AI vs. AI orchestration? For instance, when automating password change requests, do you need a 175 billion parameter public foundation model, a fine-tuned smaller model, or AI orchestration to call APIs?

As you pinpoint your AI technology, your decision impacts the other domains of AI+ Enterprise Transformation. For more insights, keep reading.

Deliver a strong data foundation

AI relies fundamentally on data. An AI+ enterprise ensures that the data used for AI is trustworthy, transparent, and has clear lineage and efficacy. Otherwise, the risks become too significant. We have all seen examples of companies delivering AI built on weak data foundations, leading to undesirable outcomes. These outcomes typically fall into one of three categories, none of which are desirable:

• Not useful: Customers remain unimpressed with your results. For example, stale data, hallucinations and more.
• Embarrassing: Offensive output emerges based on the data used in AI. For example, hate, abuse, profanity and bias.
• Financial/criminal: Violations of existing and emerging data and AI regulations. For example, copyright laws, the European Union’s Artificial Intelligence act, Digital Operational Resilience Act (DORA), data sovereignty laws and more.

An AI+ enterprise empowers architects to confidently source, prepare, transform, protect and deliver data to the required locations for AI.

Innovate and modernize applications

Innovating with new AI-based applications to deliver outstanding experiences is essential. It’s also crucial to modernize existing applications that interact with AI. if an AI-powered human resources assistant offers to perform actions for employees, it is vital to ensure that the application being called can handle increased traffic. Frequently, these actions involve calling APIs to legacy applications running on architectures unfit for handling the sudden demands of the AI assistant. This often leads to a disappointing experience due to slow response times.

An AI+ enterprise excels in delivering innovative AI applications to its customers and modernizing existing applications to meet the new demands AI presents.

Hybrid Cloud Platform

Once AI, data and applications are understood, the discussion naturally shifts to “Where do we run this solution?” In our experience, the answer depends on many factors, which can change over time, requiring a flexible platform.

Adopting an open technologies-based hybrid cloud platform enables an AI+ enterprise to make informed decisions without limiting its business.

As shown in figure 5, a hybrid cloud architecture enhances the entire business in various ways:

• Flexibility in where to train and tune large models
• Flexibility in where to train and tune smaller models
• Where to perform inferencing on-premises, in private clouds or even on edge devices
• Applications using RAG architectures experience less latency when running close to the models
• Data sovereignty laws limit data relocation, so having the ability to move AI and applications to the data is essential
• Creating an AI+ fabric that provides interconnectivity across the IT and business landscape

Continuously build and enhance apps, data and AI

When AI, data and applications run across a well-designed hybrid cloud platform, an AI+ enterprise builds pipelines and toolchains to continuously enhance and deliver with full automation. For example:

• Platform pipelines provision and update infrastructure and the software running on them using Terraform and Ansible
• Application pipelines integrate and deliver code updates for both innovative applications delivering AI experiences and modernized applications being acted upon by AI digital workers
• Data pipelines process incoming data to help ensure that the data sources used by AI are current and valid
• AI pipelines pull in data, retrain and augment as needed based on metrics such as drift and accuracy

An AI+ enterprise knows how to continually enhance applications, data and AI models throughout their lifecycle, helping to ensure that only trusted and approved AI functions go live.

Operations

Incidents occur, even in an AI-first world. However, an AI+ enterprise uses AI not only to delight customers but also to solve IT problems. With the right tools, an AI+ enterprise can significantly increase employee productivity. Examples include:

• Detecting and correcting when an application is being constrained and automating capacity increases
• Providing visibility and insight across the enterprise to enable higher levels of automation and achieve predictive maintenance
• Reducing security threats by closing security gaps before they arise. For example, by using compliance control scanning of terraform templates to fail provisioning if controls are not met.

An AI+ enterprise also recognizes that alongside the necessary tools, fostering a culture that embraces AI and trains talent is crucial. This culture encourages experimentation and expertise growth. It requires people trained to harness, evaluate, and accelerate AI, rather than fearing it.

Secure and govern AI on the hybrid cloud platform

To deploy AI, particularly gen AI, at scale in production, organizations must establish a secure and governed environment. The scale and impact of next-generation AI emphasize the importance of governance and risk controls. An AI+ enterprise mitigates potential harm by implementing robust measures to secure, monitor and explain AI models, as well as monitoring governance, risk and compliance controls across the hybrid cloud environment.

Pairing existing cloud governance with new AI governance controls is essential, requiring continual focus to comply with emerging regulatory changes, such as NIST AI Risk Management Framework, the European Union’s Artificial Intelligence act, ISO/IEC 42001 AI Management, and ISO/IEC 23894 AI Risk Management.

Get Started Today

IBM wants to work with you to become an AI+ enterprise, providing impactful use cases, strategies, architectures and hands-on experiences to:

• Understand your current trajectory
• Shape your AI+ enterprise strategy with points of view and models
• Co-create target state solutions and architectures
• Specify governance and risk posture
• Customize a business value case
• Co-develop key near-term engineering sprints to prove the value of your AI+ enterprise strategy

Future articles will delve deeper into each AI+ domain, showcasing IBM’s perspective through architectures, demos and strategies.

Originally published at https://www.ibm.com.

How to Become an AI+ Enterprise was originally published in AI Enterprise Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.