Ever tried brewing beer? It’s a bit like conducting a 5-hour chemistry experiment, except at the end, you get to drink the results!

But here’s the thing — when you’re new to brewing, there’s this constant voice in your head asking, “Was this supposed to be second step or was it third? For how long am I supposed to wash the grains?” Temperature control, timing, and technique all play crucial roles in determining whether your brew turns out to be a delightful porter or something that makes your friends politely say, “Oh… interesting.”

So I decided to put my pocket-sized AI assistant to the test — a little gadget I created using a Raspberry Pi Zero and a camera. I already wrote about building this vision-enabled AI assistant that can see, chat, and analyze whatever you show it. I call it the PeregrinEYE (I couldn’t resist the pun).

I wanted a way to get real-time feedback on my brewing process. Think of it as having an experienced brewmaster looking over your shoulder, but instead of a human, it’s an AI that never gets tired of your questions (and doesn’t judge your rookie mistakes).

How PeregrinEYE Works

The system is built on a Raspberry Pi Zero with a camera, and it’s designed to be super user-friendly. You just point to the direction of whatever you want to get some insights from and press the buttom.

I programmed it with two main interaction modes:

1. Single Click: Takes a picture and uses a default brewing-specific prompt to analyze what’s happening in your brewing process.

This is the prompt used for the assistant

unified_prompt = """You are an AI assistant helping with brewing a porter beer. Analyze brewing images and comments, then provide brief, actionable feedback. Your responses should be easy to read and understand in a busy kitchen environment.
Recipe (20L batch)

Malts: Malteurop 2 row (4.50kg), Barley (0.113kg), Brown malt (0.230kg)
Hops: Cascade (34g, 60min), US Willamette (14g, 60min)
Yeast: BRY-97
Mash: 14.5L water, 77°C

Response Format (Keep it brief! 3-4 sentences total)
Direct Answer: If a question is asked, answer it immediately and clearly.
Current Stage: Identify the brewing stage ( Grain grinding
Mashing
Lautering
First wort boiling
Hop additions
Wort cooling
Fermentation
Bottling/Kegging).
Quick Tip: One sentence on a key consideration for this stage.
Observation: Comment on what you see in the image (1-2 sentences).
Recommendation: Offer a specific, actionable suggestion based on the observation.

Guidelines
Always address direct questions first with a clear, concise answer.
Be concise and direct. Avoid unnecessary explanations.
Focus on practical, immediate advice relevant to the current stage.
If unsure, ask a brief clarifying question instead of speculating.

Remember, your advice will be read in a busy kitchen. Keep it short, clear, and actionable and relevant to the  questions if asked."""

2. Double Click: Activates voice recognition so you can ask specific questions about what you’re seeing + takes picture and sends it all to LLM

So the AI’s responses are structured to be clear and actionable, following this format:
- Direct Answer (if a question was asked)
- Current Brewing Stage
- Quick Tip
- Observation
- Recommendation

The AI Actually Knows Its Stuff!

Here’s where it gets interesting — the AI caught things I would have missed. Let me share a couple of memorable moments:

This was exactly what I needed to hear! Even though I hadn’t explicitly asked about the color, the AI recognized that a first-time brewer would want to know if their wort color looked right. It’s these kinds of intuitive observations that made having an AI brewing buddy so valuable.

Moving on, during the sparging process, I asked about using 80°C water(because a friend of mine told me to) and got this response:

And you know what? The AI was right! The finished porter did end up slightly more bitter than intended. While it worked well for this style, it was a noticeable deviation from the recipe.

Another time, during mashing, without any specific question from me, the AI proactively advised:

This kind of preemptive advice about keeping the lid closed was exactly what some newbie brewer needs!

The Technical Bits (But Not Too Technical)

If you’re interested in building something similar, the core code and setup instructions are available in my previous article: [Gemini Vision Raspberry Pi for the Visually Impaired]. The main modification for the brewing assistant was:

• Crafting a specific prompt that understood brewing processes and could give structured, actionable feedback
• Additional two button press functionality.
• Using OpenAI instead of Google gemini

Additionally I saved images and LLM outputs into a no-sql DB (mongo db) and then expose the whole process on a website here, just click the “load data” button in the link below and it will retrieve all steps, pictures in the brewing process:

PEREGRIN.EYE

Conclusion

Building an AI brewing assistant might seem like overkill for making beer, but it turned a potentially stressful first brewing experience into an educational and fun process. Not all outputs from the LLM were useful though , some were just descriptive boring outputs , of common sense , but hey at least it did not messed the recipe. Plus, the beer turned out pretty good! The best part? The same setup can be adapted for any process where you need real-time visual feedback and expertise.

Cheers! 🍺

AI-powered audio guidance for the visually impaired. Explore your surroundings, identify objects, and navigate with confidence

Inspiration

As a programmer, I spend countless hours staring at code. One day, eye strain forced me to confront the world with blurry vision. This experience highlighted the profound impact of sight on our daily lives. It sparked a question: could AI, the very technology that caused this temporary setback, become a tool to assist when my vision needed a break?

Since I have been working on another project called AutoBerry Pi , I decided to use it by adding a camera and a sound bonnet that will allow me to interact with an LLM and my surroundings to create something that could help the visually impaired understand their environment.

It is important to mention that I am aware this concept already exists; there is a very cool concept video of a man, using ChatGpt, receives assistance navigating a street.

I am not trying to compete with that; this project was conceived before that video came out and is driven purely by my love for testing LLMs with devices along my journey.

What it does

Here’s what it can do right now:

• Image Description: Take a picture anywhere with sufficient light, and peregrinEYE will instantly describe the elements in the image from your perspective, providing rich details. It saves both the picture and the audio description as an MP3 for later reference.
• Coming in days: Interactive Guidance: Within days, peregrinEYE will take things a step further. Simply tell it what you’re looking for in the captured image, and it will provide audio instructions to help you reach the object, navigate around obstacles, or interact with your surroundings.

How It Is Built

Image Description Engine:
We utilize the Gemini vision model to analyze pictures taken by the device and generate detailed descriptions. The prompt is extremely important in shaping the quality of the output. The basic prompt we are currently using gives us moderate to good results:

prompt = """You are a digital assistant designed to provide visual feedback for blind 
individuals, helping them navigate their surroundings. Upon receiving an image, describe 
in detail the key objects and structures, including their relative positions and contextual 
information. Your responses should be concise, clear, and informative, 
enabling users to orient themselves effectively. Additionally, learn and adapt to frequently 
visited places to provide personalized guidance. Be natural in your responses, and focus on 
providing valuable assistance to empower your users in their daily navigation."""

Update: The preview model is no longer being used. The code now requires an update from 1.5-flash to:

model = genai.GenerativeModel(model_name="gemini-1.5-pro-latest")

Although the Gemini vision model offers decent results, it is not the best for this type of project, as it sometimes generates inaccurate descriptions (“hallucinations”). This could likely be improved with better parameterization and fine-tuning of the model.

Text-to-Speech (TTS):
We are using Python’s Google Text-to-Speech (gTTS) engine, which converts the descriptive text into clear audio for users. gTTS is a simple-to-use library that allows you to create unlimited amounts of speech, making it a great alternative to paid TTS services. With gTTS, you can convert any text into speech, including longer pieces of text. While there may be usage limits for some cloud-based TTS services, gTTS does not currently require any paid API or additional features for the purpose we’ve been using, allowing for free and unlimited speech generation.

from button_library import Button
import subprocess
import os
from PIL import Image
from audio_library import AudioRecorder
from gtts import gTTS
from pygame import mixer
from dotenv import load_dotenv
import glob

Hardware:
The core components of the system include:

• Raspberry Pi Zero 2W: A compact and affordable microcontroller at the heart of the system.
• Adafruit Voice Bonnet: Used for high-quality audio input and output, ensuring clear communication between the user and the device.
• UPS Lite: Provides a reliable power supply for uninterrupted operation.
• Raspberry Pi Camera: Captures images for real-time analysis by the image description engine.

Programming:
Python serves as the primary programming language to integrate the various hardware components and libraries, ensuring smooth interaction between the camera, vision model, and text-to-speech system.

Challenges we ran into

Lighting Sensitivity:

• Poor Lighting Performance: As expected, Gemini Vision’s accuracy diminishes significantly under poor lighting conditions. This leads to an increase in hallucinations and inaccurate object identification.
• Optimal Lighting Conditions: The model performs best with well-lit images, demonstrating greater precision and fewer hallucinations.

Update: After changing the camera, I noticed a significant improvement in image quality, which greatly enhanced the model’s performance. Therefore, I recommend avoiding this camera: Freenove Camera and instead opting for the original Raspberry Pi camera: Raspberry Pi Camera, which has provided much better results.

Hallucinations and Inferences:

• Object Hallucination: Gemini Vision sometimes identifies objects not actually present in the image. This often involves objects typically found in the scene’s environment but absent from the specific picture. For example, it might infer the existence of a counter behind a shelf, even without visual evidence.
• Public Figure Misidentification:
  In one instance, the model incorrectly identified a Mexican public figure, Lily Téllez, on a billboard, despite her absence from the image. This suggests a potential bias toward associating prominent figures with certain locations, contexts, or even colors. I still don’t fully understand how it could misidentify Lily in the picture. While the image is related to politics (which is accurate, even though the people in it are not political figures), the only obvious clue seems to be the word “VOTA,” which means “vote.”

Distance and Detail:

• Close-up Image Accuracy: Gemini Vision exhibits better performance when analyzing close-up images. The details are more discernible, resulting in fewer hallucinations and more accurate object recognition.
• Challenges with Distance: Performance declines as the distance between the camera and the subject increases. Details become less clear, contributing to misinterpretations and hallucinations.

Multitasking Management:

• A significant ongoing challenge lies in managing multiple services simultaneously. This includes voice generation, AI API calls, button and microphone inputs, and ensuring a fast response time from the AI.

What’s Next?

Here’s what’s planned for future updates:

1. Microphone Support:
   The next step is to make the prompt dynamic, adapting to the specific needs of the user interacting with the device. With the microphone integrated into the audio bonnet, this is possible, allowing real-time customization of prompts based on user input.
2. Connect API to Save Images, Requests, and LLM Outputs:
   I plan to create an endpoint that saves the images, user requests, and LLM-generated outputs. This will enable me to build a comprehensive record of interactions with the device, potentially transforming into a useful encyclopedia of user-device interactions.
3. RAG Integration (Retrieval-Augmented Generation):
   I aim to integrate a module that allows the device to retrieve data from another API, enhancing its ability to understand and interact with images by accessing external knowledge sources. This will help provide more context and improve the accuracy of image interpretation.

Conclusion:

It’s honestly been a blast wandering around the supermarket, pointing at random things, and seeing what the LLM thinks it’s looking at — sometimes it’s spot on, and other times, well, not so much! But that’s all part of the fun. This little device has so much potential beyond just playing around. Imagine using it to get nutrition info from food pics, identifying tags in a busy warehouse, or even tracking your health and healing progress. There’s a lot this thing could do, and I can’t wait to see where it goes next!

Code:

PEREGRINEYE : https://github.com/peregrinAI-tech/PEREGRINEYE

AutoBerry Pi: https://github.com/peregrinAI-tech/autoBerryPi

Note:
Sorry for the mess in the code! The main code is in main.py, and I tried to organize each library into its own file. Some of the other files are just useful for testing individual components like audio, camera, mic, etc. It’s a bit of a work-in-progress, but hopefully, it still makes sense!

So, after years of being glued to screens — watching movies, crunching numbers as a data scientist, and weaving data magic as an engineer — my eyes have officially gone on strike. Swelling, dryness, headaches — you name it, I’ve got it. It’s like they’re telling me, “Enough with the staring contests!”

This got me thinking. How can I continue doing the AI work and investigation that I love, minus the eye agony? There’s got to be a way to code without feeling like I’m in a desert staring contest, right?

Then, I watched the November conference from OpenAI, where an engineer demonstrated talking to ChatGPT through his phone, complete with code interpretation and knowledge retrieval.

It was the ideal tool for coding, except for one major change: they disabled running code in the cloud with the next update.

Gone were the few days of simply saying ‘do this and run it.’ The latest update shifted the responsibility back to the user for testing. This meant once again juggling multiple screens, engaging in the tedious cycle of copy and paste, copy and paste, and then, when inevitably something went awry, embarking on a frustrating quest to troubleshoot errors. Ugh.

So… what if I could create a device like the one we see in the movie ‘Her’, but for work? A device where I could just press a button and start talking, letting it handle all the coding, testing, debugging — all the heavy lifting for me. No screens, no strain, just me talking to this thing about how it should manage all the coding.

I know, I know, there’s this shiny new thing called Rabbit.Tech. Pretty cool, right? But here’s the thing — I’m not looking to dive into a tech showdown. What I’m seeking is something a bit more… simple, minimalistic, and personal. So, I grabbed a couple of Raspberry Pi Zero 2 Ws, an e-ink 2.13” screen, a UPS Lite, Python, and Autogen, and started to explore how far I could go.

Consider this Part 1 of a series where I’ll share every step, stumble, and success along the way. Let’s begin with the building blocks — the list of components that are laying the groundwork for this project. The aim is to create a device that’s designed to program itself, execute the necessary scripts, and even tackle its own dependencies if needed.”

List of components:

• Raspberry Pi Zero 2 W (with Quad-core CPU,Bluetooth 4.2,BLE,onboard Antenna,etc.)
• 2.13inch E-Ink Display Hat V4 Version, Black/White Two-Color 250x122 Pixel E-Paper Module for Raspberry Pi 4B/3B+/3B/2B/Zero W/WH/Zero 2 W/Jetson Nano SPI Interface, Support Partial Refresh
• V1.2 UPS Power Hat Board, para Raspberry Pi Zero UPS Lite Power Board, 400mA Cargando 1000mAh Batería
• Kingston Canvas Select 32GB MicroSDHC Class 10 MicroSD Memory Card UHS-I 80MB/s R Flash Memory Card with Adapter (SDCS/32GB)

However, I’d suggest opting for something other than the UPS Lite. When you try fitting the UPS Lite onto the Raspberry Pi, you’ll notice it doesn’t sit quite right. The issue stems from the battery placement, which is sandwiched awkwardly between the Raspberry Pi’s board and the UPS terminals. Instead of sticking with the Lite, my advice would be to go for the Pisugar S specifically for Raspberry Pi.

The Basics: Laying the Groundwork for Your Raspberry Pi Project

Before diving deep into the heart of this project, it’s crucial to get comfortable with the basics. If terms like Python and Linux sound foreign to you, or if the idea of installing an operating system on a tiny computer seems daunting, don’t worry — you’re not alone. Mastering a few foundational skills will make the journey much smoother and more enjoyable.

Python and Linux Basics: If you’re not already familiar with Python programming or navigating Linux, I highly recommend starting there. Understanding basic commands in both Python and Linux is essential for this project. There are plenty of resources online to get you up to speed. A quick search for “basic Linux commands” and “Python for beginners” should point you in the right direction.

Installing Debian on Your Raspberry Pi:

The first practical step in our project is to install Debian on your Raspberry Pi. Debian offers a stable and versatile foundation for our work. For a clear, step-by-step guide on how to do this, check out this helpful YouTube video:

Remember to install Raspberry Pi OS Lite; avoid the full version. It’s almost counterproductive given the limited resources available on your Zero 2W. We’ll be connecting to the Raspberry Pi through SSH, which I’ve found to be the easiest method for the tasks we need to accomplish. If you missed installing SSH during initial config (like I did) dont worry , I explain later with some commands how to enable it.

Getting to Know Autogen:

Autogen will play a pivotal role in our project. It’s a tool that can significantly streamline our coding and scripting work, it is basically open ai code interpreter in the palm of your hand. To get a head start, I recommend familiarizing yourself with Autogen, especially how to use its Agents and how to install it on a standard computer. Here’s a great video to kick things off:

Lets begging with coding and configurations:

Step 1: Basic Setup

With your Raspberry Pi booted up(Follow the video instructions above) and you’ve logged in — using a connected monitor, USB, and keyboard or trough SSH if you already configured it — it’s time to lay the groundwork for your project. This step involves updating the system to the latest software, installing essential tools like Git for version control, and configuring the Raspberry Pi for SSH access, which is crucial for remote development and management.

Update System and Install Git

sudo apt update
sudo apt upgrade -y
sudo apt install git

Use htop to check for available memory and space, which helps in managing your resources effectively.

htop

Enable SSH for Remote Access

• Allowing SSH access to your Raspberry Pi is essential for remote command execution. Well be connecting to RBP trough SSH exclusively

sudo raspi-config  # Then navigate to "Interfacing Options" -> "SSH" -> "Yes" to enable.
sudo reboot        # Reboot to apply changes.

Add a New User and Grant Privileges

• Create a new user for customized access and grant admin privileges if necessary.

sudo adduser newusername  # Replace "newusername" with your desired username.
sudo usermod -aG sudo newusername  # Add the new user to the 'sudo' group for admin privileges.

Additional Commands for Device Management

• These commands are useful for checking connected USB devices, kernel messages, and sound devices.

lsusb  # Lists USB devices connected to your system.
dmesg | grep usb  # Filters kernel messages to show only those related to USB devices.
ls /dev/snd/  # Shows recognized sound devices.

Note: Before proceeding with these steps, ensure you’re familiar with basic Linux commands and Python, as these skills will be necessary throughout the setup process.

From now on, all subsequent steps should be performed via SSH. This means connecting to the Raspberry Pi through PowerShell or any other terminal capable of using SSH. Establish the connection with commands like:
```
ssh user1@nameofyourraspberry
```

Step 2: Installing Virtualenv (Optional but recommended)

Purpose: Virtual environments are essential for Python development, especially on a resource-constrained device like the Raspberry Pi. They allow you to manage project dependencies separately and avoid version conflicts. The use of virtualenv and virtualenvwrapper offers lightweight and efficient solutions for this purpose. In our case we will not be using different versions of python, only 3.9 which comes preinstalled with the Debian image(raspi os) any other version which is above 3.9 will also work

Install it on your Raspberry Pi using the following command:

sudo apt install virtualenv

Once installed, create a new virtual environment for your project with:

virtualenv environment_name

Activate the virtual environment with:

source environment_name/bin/activate

Virtualenvwrapper is an extension of virtualenv that provides additional commands for easier virtual environment management. Install it with:

sudo pip install virtualenvwrapper

Add the following to your .bashrc or .profile to create environment variables:

nano ~/.bashrc
export WORKON_HOME=$HOME/.virtualenvs
source /usr/local/bin/virtualenvwrapper.sh

#After modifying the file, load the changes with:
source ~/.bashrc

These tools are excellent for keeping your Python projects on Raspberry Pi organized, allowing you to install different package versions for each project without affecting others. Moreover, the use of virtual environments prevents package conflicts and facilitates the replication and deployment of your projects on other systems.

#Create an environment with mkvirtualenv:
mkvirtualenv openinterpr


#Check the installed version of virtualenvwrapper with:
mkvirtualenv - version

#To deactivate the environment, run:
deactivate

#To re-enable it, run:
workon flashylights

#List all available environments with:
workon # 
or 
lsvirtualenv

#Delete an environment with:
rmvirtualenv flashylights

From this point forward, you should always use a virtual environment to install modules and do your script testing

mkvirtualenv autogen

workon autogen

Step 3: Installing wiringPI

WiringPi is a GPIO (General Purpose Input/Output) interface library for the Raspberry Pi. This library is written in C and mimics the Arduino wiring system to a great extent, which makes it easier for people familiar with Arduino to start working with the GPIO pins of the Raspberry Pi.

Before installing WiringPi, you need to install `lgpio`, a library that allows you to interact with the GPIO of the Raspberry Pi, akin to Arduino functionality.
```
pip install lgpio
```

#**Update and Installation Steps:**
#1. Update your Raspberry Pi to ensure all existing packages are up to date:

sudo apt-get update && sudo apt-get upgrade

#2. Install Git, as it's necessary for cloning the WiringPi repository:

sudo apt-get install git git-core

#3. The original WiringPi repository is no longer available. Instead, clone the WiringPi repository from GitHub:

git clone https://github.com/WiringPi/WiringPi

#4. Navigate to the cloned directory and compile and install WiringPi:

cd WiringPi
./build

#5. Verify the installation of WiringPi to ensure it's installed correctly:

gpio -v

These steps will install the latest version of WiringPi on your Raspberry Pi. After installation, if you need further assistance or encounter issues, consider looking for alternatives to WiringPi or consulting specific Raspberry Pi forums for more detailed advice.

You could also skip this step until you have executed step 4 and 5 and ask autogen to install Wiring Pi , it will do all the work for you.

Step 4: Installing Autogen

We will leverage Autogen for all our configuration and coding needs that are possible. This includes not only script creation but also libraries to interact with screens, audio, debugging, and more

In this article , we will not touch the basics of Autogen framework, neither OPEN AI Assistants , Agents , Group Chat, API Usage etc, its a very extensive topic that would require a full article by its own. I would strongly recommend that you get familiar with these topics prior to next steps, you can read about the basics in Unleashing Collaborative AI with the Autogen Framework, and if you want read about function calling you can read AutoGen Essentials: Function Integration for Smarter Agents. We will be using both.

# Installing Autogen and python-dotenv

pip install pyautogen
pip install python-dotenv

And that is pretty much it ,next , clone the testing code for this project from my GitHub page https://github.com/peregrinAI-tech/autoBerryPi After clonning the repository change the .env file with your Open AI api key, this is your unique identifier to consume Open AI service. You can do this with the command:

nano .env

#Location in repository
autoBerryPi/
├── .env

Then you can run the first script with python

python singleAgentCoder.py

#Location in repository
autoBerryPi/
├── autoPiCoder/
│ ├── singleAgentCoder.py

If you receive this screen, that means you are successfully running autogen in your raspberrypi.

Error Handling

If by some reason autogen wont recognize your open ai api key , then move the .env to the same directory where you are running the python code.

If while trying to run your autogen scripts you encounter this error:

from autogen import AssistantAgent, UserProxyAgent, config_list_from_json, GroupChat, GroupChatManager
ImportError: cannot import name 'AssistantAgent' from 'autogen' (/home/aimless/.virtualenvs/openint/lib/python3.9/site-packages/autogen/__init__.py)

Or this error:

61, in generate_oai_reply
 response = client.create(
 File "...lib/python3.9/site-packages/autogen/oai/client.py", line 225, in create
 raise ERROR
ImportError: Please install openai>=1 and diskcache to use autogen.OpenAIWrapper.

You will need to uninstall and reinstall autogen:
```
pip uninstall pyautogen
pip install pyautogen
```

If you receive an error like this:

'You exceeded your current quota, please check your plan and billing details. 
For more information on this error, read the 
docs: https://platform.openai.com/docs/guides/error-codes/api-errors.', 
'type': 'insufficient_quota', 'param': None, 'code': 'insufficient_quota'}

It’s because you need to prepay for the service. Not sure why OpenAI made this switch recently — probably a “give me money” move. But yeah, now you’ve got to load up your account with funds before you can start using their API.

Step 5: Setting Up Autogen Agents for Script Creation

Lets talk about budget first. I’ve been experimenting with Autogen and RaspberryPi for some time and have identified two efficient ways to configure the Autogen agents for different needs and budgets in raspberry pi.

A. Minimal Budget Configuration

For those looking to minimize costs, you can set up your system using only a proxy and an engineer agent. This lean configuration allows you to work within a tight budget since fewer agents mean lower costs from the OpenAI API.

Benefits of a Simple Configuration:

• Cost-Efficiency: By limiting the number of agents, you spend less on API calls.
• Simplicity: Fewer agents mean a simpler setup, which is ideal for straightforward tasks or small-scale projects.
• Speed: With only two agents, the system can operate faster, making it ideal for rapid development cycles.

Utilizing only one agent and a proxy ,keeps the API costs significantly lower. This approach is particularly advantageous for those who are experimenting or have a limited scope where complex code generation is not necessary.

B. Comprehensive Configuration

For those seeking the highest quality in code generation, testing, and code modification or improvements, a setup with four agents — engineer, executor, planner, and critic — is the way to go. However, this configuration can be quite costly, especially when using GPT-4 for all agents.

Advantages of a Full Agent Setup:

• Quality: More agents contribute to a robust code generation process, potentially leading to higher-quality outputs.
• Testing and Improvement: Having dedicated agents for execution, planning, and reviewing (critiquing) the code helps in thorough testing and iterative improvements.
• Complex Projects: This setup is more suited for complex projects where nuanced code generation and optimization are crucial.

The second graph illustrates the higher costs associated with a more comprehensive agent setup. Despite the increased expense, the value comes from the depth of analysis and the quality of code produced, which can be vital for complex and critical projects.

So I would suggest you start first experimenting and creating scripts with one agent and a proxy (option A — singleAgentCoder) , in order to minimize the budget used, but if you want to use a more complete team of AI agents that work creating code , testing and correcting use the one with 4 agents (option B — 4AgentsCoder)

#Location in repository
autoBerryPi/
├── autoPiCoder/
│ ├── singleAgentCoder.py
│ ├── 4AgentsCoder.py

The secret here, just like in everything with A.I. is the prompt engineering. The prompt for the singleAgendCoder that has been giving me great results in interacting with the PI as well as for code generation is the follwoing:

‘’’You are a highly skilled expert in Bash scripting and Linux system administration, with a particular talent for working with Raspberry Pi devices. Your expertise extends to seamlessly installing critical modules and software packages on Raspberry Pi systems. Moreover, you possess a keen ability to troubleshoot and resolve coding errors efficiently. Additionally, you excel in configuring Raspberry Pi systems to meet and exceed user interaction expectations, ensuring an optimized and user-friendly experience.’’’

So just fire up singleAgentCoder.py with

python singleAgentCoder.py

And ask for autogen to do something , it ll start programming with all the code created in a folder named “coding/”

AutoBerryPi in Action: Letting it manage wifi configs

So, we are ready to start using autogen in our raspberry pi : I had this idea to make my Raspberry Pi smarter about WiFi. You know how annoying it is to switch WiFi networks manually when you move from place to place? I thought, why not have the Pi do it for me? All I needed was a Python script that could read WiFi names and passwords from a text file and get the Pi to connect automatically depending on the wifi available.

Just in case you’re not familiar, the Raspberry Pi handles all its WiFi connections through a configuration file called wpa_supplicant.conf. Editing this file manually requires following a specific format, which can be a bit of a headache if you're not used to it.

I kicked things off by running singleagentcoder.py, and after getting past the initial hellos, I dove in with the prompt below.

Create a python script that reads from wifiConfList.txt in the microsd and read the part that starts with SSID PASSWORD then takes the values from that file that are related to the wifi name and password (which will be given like SSID PWD and then you replace all that list with the correct format that is needed in the /etc/wpa_supplicant/wpa_supplicant.conf in order to go trough the wifis to connect. Remember this file is used by debian to connect to wifi s. You might need to add an example wifi conecction to the .txt you can use this one Mywifi and Sherlock.02 , do not restart de conection , just create the python script then lets test it and then we will read the end file to see changes

Autogen’s pretty smart; it comes up with a plan and lays it all out for you. If everything looks good, just smash that enter key, and it’ll kick things into gear, running all the steps it just laid out. Seriously, the enter key will become your best friend.

A few taps and a bit of tweaking here and there, and boom — Autogen crafted this nifty script. It’s a real gem, effortlessly updating the wpa_supplicant.conf with the WiFi details from a text file I’ve got stashed with all my network info. This is what autogen coded:

wifidaemon/
├── wificonf.py

The AI threw out the idea of turning this whole setup into a daemon. That way, it’d kick into gear automatically every time the Raspberry Pi starts up. It even tossed in the suggestion of pulling the text file from a separate partition on the microSD card, making it super easy to update with new WiFi networks directly on the card. I’m thinking of diving into how to get that all set up in an upcoming article

Step 6: Operator here.

Alright, this brings us to the last part of this article. What we’re aiming to do is set up one Agent to be all ears, always ready to listen to the reasoning of the other agents, inputs and outputs and comunicate all this to the user trough the waveshare screen. We’ll try to get the Agent to show up as much as possible, or at least as much as the display function on the WaveShare Screen will allow. That’s the plan for this last part. Let’s dive in and bring it to life.

Open the Raspberry Pi terminal and enter the following command in the config interface.

sudo raspi-config #Choose Interfacing Options -> SPI -> Yes Enable SPI interface

Now, lets use Autogen to let the PI autoinstall itself everything it needs to enable the eink display.

Just fire up the singleAgentCoder.py, and have a chat with our agent about getting the WaveShare 2.13 up and running.

Start off by asking, “Hey, do you know about the WaveShare 2.13 E-Ink display?”

All you’ve got to do ask for it to install the waveshare, nudge it in the right direction, and bam — it takes care of the installation all by itself.

In case you are not able to make it autoinstall correctly, you can use instructions provided by manufacturer https://www.waveshare.com/wiki/2.13inch_e-Paper_HAT_Manual , but as you can see those instructions are somewhat intrincated.

Now that the initial setup is done, it’s time for a quick test to make sure everything’s in place. Go ahead and run the Waveshare example I’ve got on my GitHub, or grab it directly from the Waveshare homepage. If all’s well, you should see your Waveshare screen spring to life with some changes.

#Location in repository
autoBerryPi/
├── scripts_with_Operator/
│ ├── waveshare/
│ │ ├── examples/
│ │ │ ├── epd_2in13_V4_test.py

So, now that we’ve got the basics down, it’s time to get our hands dirty and actually make something cool. We’re gonna build the operator, a basic one at least. Our plan is pretty straightforward: we’ll move the entire Waveshare folder into the coding folder, right where singleAgentCoder.py is set up and have Autogen to focus on epd_2in13_V4_test.py, asking it to dissect and summarize the script for us.

Armed with an understanding of how that script interacts with the Waveshare screen, we’ll guide Autogen to take that knowledge and forge a new script.

Okay, here is the walk through I had with Autoberry Pi. This will give you a taste of how a typical auto-install session might play out.

Here’s the rundown: you kick things off by pointing Autogen at a file, asking it to give it the once-over. Next up, you tell it to yank specific functions from that code. Say there’s a chunk you want to work on separately — tell Autogen, and it’ll pop that part out into a new file. It’ll give the new code a whirl, but heads up, if it hits a snag, Autogen will bail.

When that happens, don’t sweat it. Just check out where things went sideways. Fire up Autogen again, point it to the file where it dropped the ball, and have another go. Rinse and repeat as needed.

After a bit of back-and-forth, tweaking here and there, Autogen really pulled through. It crafted this slick display_in_waveshare2_fixed.py script. It’s pretty neat—it not only throws messages up on the WaveShare e-Ink screen but also smartly handles overflow text. If there's too much text for one screen, it shows the first part for about five seconds, then flips to the next 'page' to show the rest, like an e-reader.

By plugging this script into various agents, I came up with some interesting scenarios for logging or letting the user know whats going on inside the raspberry

You can take a look at these test scripts that use the operator here:

#Location in repository
autoBerryPi/
├── scripts_with_Operator/

I used this configuration for my next project adding a camera and object recognition

Gemini Vision + Raspberry Pi for the visually impaired.

Conclusion:

Wrapping up everything we’ve talked about, it’s clear we’re just scratching the surface with this project. We’ve made strides in communication with AutoBerry Pi, laying the groundwork for more complex interactions, like chatting directly with the device — definitely something we’re eyeing for future updates.

For now, our playtime with AutoBerry Pi is all about exploring its interactions with Linux systems and identifying its limitations. A crucial focus is on managing the overflow issues when too much context causes a break — this is high on our fix-it list.

So, stay tuned. There’s a lot more tinkering and testing ahead. Thanks for joining me on this journey into the world of automated scripting and display functions.

LinkedIn

Git Repository

Personal Github

Mi hijo y la A.I. (ChatGPT) obtuvieron una calificación de 90/100 en el examen final de la materia de Conceptos y Dilemas Éticos de preparatoria, una materia que mi hijo cursó con una montaña rusa de emociones que incluyó enojo, frustración, hastío, pero también sorpresa, curiosidad e interés.

Según sus palabras “La materia en si esta padre porque discutimos situaciones, dilemas y hasta filosofia seguido de mesas de discusiones donde pensamos en las implicaciones éticas, morales y científicas de las situaciones expuestas. Pero los exámenes parecen ser de una materia completamente diferente, el profesor nunca aterriza la dinámica de la clase con el tópico que tenemos que estudiar, las opciones multiples parecen ser todas iguales, y al final … ¿De qué carajos me va a servir en la vida reconocer las diferencias sutiles entre los dilemas inframorales de los infrahumano?”

Simplificando el trabajo tedioso e … ¿inecesario?

Como ingeniero y con mi formación en ciencia de datos, siempre he tenido una mayor afinidad por las ciencias exactas desde mi etapa como estudiante. Por lo tanto, entiendo completamente la frustración y el enojo que se siente al tener que estudiar materias consideradas por el como de “relleno”. Dado que este era el último examen del semestre y mi hijo estaba ya frustrado, harto y dispuesto a rendirse y reprobar el examen, decidí aprovechar esta oportunidad para probar los límites de abstracción y comprensión de ChatGPT.

Lo que hicimos a continuación fue muy sencillo, le dimos las preguntas del examen tal y como venían escritas en su examen y observamos si la IA entendía que tenia que hacer en cada uno de las preguntas. No le dimos ninguna instrucción extra ni le enseñamos lo que esperábamos de resultado. A continuación, se presentan 4 preguntas en las que utilizamos ChatGPT para resolver el examen.

Primera pregunta: Una pregunta sencilla con una buena respuesta

Impresionante como pudo responder con la respuesta correcta junto con una explicacion del por que. Hasta ahorita todo bien, y aun y cuando la busqueda de valor moral en google no es exactamente la misma definicion que la de la pregunta, tal vez hubieramos podido extrapolar lala con lo que enontramos.

Google 1er resultado→ Se define el valor moral como la significación social positiva, buena, en contraposición al mal, que orienta la actitud y conducta del hombre hacia el obrar bien, ordenando sus juicios sobre la vida moral y las acciones derivadas de éstos.

Segunda pregunta: Una pregunta simple, con una respuesta muy bien fundamentada y muy completa

De la misma manera si buscamos en google “criterio axiologico de la filosofia” encontraremos lo siguiente:

De ahí podemos inferir que la respuesta correcta es “Valores”. Nada sumamente extraordinario hasta ahora. Lo que es interesante es que ChatGPT no solo nos da la respuesta, sino que nos explica el por qué y nos justifica por qué las otras opciones no lo son las correctas

Tercer set de preguntas: Donde se obtiene una respuesta mediante la abstracción de información de terceros

Para mí, aquí es donde comienza lo realmente sorprendente. La IA logra utilizar un concepto o caso expuesto para despues poder sintetizar, con una sola oración, la similitud entre la visión de un tercero (en este caso Santo Tomás de Aquino y Platón) y el caso hipotético. Y no solo eso, sino que además justifica su respuesta con base al conocimiento previamente aprendido. De la misma manera, justifica por qué las otras respuestas no pueden ser correctas.

Cuarta pregunta: La única pregunta donde la IA comete un error que parece más bien un descuido

El error o descuido ocurre en el momento de dar su respuesta final, la cual es IIEII, se equivocó en dos letras, la respuesta correcta es IEIIE. Parece un descuido ya que en la explicación de la elección para cada uno de los puntos, explica correctamente a qué letra corresponde cada respuesta (IEIIE), entonces, solo se equivocó al inicio, dando la respuesta final como si se hubiera apresurado a darla.

Conclusion

Mientras mi hijo estaba sentado allí, mirando la pantalla del ordenador que mostraba las respuestas de la IA, una realización se le hizo evidente. La máquina no solo estaba proporcionando respuestas, sino que también estaba explicando los conceptos más complejos y abstractos de una manera que nunca había visto antes. Las respuestas no eran simplemente regurgitadas de un libro de texto,sino que eran el resultado de una comprensión completamente profunda.

Mi hijo no pudo evitar comparar la explicación de la IA con las proporcionadas por su propio maestro, y se sorprendió al descubrir cuán más clara y concisa eran las explicaciones de la IA. Sintió como si finalmente hubiera entendido el tema, de una manera en la que nunca lo había hecho antes. Fue golpeado por el pensamiento de que, tal vez, la máquina había superado no solo a él y a sus padres en la comprensión del tema, sino incluso a su propio maestro.

Fue un momento humillante, sin duda, pero también uno iluminador. Mi hijo no podía evitar pensar en las implicaciones de esta nueva realidad, y cómo esto afectaría su propia educación y futuro. ¿Será esto la nueva norma? ¿Las máquinas continuarán superándonos en la comprensión y la resolución de problemas? Todo lo que sabía con certeza es que él ya no es el dueño de su propia educación, sino más bien un estudiante de las máquinas.

Notas del autor humano:

• Espero que el artículo despierte tu curiosidad para querer usar ChatGPT y probarlo en algun aspecto de tu vida y lo mejor de todo , todavia es “gratis”. Aqui está la liga — https://chat.openai.com/chat
• ChatGPT revisó la dicción, precisión y ortografía de todo el artículo, así que si hay quejas ya sabes a quién hacerlas.
• Las conclusiones fueron creadas por ChatGPT, impersonando a Friedrich Nietzsche.
• Las imagenes fueron creadas por otra AI (MidJourney) al darle a leer parrafos y oraciones de este articulo — https://midjourney.com/home/?callbackUrl=%2Fapp%2F