TL;DR: Be proactive and master the art of harnessing coding agents.

I sometimes ask my mentee a question during mock interviews: “Prove to me that you’re better than a Mac Mini.”

It sounds harsh, maybe even politically incorrect. But as a startup CTO, I think it’s becoming a very real question. Some of our mentees have gone on to land jobs at places like JPMorgan, engineer in hudge fund, and promising startups, and I genuinely believe the expectations for junior engineers are changing faster than most people realise.

Let me explain why.

My coding setup: 3 machines, 7 coding agents

Right now, I have three Apple machines running coding agents, each with its own responsibilities:

• Mac Mini M4, 16GB: Two Claude Code instances focused on feature implementation and infrastructure improvements.
• Mac Mini, 16GB: Two Claude Code instances handling technical debt cleanup and reliability improvements.
• MacBook Pro M4 Pro, 48GB: Claude Code instances working on features and hotfixes, alongside Codex for automated QA against our development environment.

These agents work in parallel, handling everything from implementing features to maintaining code quality and running tests. The goal is to keep development moving around the clock, even when I’m not sitting in front of a computer.

I manage the sessions through T3 Code. It’s in alpha, still some bugs, but it allows me to check progress, review what agents are doing, and provide instructions from my phone. I can do this on the train, between meetings, or even while having dinner with friends. That’s probably not the healthiest work-life balance, but it’s the reality of building an early-stage startup. As a CTO, work tends to follow you everywhere.

Now think about the economics. A Mac Mini costs a few hundred pounds. Add a Claude Max subscription, configure the environment properly, and suddenly you have a coding agent that can implement features, write tests, fix bugs and work through technical debt. It doesn’t get tired, doesn’t complain about repetitive tasks, and can keep working as long as the environment and task allow it.

This leads to an uncomfortable question: if your primary value as a software engineer is taking a well-defined ticket and turning it into code, what makes you more valuable than that Mac Mini?

I don’t think the answer is writing better code or typing faster. Increasingly, I think it comes down to two skills.

1. Be proactive: see problems before someone assigns them

Coding agents are already remarkably good at implementation, especially when given precise instructions and sufficient context. What they still struggle with is understanding the bigger picture of a product and independently recognising what should be improved next.

For example, I might be looking at our operational dashboard and suddenly think: We’re missing a chart that shows the end-to-end journey of a message. That would make debugging much easier. Or while working with our Telegram integration, I might realise that adding a search tool would give agents much better context from previous conversations.

Another day, I might get frustrated that changing LLM models for different clients requires too many configuration changes. That frustration becomes an idea: Why don’t we introduce model aliases so we can switch the underlying model without touching every client configuration?

These aren’t tasks someone assigned to me. They come from using the product, understanding the architecture, noticing friction and asking whether something could work better.

Of course, you can ask a coding agent to review a system and suggest improvements. Sometimes it will come up with genuinely useful ideas. But someone still needs to understand the business, recognise which problems matter, connect different parts of the product and decide what deserves attention.

That’s what I mean by being proactive. Don’t wait for someone to create a Jira ticket. Explore the product, question existing decisions, identify bottlenecks and propose improvements. The most valuable engineer isn’t necessarily the person who finishes the most tickets, but the one who discovers what the team should be working on.

2. Learn to harness coding agents, not compete with them

Here’s another question I would ask during an interview: “If I give you two or three Mac Minis and expect five good PRs every day, how would you make that happen?”

The answer isn’t simply opening five Claude Code sessions and giving each one a task. Anyone can do that. The challenge is building an environment where multiple agents can work productively without constantly stepping on each other’s toes.

You need to think about how tasks are decomposed, how to isolate coding environments with separate worktrees or repositories, and how to provide each agent with the right context. You also need automated validation: linting, unit tests, integration tests, end-to-end tests and perhaps another agent reviewing the code before a human looks at it.

And what happens when things go wrong? What if two agents modify the same interface? What if one introduces a change that breaks another feature? What if the tests pass, but the implementation fundamentally misunderstands the product requirements?

This is where engineering experience and system design become important. You’re no longer just writing code; you’re designing a development process that allows machines to write code reliably.

I think of this as harness engineering. The coding agent is the engine, but the harness determines how effectively that engine can operate. A powerful model with poor context, unclear instructions and unreliable testing can create more problems than it solves. A well-designed harness, on the other hand, can turn a few relatively inexpensive machines into a surprisingly productive engineering setup.

Five PRs a day sounds impressive, but five poorly designed PRs just create technical debt faster. The objective isn’t to maximise the amount of code generated. It’s to maximise the amount of useful, reliable software delivered.

AI hasn’t changed software engineering. It has accelerated it.

Despite all the excitement around coding agents, I don’t think AI has fundamentally changed the principles of software engineering. Good architecture still matters. So do testing, reliability, maintainability, security and understanding what customers actually need.

What has changed dramatically is the speed at which we can explore ideas, implement solutions and get feedback.

Previously, you might spend several weeks implementing an architectural approach before discovering its limitations. Today, you can discuss multiple approaches with an agent, ask it to compare their trade-offs, prototype different implementations and evaluate the results in a fraction of the time.

But there’s an important consequence that I think we aren’t discussing enough.

Traditionally, junior engineers develop into senior engineers through experience. They write code, encounter problems, make mistakes, debug complex systems and gradually build an intuition for architecture and engineering trade-offs. That experience is what eventually allows them to make good technical decisions.

Now, coding agents can handle much of the implementation work that used to provide those learning opportunities. If junior engineers simply accept whatever the agent produces, they might become extremely productive without developing the understanding needed to become senior engineers.

Being able to generate code is not the same as understanding software engineering.

I think the solution is to become much more deliberate about learning. When an agent suggests an architecture, ask why. Ask it to propose alternatives, challenge its own design and explain what might fail as the system scales. Discuss the trade-offs, experiment with different approaches and compare the results.

Most importantly, make the final judgment yourself and understand the reasoning behind it. Every interaction with an agent should be an opportunity to develop your own engineering knowledge, rather than simply outsourcing another decision.

AI can make you faster, but it doesn’t automatically make you better.

So, can you prove you’re better than a Mac Mini?

I don’t believe junior software engineers are becoming irrelevant. In fact, I think this is an incredibly exciting time to enter the industry. You have access to tools that allow you to build, experiment and learn at a pace that would have been unimaginable a few years ago.

But the bar is changing. Writing code is becoming less of a differentiator, while recognising valuable problems, understanding systems, making sound decisions and orchestrating AI agents are becoming increasingly important.

If I were interviewing a junior engineer today, I wouldn’t be particularly interested in whether they could write Python faster than Claude Code. I’d want to know whether they could take an ambiguous product problem, turn it into a clear technical plan, coordinate a few coding agents to implement it, and take responsibility for the quality of the result.

Even better, I’d want them to come back with an improvement I hadn’t thought of.

So when I ask, “Prove you’re better than a Mac Mini,” I’m not asking you to compete against a machine at writing code. I’m asking whether you can bring the curiosity, initiative and judgment that make those machines genuinely useful.

Because the future of software engineering isn’t about how much code you can write yourself. It’s about how much you can accomplish, how well you understand what you’re building, and how effectively you can turn your ideas into reliable software.

Don’t compete with the Mac Mini. Learn to master it.

Prove You’re Better Than a Mac Mini was originally published in Binome on Medium, where people are continuing the conversation by highlighting and responding to this story.

GitHub - ayomaska18/Smart_Contract_Agent: An AI-powered smart contract development assistant developed using Binome's Graphite workflow framework

Understanding Graphite

Graphite is an event-driven framework that enables developers to create AI agents or workflows composed of modular components, known as nodes and topics. Each node can perform specific actions, respond to events, or communicate with other nodes. Topics are used for message queuing for the workflow, essential for Pub/Sub interactions. Core Features, including observability, idempotency, and auditability, make it straightforward to build, trace, and test workflows and agents with fully customisable functionality.

During my internship, understanding Graphite’s architecture was key to designing assistants that could handle real-world tasks like building a whole smart contract, generating and deploying a workflow, or building entire project structures from natural language queries.

Why Smart Contract Assistant?

When my internship started in mid-June, one of the co-founders and the technology lead at Binome asked me what kind of projects or applications I wanted to work on — whether it was model training or building agentic applications. At that time, Agents, RAGs, and MCPs, all these terms were all completely new to me, so I decided to dive deeper into them. He then tasked me with coming up with a project idea. It could be in finance, healthcare, or any domain that interests me. Immediately, crypto came to mind, which has always been interesting for me. But I wondered: how could I integrate cryptocurrencies with agents? What value could AI bring to the world of crypto? How could I build applications that improve the experience for users, traders, and builders?

The answer became clear: I wanted to build tools and infrastructure that could make blockchain and smart contracts more accessible to people who might not be familiar with crypto. While platforms like Pump.fun allow users to deploy tokens easily, they often lack flexibility and customisation. On the other hand, tools like Remix provide full control for writing smart contracts with languages like Solidity, with manual compilation and deployment of the contract, but they can be intimidating and unintuitive for newcomers. My goal was to bridge this gap — creating applications that combine the power of AI agents with blockchain technology, enabling users to interact with, create, and deploy smart contracts in a more intuitive and guided way without extensive knowledge of coding.

How does the Smart Contract Assistant work?

The Smart Contract Assistant is designed to guide users through the entire smart contract lifecycle, from understanding requirements to deploying contracts on a blockchain. It works as an AI-powered agent composed of multiple LLM nodes and function call nodes, each responsible for a specific task.

Understanding Users’ Requirements

The workflow begins by receiving a user query from the agent_input_topic. Queries can be simple, such as “What kind of smart contract can you generate or deploy?” or “How do I deploy my smart contract to the Ethereum blockchain?” These queries are handled by the reasoning_node. If no special action is required, the reasoning_node directly provides an answer back to the user via the agent_output_topic.

However, for more complex queries like “Can you generate me an ERC-20 token with name Binome, and with supply 10000 with burnable features”, this is where the fun begins, instead of directly going to the agent_input_topic, the reasoning_node will proceed with the request to the ActionNode, which will then call the Required function tools from the MCP server, and return the desired result to the user.

Example reasoning response:

{
  "reasoning": "The user has provided specific details for an ERC20 token: Name: MyToken, Symbol: MTK, Initial supply: 1000000, Features: mintable (which requires ownable). I have all the necessary information to generate this contract.",
  "requires_tool_call": true,
  "tool_call_reasoning": "Need to call generate_erc20_contract with the specified parameters to create the ERC20 token contract",
  "tool_result": null,
  "confidence": 0.95,
  "requires_deployment": false
}

MCP server

Graphite allows nodes to connect to any MCP server with ease, just by using the MCPTool.builder() method to connect to your dedicated MCP server as a tool, and can be injected into any nodes that you wish to call the tools that you defined on the MCP server.

mcp_config: Dict[str, StreamableHttpConnection] = {
        "smart-contract-server": StreamableHttpConnection(
            url=mcp_server_url,
            transport="http"
        )
    }
mcp_tool = await MCPTool.builder().connections(mcp_config).a_build()

When the reasoning_node detects user wanting to generate/compile a contract, it will go down the action route, the action node will prepare the input params for the tool_execution_node to call the MCP tools.

Contract generation:

generate_erc20_contract(contract_name="MyToken", token_name="My Token", token_symbol="MTK", ...)

Contract compilation:

compile_contract(solidity_code="pragma solidity ^0.8.0; contract...")

Human In The Loop (HITL)

Once a user has compiled their contract or provided an already compiled contract and requests deployment to a blockchain (currently supporting Ethereum Sepolia), we will proceed to the deployment route, where deployment_request_node will prepare the necessary deployment details, including gas price, chain ID, constructor arguments, etc. And prepare_deployment_node will be calling the prepare_deployment_transactiontool from the MCP server using the input details, which will send from the prepare_deployment_output_topic to the frontend, and require the user to sign the transaction to pay the gas to deploy the contract.

Once the user has signed the transaction, the signed transaction will be sent back to the assistant through the deployment_approval_topic and the workflow will continue, the deployment_approval_nodewill prepare the input details and call the broadcast_signed_transaction tool to broadcast the signed transaction to the blockchain from the broadcast_deployment_node.

Final Response

The final response will depend on what the user queries. Here are the possible query responses from the assistant:

Greetings/general queries:

{
  "status": "completed",
  "summary": "Hello! I'm your Smart Contract Assistant. I can help you generate ERC20 tokens, ERC721 NFTs, compile contracts, and handle deployments. What would you like to work on?",
  "results": null,
  "next_actions": ["Ask me to generate a token", "Request contract compilation", "Get help with deployment"],
  "artifacts": null,
  "warnings": null
}

Contract generation with tool results:

{
  "status": "completed", 
  "summary": "Successfully generated MyToken ERC20 contract with mintable functionality. The contract includes standard ERC20 features plus minting capability restricted to the owner.",
  "results": "{\"contract_type\": \"ERC20\", \"contract_name\": \"MyToken\", \"features\": [\"mintable\", \"ownable\"], \"solidity_code\": \"pragma solidity ^0.8.19;\\n\\nimport '@openzeppelin/contracts/token/ERC20/ERC20.sol';\\nimport '@openzeppelin/contracts/access/Ownable.sol';\\n\\ncontract MyToken is ERC20, Ownable {\\n    constructor() ERC20('MyToken', 'MTK') {}\\n    \\n    function mint(address to, uint256 amount) public onlyOwner {\\n        _mint(to, amount);\\n    }\\n}\"}",
  "next_actions": ["Compile the contract", "Review the Solidity code"],
  "artifacts": ["MyToken.sol"],
  "warnings": ["Remember to compile before deployment"]
}

Successful deployment with tool results:

{
  "status": "completed",
  "summary": "Successfully deployed MyToken contract! Your ERC20 token is now live on the blockchain and ready to use.",
  "results": "{\"deployment_status\": \"success\", \"contract_name\": \"MyToken\", \"contract_address\": \"0x742d35Cc6538C21f1b2cF9BCA59CDf3f3aDBc123\", \"transaction_hash\": \"0x1234567890abcdef...\", \"block_number\": 18500000, \"gas_used\": \"1205843\"}",
  "next_actions": ["Verify contract on Etherscan", "Mint initial tokens", "Set up token distribution"],
  "artifacts": ["Contract Address: 0x742d35Cc6538C21f1b2cF9BCA59CDf3f3aDBc123", "Deployment Transaction: 0x1234567890abcdef..."],
  "warnings": ["Remember to verify your contract on Etherscan for transparency"]
}

The responses are well-structured by leveraging Pydantic model to reduce LLMs from hallucinating and ensure consistency.

Any future developments?

Yes, of course! There are a lot of improvements and features that can be added to the assistant.

1. Improving contract generation capability

The assistant only supports basic contract generation, including the ERC-20 standard and the ERC-721 standard. There is potential to expand its capabilities to generate more complex contracts, including stablecoins, Real World Assets (RWA), and Decentralised Autonomous Organizations (DAO), etc. As well as generating contracts on different non-ethereum-based chains, including Solana, which uses Rust for its smart contracts. For clients with specialized needs, custom contract generation can be added for clients that has a specific task that they want to execute on the blockchain.

2. Supporting multi-blockchain support

Instead of only allowing users to deploy contracts on Ethereum Sepolia. Expanding this to include multiple Ethereum-based chains, such as Arbitrum and Base, as well as non-Ethereum-based chains like Solana and Hyperliquid, would significantly increase the assistant’s usability and appeal, which would then attract a wider range of audiences.

3. Automating security audits

The idea is for the assistant to automatically check the generated contracts for common vulnerabilities, coding errors, or inefficiencies before deployment. This can be achieved by integrating existing smart contract analysis tools or frameworks, such as:

• Slither — A static analysis framework for Solidity that detects vulnerabilities like reentrancy, uninitialized storage, or arithmetic issues.
• MythX — A cloud-based security analysis platform that can run automated tests against Ethereum smart contracts.

By incorporating these tools, the agent could provide real-time alerts and suggestions, flagging risky code patterns and recommending safer alternatives. This ensures that even users without deep blockchain knowledge can deploy more secure and reliable smart contracts.

Moving on to Graphite Workflow Builder…

After completing my first project with the Smart Contract Assistant, I was given a second task — one that was both exciting and challenging. I was asked to build an assistant capable of generating an entire project demo using Graphite. Yes, you read that correctly: using Graphite to build a Graphite demo! This assistant was intended to showcase the framework to potential clients, investors, and even university students, making complex workflows easy to understand and interact with.

The challenging part is that we need to teach the LLM to understand the Graphite framework to accurately generate the demo.

How does the Graphite Workflow now work?

The workflow primarily consists of two routes: firstly, the general query route, which answers users' questions without generating any demos; secondly, the generation route, which generates the entire project demo in one step based on user requirements.

Planning

The reasoning_node is responsible for planning out the project structure and required resources from the Graphite repo for generation.

Example reasoning_node response:

{
  "reasoning": "Brief analysis of user requirements",
  "requires_code_generation": true,
  "required_resources": ["specific_graphite_components_needed"],
  "code_generation_plan": "specific code generation plan",
  "required_files": {
    "main.py": "Main application code with core workflow logic",
    "requirements.txt": "Python dependencies needed for the project",
    "README.md": "Setup instructions and usage documentation"
  },
  "confidence": 0.95
}

It lists out the required resources, which tells the data_retrieval_node what class, functions, or examples we need for generation, and the required_files field listing out what files will be included in the project folder.

Data Retrieval

The data_retrieval_node is responsible for preparing input parameters for the data_retrieval_tool_node, which calls a custom tool I defined called GraphiteTool. To enable quick and accurate retrieval of relevant resources from the Graphite codebase, I created an index using the Abstract Syntax Tree (AST) library to parse the entire repository. With a pre-defined summary of what resources can be fetched included in the prompt template, the data_retrieval_node can efficiently query the GraphiteTool with the correct parameters, allowing the assistant to provide precise and contextually relevant information from the codebase.

Example Graphite repo index:

"workflows": {
    "grafi_workflows_workflow": {
      "path": "grafi/workflows/workflow.py",
      "file_name": "workflow.py",
      "size_bytes": 3762,
      "classes": [
        {
          "name": "Workflow",
          "line_number": 24,
          "docstring": "Abstract base class for workflows in a graph-based agent system.",
          "methods": [
            {
              "name": "stop",
              "args": [
                "self"
              ],
              "docstring": "Stop the workflow execution. This method can be called by an assistant to stop the workflow during execution.",
              "is_async": false,
              "line_number": 36
            }, ...

Example function calls to retrieve resources from the index:

# multiple function calls
get_graphite_resource(grafi_workflows_impl_event_driven_workflow, include_source_code=true)
get_graphite_resource(grafi_nodes_node, include_source_code=true)
get_graphite_resource(grafi_tools_function_calls_impl_google_search_tool, include_source_code=true)

Code generation and validation

Once the necessary resources are retrieved, the real work begins. The code_generation_node is responsible for generating all the required files for the demo project, including README.md, main.py, and requirements.txt.

Example response for code_generation_node:

{
  "generated_files": [
    {
      "filename": "main.py",
      "content": "ACTUAL PYTHON CODE CONTENT HERE"
    },
    {
      "filename": "requirements.txt",
      "content": "ACTUAL REQUIREMENTS LIST HERE"
    },
    {
      "filename": "README.md",
      "content": "ACTUAL MARKDOWN README CONTENT HERE"
    }
  ],
  "explanation": "how the code works and what it does",
  "components_used": ["list", "of", "Graphite", "components", "used"]
}

After generating the required code, we will proceed to the code_validation_node, with the intention of ensuring the project is correctly generated. We will go through various steps, including syntax validation, dependencies installation validation, import validation, and code execution validation. All of these checks are performed by a custom code_validation_tool integrated with the code_validation_tool_node

If any validation test fails, the workflow loops back code_generation_node via the code_regeneration_topic to regenerate the flawed code. This cycle continues until the project passes all validation tests, ensuring a fully functional and error-free demo.

Example response from code_validation_tool_node:

{
  "validation_passed": true/false,
  "error": "Error message if validation failed",
  "generated_files": "Original files_json for downstream nodes",
  "project_description": "Project description for context",
  "details": {
    "project_name": "demo_project",
    "validation_id": "abc123",
    "main_file": "main.py",
    "tests_run": ["syntax", "dependencies", "imports", "execution"],
    "all_passed": true/false,
    "details": {
      "syntax": {"passed": true, "message": "Syntax validation passed"},
      "dependencies": {"passed": true, "message": "Dependencies installed successfully"},
      "imports": {"passed": true, "message": "Import validation passed"},
      "execution": {"passed": true, "message": "Execution validation passed"}
    }
  }
}

File Shipping

Given that we have passed all the validation tests, the workflow reaches its final step: bundling all the generated files and shipping them back to the user. Thefile_node and file_tool_node are responsible for zipping all the project files and storing them on the backend Docker container, and exposing an API endpoint for users to download the whole project as a single zip. This is done through a file_node function tool that I have defined.

Example response from file_tool_node:

{
  "success": true,
  "file_id": "abc123",
  "project_name": "university_agent",
  "description": "AI agent for university queries",
  "files_saved": ["university_agent.py", "requirements.txt", "README.md"],
  "download_urls": {
    "individual_files": {
      "university_agent.py": "http://localhost:8000/api/files/abc123/download/university_agent.py",
      "requirements.txt": "http://localhost:8000/api/files/abc123/download/requirements.txt",
      "README.md": "http://localhost:8000/api/files/abc123/download/README.md",
      ".env": "http://localhost:8000/api/files/abc123/download/.env"
    },
    "zip_all": "http://localhost:8000/api/files/download/zip/abc123"
  },
  "expires_in_hours": 24
}

The user can also selectively download the individual files they need.

Final Response

The final results are then structured by the output node and returned to the user.

Example response from output_node:

{
  "status": "completed",
  "summary": "Description of the generated project and its functionality",
  "generated_code_link": {
    "individual_files": {
      "university_agent.py": "http://localhost:8000/api/files/abc123/download/university_agent.py",
      "requirements.txt": "http://localhost:8000/api/files/abc123/download/requirements.txt",
      "README.md": "http://localhost:8000/api/files/abc123/download/README.md"
    },
    "zip_all": "http://localhost:8000/api/files/download/zip/abc123"
  },
  "setup_guide": "1. Download the ZIP file\n2. Extract the files\n3. Run: pip install -r requirements.txt\n4. Execute: python [main_file].py",
  "warnings": ["Files expire in 24 hours", "Make sure to have Python 3.8+ installed"]
}

The whole workflow/Agent ensures that the correct Graphite resources are fetched, the code is running, and files can be downloaded by the user.

Improvements?

Yes! Just like the Smart Contract Assistant, there are several areas where the Workflow Builder can be improved and made more robust:

• Increase the complexity of the generated project — Every LLM has its own context window limits; therefore, it restricts the complexity of code that a single code generation node can produce. One possible solution is to add multiple code generation nodes working in parallel and then combine their outputs. This would allow the assistant to generate larger and more complex projects for demonstration purposes.
• Improve workflow efficiency — Currently, generating a complete project can take several minutes. While this is acceptable for internal use or university demos, a more formal or commercial setting requires faster results. Potential improvements include using parallel LLM nodes, introducing asynchronous operations, or optimizing function calls to reduce the overall execution time. etc.
• More robust tools — The assistant currently relies on 3 tools: the Graphite tool, the File tool, and the Code Validation tool, which should be sufficient for small-scale demos. However, if the app is going to expand or to a broader audiences, more tools can be added to make the app more robust and convenient, other than improving the existent 3 tools, adding more tools would also be a viable option to proceed, examples would be a Demo Execution tool, which allows users to run the demo workflow directly on the frontend chat interface, another feature would be integrating graphical interface, like Grafi-Dev to visualize the demo workflow on the frontend, this could greatly improve the understandablility of the generated demo.

What did I learn?

As a recent Computer Science graduate, before starting this internship, I had absolutely no idea what AI agents are, nor how to set up an MCP server, nor where to define prompt templates for LLMs. This internship has been an incredible learning journey, exposing me to the full lifecycle of building agentic applications. I learned how to design modular workflows, structure nodes, and connect them to external tools and APIs. I also gained hands-on experience with prompt engineering, orchestrating complex LLM interactions, and debugging multi-step workflows. Beyond the technical skills, I developed a deeper understanding of how AI agents can be applied in real-world scenarios, from automating smart contract deployment to generating fully functional project demos. This experience has significantly expanded my knowledge and confidence in building sophisticated AI-driven applications.

Here are some important tips/ideas that I have learnt during the internship when building agentic applications or even normal projects:

• Always use structured output for LLM nodes — I learned that having a clearly defined output format for each node is crucial. Structured outputs reduce hallucinations, make debugging easier, and ensure that downstream nodes can reliably process the results. For example, instead of returning free-form text from an LLM, defining a JSON schema or specific fields allows the agent to act consistently and predictably. Especially for Graphite, data can be sent to topics based on whether the field(s) meet the condition of the topic
• Pay attention to prompt engineering — The quality of an LLM’s output depends heavily on how the prompts are written. I discovered that carefully crafting prompts, providing context, examples, and clear instructions, can dramatically improve performance. Prompt engineering isn’t just about asking questions — it’s about anticipating edge cases, handling unexpected responses, and guiding the LLM to produce actionable results.
• Utilize coding assistants — Before the internship, I only used web-based LLMs to ask questions, pasting code whenever there were bugs, or to ask for suggestions. Now I have discovered coding assistants, namely Claude Code, and it was a game-changer for me. Claude Code not only helped me write code faster, but it also allowed me to focus on higher-level design and logic rather than getting bogged down in boilerplate code. Just remember, don’t overly rely on these coding assistants; make sure you truly understand what the code assistant suggests and have your own reasoning before applying the changes.
• Ask questions and seek clarification — During my journey in building the projects, I have encountered several issues and have caused a lot of confusion. Sometimes, you might have just encountered a bug or issues that are actually from the software or framework itself, but not by your own. Luckily, all people from Binome are happy and willing to help me. Therefore, when you are stuck on an issue for quite a bit of time and have not found a solution, speak up to colleagues, mentors, or teammates. I am sure that there is always someone that are willing to help you out.

What’s next for me?

The past three months have been an incredibly exciting and rewarding journey. During my internship at Binome, I had the chance to work alongside talented individuals who are good at presenting, developing, and designing. Collaborating with such a team has been both inspiring and educational, and I feel fortunate to have had the opportunity to contribute to meaningful projects.

Looking ahead, I’m excited to continue working closely with Binome. I hope to take on more challenging and innovative projects, further exploring the potential of AI agents and workflow automation. My goal is to apply the skills and insights I’ve gained during this internship to build tools that are not only technically robust but also have real-world impact, making complex technologies more accessible to users and developers alike. This experience has reinforced my passion for combining AI, software development, and problem-solving, and I look forward to what the next chapter brings.

Developing AI-Powered Assistants with Graphite: Lessons from My Internship was originally published in Binome on Medium, where people are continuing the conversation by highlighting and responding to this story.

GitHub - binome-dev/graphite: Graphite Agentic Framework by Binome Technologies

If you’ve been following our updates regularly, you may have seen the launch of Graphite — an exciting new AI agent framework.

Introduction to Graphite — An Event Driven AI Agent Framework

Graphite is our open-source framework designed to help you build a reliable, flexible and observable AI assistant that can be customized to understand your domain. Whether it’s finance, healthcare, real estate or something totally niche, Graphite empowers building domain specific AI agents that know what they’re doing.

Next, we’ll walk through a simple — yet highly extensible — example of an AI-driven real-estate consultation.

Graphite in Action — KYC Agent for Property Search

We began with a classic “Know Your Client” (KYC) pattern — a foundational workflow whenever trust, compliance, and personalized service overlap. In the Graphite repo, we shared a basic KYC example but now we’re taking it a step further with a real-world use case: a property search assistant. This demo leverages Graphite’s powerful event driven workflow to manage human-in-the-loop interactions.

In real estate, truly knowing your client is the cornerstone of trust , and the key to offering advice that fits them perfectly. Before suggesting any properties, a skilled agent pauses to understand who the client is, what they’re looking for, how much they can spend, and which features are “must-haves” versus “nice-to-haves.”

A naïve, one-size-fits-all bot fires off blunt queries: “Budget?” “Beds?” “Post code?” — collecting data but not really helping the client. If client knows all the above, they can just search online. A domain-specific agent does the opposite:

1. Warm introduction — Welcomes the prospect and sets expectations.
2. Contextual probing — Surfaces lifestyle goals (“family size,” “walkable nightlife”) rather than just numeric filters.
3. Progressive disclosure — Reveals options in stages, validating assumptions and updating search parameters on the fly.

By encoding that consultative flow — greeting → rapport → nuanced discovery → tailored recommendations — Graphite demonstrates how specialized agents outperform generic chatbots. Domain knowledge isn’t a nice-to-have; it’s the difference between “Here are 500 listings” and “Here are the three homes that feel made for you.”

We’ve taken this flow and implemented it using Graphite. Below, you’ll see how each part of the agent’s thought process translates into modular components all working together to make smarter decisions faster.

How the Property Search Agent Works with Graphite

Graphite’s core strength lies in its modular, event-driven architecture. Think of it as a well-rehearsed dance: every step emits an event, and every event triggers the next move. Because each event is logged, you can always trace how the agent reached its decisions.

User Interaction

The conversation starts with a free-form prompt such as:
“Hello, I want to find a property in London.” From that moment, PropertyAssistant—Graphite’s high-level assistant for this workflow—takes over. The workflow owns the entire life-cycle: routing messages, storing context, and emitting the events that drive the downstream nodes.

Build User Profile

This step Powered by a LLM Node — User Profile Builder Node, it build the user profile for property recommendation, the data points are:

1. location (city, neighbourhood)
2. profession & salary band
3. household size (solo, couple, family)
4. lifestyle needs (home office, schools, pets, nightlife)

This node will decide if all the required information is gathered, and produce either “Finished” or “Missing <data point>” to user_profile_builder_result_topic

Decide Next Step

Given the result of User Profile Builder Node, the Action Node inspects the content and chooses one of two paths:

1. Enough data? -> route to the Summary Node.
2. Missing data? -> invoke the Human Request Function Node with a concise follow-up question (“What a comfortable monthly payment?”).

Human In The Loop (HITL)

When the workflow needs more detail, the Human Request Function Node publishes the follow-up question (generated by the Action Node) to the system reserved topic human_request_topic. The Assistant subscribes to that topic, shows the question to the user, and routes the user’s reply to the next node for parsing.

Observe User Reply

The Observation Node, powered by an LLM, parses the user’s latest reply, converts key details into structured facts — e.g., budget ≈ £2 200 pcm, needs home office — and passes them to the User Profile Builder Node. The updated profile re-enters the loop, ready for the next Action Node decision.

Final Response

With a full profile in hand, the Summary Node (LLM Node) will renders a concise user preferred property snapshot (“3 bedrooms, price up to 400,000 , West of London”). And generates initial property recommendations from the LLM’s internal knowledge.

Because the user’s requirements are stored as a fully structured snapshot, wiring in a live property API (Rightmove, Zoopla, etc.) is almost effortless. The agent serialises the profile — “a 3-bed flat in West London, house price up to 400,000, near good primary schools.” to a function call parameters like

{
 "type": "flat",
 "beds": 3,
 "area": "West London",
 "maxPrice": 400000,
 "filters": ["near_primary_school"]
}

and call a search API. That crisp contract between profile and API is what makes the system plug-and-play and instantly extensible.

Example

Let’s walk through a real interaction to show how the agent works in practice. From the very first message, the assistant takes on a warm, conversational tone:

Notice how instead of asking for budget or number of rooms directly, the assistant collects contextual signals — like profession and who will live there — to infer those details.

As the assistant gathers information, it doesn’t just ask questions, it’s building a profile in the background. Rather than asking users to manually fill out a form, the assistant infers details like:

• Who will live there → determines space/room needs.
• neighbourhoods and location -> where you may love to find the new home.
• Job, monthly payment an saving→ estimate budget.

Based on the user provided information, agent get the summary of the recommendation, such as location, number of rooms, and budget. And the agent also provide some recommended places (based on the knowledge of LLM model).

What’s Next — Smarter, Deeper, and Truly Live

Today the agent builds a profile from your chat alone, and based on LLM, which could cause hallucination (like above recommended properties). Next, we’re expanding its reach:

• Real-time data sources — On demand, the agent will query live feeds for sales trends, neighbourhood crime stats, school-performance scores, and commute times, enriching its reasoning with fresh, objective facts.
• Tool-aware reasoning — Instead of relying solely on the LLM’s memory, the workflow will call specialised tools (maps APIs, valuation models, census datasets) whenever a question requires them, then fuse those results back into the dialogue.
• End-to-end property search — We’re wiring in production APIs from Rightmove, Zoopla, and others, so the assistant can turn your profile into a real query, fetch current listings, and return book-ready links.
• Hyper-personal ranking — Listings will be scored against your exact constraints — budget, commute, child-friendly amenities — so the first page of results already feels “just right.”

With live data, external tools, and genuine listing feeds in the loop, the assistant moves from prototype to practical house-hunting partner.

Conclusion

We show the example built by Graphite, which can turn the messy, moving parts of an AI assistant into clean, swappable blocks. To highlight

• Event-driven and modular — every node is independent but coordinated, so adding RAG, a human-review step, or a brand-new API is literally plug-and-play.
• Transparent by default — every event is logged and replayable, making debugging and compliance painless.
• Built to scale — prototype on a weekend, ship to production on Monday without rewriting the workflow.

The Property Search Agent is just one proof-point. Any domain — support, finance, healthcare — can get the same rock-solid orchestration and observability.

AI is moving fast. Graphite helps you move faster without breaking things. Grab the repo, wire up your own nodes, and show us what you build — because the next great assistant won’t replace humans; it’ll work right alongside us.

Graphite in Action: A Demo of the Property Search Agent was originally published in Binome on Medium, where people are continuing the conversation by highlighting and responding to this story.

In the world of AI development, AI agents are becoming an essential part of process automation. However, very often building these agents feels very unintuitive and frustrating to optimize, as they are rigid and hard to debug. Popular agent frameworks like LangChain often rely on deeply nested chains, implicit state, and runtime patching via callbacks making them very difficult to work with, especially when dealing with multi-step tool use or unexpected outputs from an LLM(source).In this article I would like to share my personal experience of using Graphite, a new agent development framework that embraces simplicity, event-driven logic and developer transparency.

GitHub - binome-dev/graphite: Graphite Agentic Framework by Binome Technologies

Why Graphite you might ask?

At its core, Graphite utilizes a publish-subscribe(Pub-sub) system which allows user to use the following features:

• Loose coupling: In the Graphite framework, the Nodes subscribe to topic and publish results. This mean you can make any changes to your agent pipeline without accidently breaking the rest of it. Nodes in Graphite subscribe to and publish messages via named topics. That means you can swap out a node, add new behaviors, or reorder flows without touching the rest of the workflow.
• Event-driven workflow: Instead of a strict hard-coded logic embedded in the agent, Graphite allows you to modify the agent’s behaviour based on specific events. Think of it as having a conversation, each node listens, thinks, responds and then passes the information naturally instead of strict procedural logic. This type of architecture encourages active thinking in events, rather than imperative chains.
• Observability: Graphite comes with built-in support for Phoenix, allowing you to trace your assistant’s full logic out of the box, the user will be able to trace the assistant’s full logic, meaning you will be able to see which node has ran, what tools have been used and what was published. This allows you to debug your code easily as you will immediately see what has gone wrong rather than blind trial and error.

For more detailed introduction, please find out in this introduction.

Introduction to Graphite — An Event Driven AI Agent Framework

To try out the framework, I decided to build an AI agent that reads event details from an image (like a flyer) and adds them directly to your Google Calendar. If any information is missing like the title or start time, the assistant automatically follows up with the user to clarify before continuing.

This was a strong use case for Graphite: it involved multiple decision points, human-in-the-loop feedback, a need for both language understanding and function execution, and integration with external tools and APIs.

Agent Design with Graphite

Here is my approach towards designing the workflow of the agent

• VisionNode: this LLM node will be acting as an entry point of the agent’s workflow. It listens for when an image, typically a base64-encoded string is being uploaded to the agent, and then uses language models to extracts key event details such as: title, date, time and location. It doesn’t make any decisions, its sole purpose is to read the image and pull key information which will then be passed onto the next node for evaluation.
• ActionNode: this LLM node responsible for evaluating extracted event details from VisionNode. It doesn’t reprocess the image but instead is utilising language models to check whether the extracted information is sufficient, based on criteria provided in the system prompt. If the details meet the criteria, the data is passed on to the CalendarNode to add the event. If something is missing, it’s sent to HumanRequestNode to request additional input from the user.
• HumanRequestNode: This function node listens on hitl_call_topic and invokes the ask_user tool, which is a function defined by the user. Rather than composing messages with the language model, this tool emits structured function calls that can later be rendered into natural user-facing prompts. Once the tool call is issued, the request is passed back to the user via the human_request_topic.
• CalendarNode: Once all the required information is present, this tool node will be triggered. This node - similarly to HumanRequestNode — doesn’t utilise a language model, but instead uses a predefined function that integrates with Google Calendar API. It operates using a service account, authorized via a JSON key file, which allows it to create calendar entries on behalf of the user without requiring manual login. Once the event is successfully posted, it passes the result to the next node for confirmation.
• ConfirmationNode: After the event has been successfully posted on the calendar, this LLM node will be triggered. It will be using a language model to craft a final confirmation message for the user, summarizing the event that was added. This message includes the key event details, like title, date, and time, and signals that the assistant has completed the task. The output is then published to the user-facing channel to close the loop.

Graphite Agent Code Breakdown

In this section, we’ll walk through the code in three parts: Topics, Nodes, and finally the Assistant, which serves as the agent’s interface.

Topics

Before diving into nodes, let’s revisit how Graphite uses topics. These are named communication channels that connect the nodes. Each topic can also have a condition attached, which lets you implement decision logic declaratively without putting that logic inside the node itself.

To initiate the workflow, we need a root topic the first sort of communication channel between the user and the agent. In this example as can be seen on the image, that’s the agent_input_topic.

from grafi.common.topics.topic import Topic, agent_input_topic

When the user uploads an image (like a flyer), the image is published to this topic. The VisionNode subscribes to agent_input_topic, making it the first node to be triggered in the pipeline. This node is responsible for extracting event details from the image, and its output then flows through the rest of the system via downstream topics.

Now let’s look at how we define a topic that only activates if event information is incomplete and the agent needs to ask the user for more details:

hitl_call_topic = Topic(
    name="hitl_call_topic",
    condition=lambda msgs: msgs
    and msgs[-1].tool_calls
    and msgs[-1].tool_calls[0].function.name == "ask_user",
)

This condition will check if the last message published from the agent to the topic was a call to trigger the ask_user. This is done by using:

• msgs[-1] which will check the last message posted
• msgs[-1].tool_calls checks if there was any function call in the latest message
• tool_calls[0].function.name == “ask_user” will check whether the function that is being called is specifically ask_user.

In Graphite, the condition parameter of a topic simply expects a boolean expression: something that returns True or False.

Another core topic is human_request_topic, which acts as the return path for user responses during human-in-the-loop interactions. This topic is used when the assistant needs to ask the user for missing information, and the user’s reply should re-enter the workflow.

It’s provided directly by the Graphite framework and can be imported like this:

from grafi.common.topics.human_request_topic import human_request_topic

If not conditions need to be met, the topic will be defined the following way:

event_extracted_topic = Topic(name="event_extracted_topic")

Nodes

Now let’s walk through the key parts of the Node design to see how it looks in practice, starting with the VisionNode.

vision_node = (
     LLMNode.Builder()
     .name("VisionNode")
     .subscribe(agent_input_topic)
     .command(
      LLMResponseCommand.Builder()
        .llm(
             OpenAITool.Builder()
             .name("VisionLLM")
             .api_key(self.api_key)
             .model(self.model) 
             .system_message(self.event_extraction_system_message)  
             .build()
         )
         .build()
     )
     .publish_to(event_extracted_topic)
     .build()
    )

Let’s focus a bit more on the important parts of the code and what exactly happens there:

1. .Builder(): Each class (like LLMNode, LLMResponseCommand, or OpenAITool) uses a Builder pattern to let you define properties step by step. This keeps the code clean and avoids one giant constructor with too many arguments. You configure the object using chained methods like .name(), .model(), etc., and finally call .build() to create the actual instance.
2. .subscribe(agent_input_topic): This tells the node where to listen for incoming messages. In this case, it listens on agent_input_topic, which is the root topic triggered when the user uploads an image. Once a message is published to this topic, VisionNode gets activated.
3. .command(…): This defines what the node actually does when triggered. For LLM-based nodes like this one, it runs an LLM call using the configuration inside LLMResponseCommand. This includes the model, system message, and API key all encapsulated in the nested OpenAITool.Builder().
4. .publish_to(event_extracted_topic): Once the command executes, the node publishes its result to the next topic in the pipeline — here, it’s event_extracted_topic, which triggers the next node.
5. .build: After you’ve set all the necessary configuration via the builder methods, calling .build() finalizes and constructs the actual object. It ensures all internal components are initialized and ready for runtime use. This makes your agent logic more maintainable and flexible to extend later.

With the output of that node now published to event_extracted_topic, the next step in the workflow is the ActionNode, which subscribes to that topic.

action_node = (
    LLMNode.Builder()
    .name("ActionNode")
    .subscribe(
        SubscriptionBuilder()
        .subscribed_to(event_extracted_topic)
        .or_()
        .subscribed_to(human_request_topic)
        .build()
    )
    .command(
        LLMResponseCommand.Builder()
        .llm(
            OpenAITool.Builder()
            .name("ActionLLM")
            .api_key(self.api_key)
            .model(self.model)
            .system_message(self.action_llm_system_message)
            .build()
        )
        .build()
    )
    .publish_to(hitl_call_topic)
    .publish_to(complete_info_topic)
    .build()
)

You might notice that ActionNode subscribes to two different topics: event_extracted_topic and human_request_topic. This is done using the SubscriptionBuilder, which lets you combine multiple topic subscriptions in a declarative and readable way:

This setup means the node will be triggered in two cases:

1. When new event data is extracted by the VisionNode and published to event_extracted_topic.
2. When the user responds to a follow-up question and the message is published back to human_request_topic.

ActionNode will either trigger HumanRequestNode or CalendarNode, both requiring additional functions defined by the user as can be seen below:

calendar_node = (
    LLMFunctionCallNode.Builder()
    .name("CalendarNode")
    .subscribe(complete_info_topic)
    .command(
        FunctionCallingCommand.Builder()
        .function_tool(self.add_event_to_calendar)
        .build()
    )
    .publish_to(calendar_response_topic)
    .build()
)

Where add_event_to_calendar is an additional function defined by the user, in this case it integrates the Google Calendar API allowing the agent to post events on the behalf of the user.

That’s the nice thing about Graphite: you can see your logic as a series of event reactions.

Assistant Setup

Finally, here’s how everything comes together in a real application. In this case, I used FastAPI to wrap the assistant into a lightweight web service, exposing endpoints for image uploads and follow-up messages. In Graphite, the assistant is the top-level object that interfaces with users — essentially, the fully assembled agent.

It’s important to mention that each request is tied to an ExecutionContext object, which carries the a set of ids to manages identifiers across message life cycles (detail can be found here). This allows the assistant to maintain memory across messages crucial for asking users for missing info and then following up with additional context.

execution_context = ExecutionContext(
    conversation_id=req.conversation_id,
    execution_id=uuid.uuid4().hex,
    assistant_request_id=uuid.uuid4().hex,
)

The assistant is instantiated once with all the system prompts injected into the correct nodes at build time:

assistant = (
  ImageToCalendar.Builder()
  .api_key(openai_key)
  .event_extraction_system_message(event_extraction_system_message)
  .action_llm_system_message(action_llm_system_message)
  .observation_llm_system_message(observation_llm_system_message)
  .summary_llm_system_message(summary_llm_system_message)
  .model("gpt-4o")
  .build()
)

The list of all system prompts used by the assistant can be seen here.

Example

Now the assistant is live and ready to use by the user.

Here’s an example of how it works. If I upload the following image:

I will be able to get the following output from my agent:

As mentioned previously, Graphite framework integrates beautifully with observe platform like Arize Phoenix, which means every message passed between nodes, every tool call, every function response it’s all traceable.

To capture the traces, start the phoenix locally using:

phoenix serve

You can open the Phoenix UI and:

• Click into a conversation trace
• See which node was active and why
• View the exact tool call made and the output it returned

Here’s what a typical trace looks like:

Each block represents a node execution, showing input, output, and any triggered function tools.

Final Thoughts

Graphite is hands-down the cleanest and easiest to use agent development framework that you will be able to come across. The pub/sub architecture means you’re never locked into one way of doing things — you can react to events, swap out logic, and compose workflows that evolve with your needs. And it’s not just about connecting nodes together. Graphite actually helps you deal with the messy stuff things like working with external APIs, coordinating between LLM calls and function execution, or tracing what happened when something breaks.

And yes, it’s open source, feel free to check out the GitHub repository if you’re curious, suggestions, and improvements are always welcome.

And here is the link to this specific agent if you want to try it yourself!

Graphite for AI Agents: Event-Driven Design Made Simple was originally published in Binome on Medium, where people are continuing the conversation by highlighting and responding to this story.

Graphite is an open-source framework for building domain-specific AI assistants using composable agentic workflows. Please check our post for more detail. In this post we will dive into Graphite event driven architecture.

Introduction to Graphite — An Event Driven AI Agent Framework

Event-driven architecture is at the heart of Graphite, empowering flexible, modular workflows that can adapt to complex business requirements. By leveraging publish/subscribe (pub/sub) patterns, Graphite orchestrates node execution in a way that gracefully handles everything from simple linear flows to branching and cyclical dependencies — all while ensuring the correct sequence of messages for large language models (LLMs) and other downstream tasks.

The Benefits of Event Driven Architecture

In traditional, coupled workflows, tasks often execute in a fixed sequence: when one step finishes, the next begins. By contrast, event-driven systems allow any component to “publish” an event, while interested “subscribers” automatically respond. This approach delivers significant advantages:

• Flexibility: Add or remove nodes and branches without re-engineering the entire flow.
• Scalability: Efficiently handle complex interactions with multiple data sources, functions or AI services.
• Modularity: Encapsulate functionality in discrete nodes that each listen for specific events and publish their results.

To harness these benefits, Graphite uses concept of Topics — queues that store messages in the FIFO pattern. This design offers a clear, auditable trail of each event passing through your workflow, simplifying both debugging and compliance in complex systems.

Inside Graphite’s Event Driven Architecture

Graphite’s event-driven workflow consists of four primary components:

1. Topic: the message queues
2. Node: the processing unit
3. Event: the records of every state change
4. Event Store: the durable log of all events

Below is an example showing a node’s execution lifecycle and the sequence of events it generates.

Next, we’ll examine the Topic and Node components — alongside pub and sub events — to illustrate how Graphite’s event-driven workflow operates.

Topic

Under the hood, each Topic is like a temporary, in-memory event queue that lasts for the duration of a single request, with a map which store the consume offset of each consumers.

At the start of each request, Graphite initializes its assistant Topics. Every incoming message—encapsulated as a PublishToTopicEvent—is appended to the appropriate Topic (subject to a customizable publish condition) message queue. A node consumed from a topic will produce ConsumeFromTopicEvent. Under the hood, Graphite’s Topic class maintains, for each consumer node, a map of consumed offsets so it always knows which messages have been processed and which are still pending.

By using the execution_context, topic_name, and offset fields from PublishToTopicEvent and ConsumeFromTopicEvent, Graphite can rebuild Topic of unfinished assistant request by restoring its message queue from the event store. When a workflow must be resumed, Graphite reinstates the Topic with the exact offsets so it picks up precisely where it left off.

Node

In Graphite, a node is an execution unit that subscribes to one or more input (“sub”) topics and publishes results to one or more output (“pub”) topics. Each node encapsulates its subscription logic — the subscribe_to condition—which determines exactly when it should trigger. Here is the code example of the subscribe_to condition

SubscriptionBuilder()
.subscribed_to(agent_input_topic)
.or_()
.subscribed_to(human_request_topic)
.build()

Once triggered, the node:

• Consumes events from its subscribed topics
• Processes those events
• Publishes the resulting events to its output topics

Those newly published events can then trigger downstream nodes that subscribe to the same topics.

Below is an illustration of a single node with its corresponding sub- and pub-topics in Graphite’s event-driven workflow. Whenever new events arrive on a topic, the workflow evaluates each subscribing node’s subscribe_to condition; if the condition is satisfied, that node is added to the execution queue.

Once a node completes execution, the workflow publishes its response to the downstream topics. Within each Topic object, the topic’s publish condition is evaluated to determine whether the incoming message should be accepted or ignored.

In the next section, we’ll explore strategies for resolving circular dependencies between nodes, as well as best practices for preserving a complete, queryable history of all messages as they flow through your workflow.

Handling History and Circular Flows

AI systems, particularly those involving LLMs, often require loop of “reasoning”. For instance, in a ReAct agent workflow, an LLM-based “Thought”, “Action“ and “Observe“ nodes might repeatedly query in a loop (“Reasoning → acting → observing -> reasoning…”) before finalizing an answer.

This can create loops and branching flows that make it tricky to maintain a coherent memory or message history. Graphite solves this with time travel and time-weighted topology sort.

Time Travel

Even with multiple loops or cycles, events in Graphite can be “stretched“ as a “Directed Acyclic Graph (DAG)“. By tracking each event’s offset and storing them in Topics, Graphite can “time travel” back to reconstruct which messages existed at each step. In the ReAct example above, the node forms a cycle, but events still unfold in that linear sequence. The illustration below shows how it would appear if the Thought node ended its loop on the third iteration.

Time-Weighted Topology Sort

For workflows that resemble general graphs (with branches and circular dependencies), Graphite “stretched“ the events into a DAG, and then is able to sort events by combining traditional topological ordering with timestamps. Events with the same number of dependencies (“in-degree”) are ordered by the time their events were published. This ensures the most recent and contextually relevant events are processed in the correct sequence.

Now let’s look at the weighted topology sort. Suppose you have an events lineage like following:

The rules of our time weighted topology sort extended the topology sort such that if there are same in-degree events, then we will sort the corresponding event by timestamp of the event. It will guarantee that the events which are close to the current invoked now will always closer. For above the possible sorted for next input will be:

Event 1 → Event 3 → Event 2 → Event 4 → Event 5

Example

We will demonstrate the event-driven workflow of our ReAct agent. Below is the workflow captured by our in-house debugging tool, which we plan to open-source once testing is complete.

After we asked this question:
What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into?

Here is the response:

Here are all the events generated from the execution.

We can see it took 1 circle of ReAct to get the answer. Which is

You can also select a node — for example, the ActionNode, which will displays the corresponding filtered events.

Switching the event display mode lets you view raw events — essential for tracing, debugging, and auditing

Conclusion

Graphite’s event-driven workflow brings a robust, modular design to AI-driven applications. By blending pub/sub patterns, temporary in-memory Topics, time travel, and time-weighted topology sorting, the system tackles the challenges of complex branching, circular dependencies, and historical context retrieval. Whether you’re an AI engineer building cutting-edge tools, a software developer streamlining your application pipeline, or an product owner seeking a forward-looking platform, Graphite’s architecture highlights how thoughtful, event-centric design can unlock next-level capabilities for AI-enabled products.

Graphite Deep Dive — Event Driven Workflow was originally published in Binome on Medium, where people are continuing the conversation by highlighting and responding to this story.

Graphite is an open-source framework for building domain-specific AI assistants using composable agentic workflows. It offers a highly extensible platform that can be tailored to unique business requirements, enabling organizations to develop custom workflows suited to their specific domains.

We’ve mentioned this product in our previous posts, and we’re proud to share that we have now open sourced Graphite on GitHub! Check it out and give us a Star ⭐ there if you like it!

GitHub - binome-dev/graphite: Graphite Agentic Framework by Binome Technologies

You might be wondering, “With AI solutions like ChatGPT, Claude, and various agentic platforms and frameworks already available, why create another one” The short answer: we identified a gap in solving real-world problems with AI tools. Many generic agents — like typical ReAct or CoT agents — struggle with mission-critical tasks where mistakes can be costly.

Graphite provides a straightforward and controllable framework for building workflows, allowing large language models (LLMs) not just to reason and plan, but to operate within a defined, auditable, and easily restorable workflow. It also includes essential features such as observability, idempotency, and auditability out of the box.

In this post, we’ll introduce Graphite briefly, from its architecture to its key features, and provide a simple example: using Graphite to build a Know-Your-Client AI assistant.

The Architecture: Simple, Powerful, and Composable

Graphite is structured into three conceptual layers — Assistants, Nodes, and Tools:

• Assistants orchestrate the workflow and manage the conversation state.
• Nodes encapsulate discrete logic, each dedicated to a specific function (e.g., an LLM call or function execution).
• Tools are pure functions responsible for executing tasks, such as calling an API or running a Python function.

Additionally, Graphite employs event sourcing pattern to record every state change. Whenever an Assistant, Node, or Tool is invoked, responds, or fails, a corresponding event is generated and stored in the event store.

The command pattern cleanly separates request initiators from executors through well-defined Command objects and handlers. These Commands carry all the necessary context, enabling nodes to invoke tools in a self-contained and straightforward manner.

The nodes coordinate through a lightweight, Pub/Sub workflow orchestration mechanism.

• Workflow orchestrates interactions among nodes using a Pub/Sub pattern with in-memory message queuing.

This architecture unlocks plug-and-play modularity — you can build workflows like Lego bricks, add or remove nodes, and isolate failures with surgical precision.

The Features — What Makes Graphite Shine

The core design principles we’ve introduced above shows that set Graphite apart from other agent frameworks are:

1. A Simple 3-Layer Execution Model
   Three distinct layers — assistant, node, and tool — manage execution, while a dedicated workflow layer oversees orchestration.
2. Pub/Sub Event-Driven Orchestration
   Communication relies on publishing and subscribing to events, ensuring a decoupled, modular flow of data throughout the system.
3. Events as the Single Source of Truth
   All operational states and transitions are recorded as events, providing a uniform way to track and replay system behavior if needed.

Combining these elements, Graphite provides a production-grade AI application framework capable of operating reliably at scale, handling failures gracefully, and maintaining user and stakeholder trust. Four essential capabilities form the backbone of this approach:

1. Observability
   Complex AI solutions involve multiple steps, data sources, and models. Graphite’s event-driven architecture, logging, and tracing make it possible to pinpoint bottlenecks or errors in real time, ensuring that each component’s behavior is transparent and measurable.
2. Idempotency
   Asynchronous workflows often require retries when partial failures occur or network conditions fluctuate. Graphite’s design emphasizes idempotent operations, preventing pub/sub data duplication or corruption when calls must be repeated.
3. Auditability
   By treating events as the single source of truth, Graphite automatically logs every state change and decision path. This level of detailed record-keeping is indispensable for users working in regulated sectors or who need full traceability for debugging and compliance.
4. Restorability
   Long-running AI tasks risk substantial rework if they fail mid-execution. In Graphite, checkpoints and event-based playback enable workflows to resume from the precise point of interruption, minimizing downtime and maximizing resource efficiency.

Together, these capabilities — observability, idempotency, auditability, and restorability — distinguish Graphite as a framework for building robust and trustworthy AI applications. Below is a detailed breakdown of how Graphite implements each feature.

Show Your Code — KYC Assistant

When designing an AI-based workflow, keep in mind that large language models introduce uncertainty. It’s helpful to follow Occam’s razor principle, which means the simpler the workflow, the better.

Design the workflow

Suppose you want to create a “Know Your Client (KYC)” assistant for a gym registration process, with human-in-the-loop (HITL) functionality. The user must provide their full name and email address to finish the registration workflow. And if anything is missing, the workflow pauses and asks client for more information. Clearly the real world problem would need more information, but here we just simplify it for this demo.

Build the assistant

Firstly, install Graphite from pip

pip install grafi

From above workflow graph, we will need add following components:

7 topics:

• agent input topic (framework provided)

from grafi.common.topics.topic import agent_input_topic

• user info extract topic

user_info_extract_topic = Topic(name="user_info_extract_topic")

• human call topic

hitl_call_topic = Topic(
            name="hitl_call_topic",
            condition=lambda msgs: msgs[-1].tool_calls[0].function.name
            != "register_client",
        )

• human in the loop topic (framework provided)

from grafi.common.topics.human_request_topic import human_request_topic

• register user topic topic

register_user_topic = Topic(
            name="register_user_topic",
            condition=lambda msgs: msgs[-1].tool_calls[0].function.name
            == "register_client",
        )

• register user respond topic

register_user_respond_topic = Topic(name="register_user_respond")

• agent output topic (framework provided)

from grafi.common.topics.output_topic import agent_output_topic

5 nodes:

• LLM Node [User info extract node] to extract name and email from user input

user_info_extract_node = (
    LLMNode.Builder()
    .name("ThoughtNode")
    .subscribe(
        SubscriptionBuilder()
        .subscribed_to(agent_input_topic)
        .or_()
        .subscribed_to(human_request_topic)
        .build()
    )
    .command(
        LLMResponseCommand.Builder()
        .llm(
            OpenAITool.Builder()
            .name("ThoughtLLM")
            .api_key(self.api_key)
            .model(self.model)
            .system_message(self.user_info_extract_system_message)
            .build()
        )
        .build()
    )
    .publish_to(user_info_extract_topic)
    .build()
)

• LLM Node [action] to create action given extracted information

action_node = (
    LLMNode.Builder()
    .name("ActionNode")
    .subscribe(user_info_extract_topic)
    .command(
        LLMResponseCommand.Builder()
        .llm(
            OpenAITool.Builder()
            .name("ActionLLM")
            .api_key(self.api_key)
            .model(self.model)
            .system_message(self.action_llm_system_message)
            .build()
        )
        .build()
    )
    .publish_to(hitl_call_topic)
    .publish_to(register_user_topic)
    .build()
)

• Function Tool Node [human-in-the-loop] to ask user for missing information if any

human_request_function_call_node = (
    LLMFunctionCallNode.Builder()
    .name("HumanRequestNode")
    .subscribe(hitl_call_topic)
    .command(
        FunctionCallingCommand.Builder()
        .function_tool(self.hitl_request)
        .build()
    )
    .publish_to(human_request_topic)
    .build()
)

• Function Tool Node [register user] to register the client

register_user_node = (
    LLMFunctionCallNode.Builder()
    .name("FunctionCallRegisterNode")
    .subscribe(register_user_topic)
    .command(
        FunctionCallingCommand.Builder()
        .function_tool(self.register_request)
        .build()
    )
    .publish_to(register_user_respond_topic)
    .build()
)

• LLM Node [response to user] to draft the final respond to user

user_reply_node = (
    LLMNode.Builder()
    .name("LLMResponseToUserNode")
    .subscribe(register_user_respond_topic)
    .command(
        LLMResponseCommand.Builder()
        .llm(
            OpenAITool.Builder()
            .name("ResponseToUserLLM")
            .api_key(self.api_key)
            .model(self.model)
            .system_message(self.summary_llm_system_message)
            .build()
        )
        .build()
    )
    .publish_to(agent_output_topic)
    .build()
)

And here is the source code — KYC assistant

Then let’s write a simple code to test it:

import json
import uuid

from kyc_assistant import KycAssistant
from grafi.common.decorators.llm_function import llm_function
from grafi.common.models.execution_context import ExecutionContext
from grafi.common.models.message import Message
from grafi.tools.functions.function_tool import FunctionTool


class ClientInfo(FunctionTool):

    @llm_function
    def request_client_information(self, question_description: str):
        """
        Requests client input for personal information based on a given question description.
        """
        return json.dumps({"question_description": question_description})


class RegisterClient(FunctionTool):

    @llm_function
    def register_client(self, name: str, email: str):
        """
        Registers a user based on their name and email.
        """
        return f"User {name}, email {email} has been registered."


user_info_extract_system_message = """
"You are a strict validator designed to check whether a given input contains a user's full name and email address. Your task is to analyze the input and determine if both a full name (first and last name) and a valid email address are present.

### Validation Criteria:
- **Full Name**: The input should contain at least two words that resemble a first and last name. Ignore common placeholders (e.g., 'John Doe').
- **Email Address**: The input should include a valid email format (e.g., example@domain.com).
- **Case Insensitivity**: Email validation should be case insensitive.
- **Accuracy**: Avoid false positives by ensuring random text, usernames, or partial names don’t trigger validation.
- **Output**: Respond with Valid if both a full name and an email are present, otherwise respond with Invalid. Optionally, provide a reason why the input is invalid.

### Example Responses:
- **Input**: "John Smith, john.smith@email.com" → **Output**: "Valid"
- **Input**: "john.smith@email.com" → **Output**: "Invalid - Full name is missing"
- **Input**: "John" → **Output**: "Invalid - Full name and email are missing"

Strictly follow these validation rules and do not assume missing details."
"""


def get_execution_context():
    return ExecutionContext(
        conversation_id="conversation_id",
        execution_id=uuid.uuid4().hex,
        assistant_request_id=uuid.uuid4().hex,
    )


def test_kyc_assistant():
    execution_context = get_execution_context()

    assistant = (
        KycAssistant.Builder()
        .name("KycAssistant")
        .api_key(
            "YOUR_OPENAI_API_KEY"
        )
        .user_info_extract_system_message(user_info_extract_system_message)
        .action_llm_system_message(
            "Select the most appropriate tool based on the request."
        )
        .summary_llm_system_message(
            "Response to user with result of registering. You must include 'registered' in the response if succeed."
        )
        .hitl_request(ClientInfo(name="request_human_information"))
        .register_request(RegisterClient(name="register_client"))
        .build()
    )

    while True:
        # Initial User Input
        user_input = input("User: ")
        input_data = [Message(role="user", content=user_input)]

        output = assistant.execute(execution_context, input_data)

        responses = []
        for message in output:
            try:
                content_json = json.loads(message.content)
                responses.append(content_json["question_description"])
            except json.JSONDecodeError:
                responses.append(message.content)

        respond_to_user = " and ".join(responses)
        print("Assistant:", respond_to_user)

        if "registered" in output[0].content:
            break


if __name__ == "__main__":
    test_kyc_assistant()

Test, Observe, Debug, and Improve

Graphite uses OpenTelemetry and Arize’s OpenInference to trace and observe the assistant behaviour.

To capture the traces locally, we can start phoenix locally.

phoenix serve

Here is an example of the conversation between user and assistant

> User: Hi, I'd like to sign up for your gym. Could you help me with the process?
> Assistant: Please provide your full name and email address to sign up for the gym.
> User: my name is Craig Li, and here is my email: craig@example.com
> Assistant: Congratulations, Craig! You are now registered at our gym. If you have any questions or need assistance, feel free to ask!

Looks good. But if you push the boundary of the tests, you will hit the error real quick. Like this

> User: Hi, how are you today? 

An error occurs, and when we debug with Phoenix, we can pinpoint its exact location:

The issue originates from the Action LLM. Given the input, it should select the appropriate tool, but instead, it generates a plain string response.

To fix this, we can provide clear instructions: if the user asks a question unrelated to gym registration, the LLM should use the request_client_information tool to respond politely and ask if they need assistance with gym registration.

so let’s update action LLM system prompt

.action_llm_system_message(
            "Select the most appropriate tool based on the request. if the user asks a question unrelated to gym registration, the LLM should use the `request_client_information` tool to respond politely and ask if they need assistance with gym registration."
        )

and try again

> User: Hi, how are you today? 
> Assistant: I'm here to assist you with gym registration. Could you please provide your full name and email?

You’ve now built an agent that can assist with gym registrations! You can find the assistant’s code at this link and an example is available here.

Summary

Graphite is an open-source framework for building domain-specific AI assistants through composable, event-driven workflows. With loosely coupled components — assistants, nodes, tools, and workflows — Graphite enables modular design, where events serve as the single source of truth. Its architecture supports complex logic like LLM calls, function execution, and RAG retrieval, all coordinated via dedicated pub/sub topics and commands. Built-in support for observability, idempotency, auditability, and restorability makes it production-ready, allowing workflows to scale, recover, and adapt to compliance needs. Whether you’re building conversational agents or automated pipelines, Graphite offers a flexible, extensible foundation for real-world AI systems.

Introduction to Graphite — An Event Driven AI Agent Framework was originally published in Binome on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this post we will continually discuss LLM-based AI Agent design patterns, building on the foundations from Designing LLM-Based Agents: Key Principles — Part 1 . Previously, we introduced high-level AI Agent design patterns and highlighted the concept of agentic workflows. Now, we will take a closer look at the agentic workflow and demonstrate how to integrate software design patterns to make AI Agents robust, scalable, and observable.

Before diving into the design patterns, let’s clarify three important terms that will be used throughout this post:

• Agentic AI: A type of AI that can make decisions and take actions autonomously. It’s designed to adapt to changing environments and events, and can work with limited human supervision. Agentic AI includes complex behaviours like collaboration, problem-solving, and decision-making in unpredictable or open-ended environments. Think of Agentic AI as a estate agency. It is a system to reach out to potential clients, handle a variety of buying and selling scenarios, and generally interact fluidly with the market.
• AI Agent: An AI Agent is an autonomous software entity designed to perceive its environment, reason about it, and take actions to achieve specific goals. Think of an AI Agent as an employee in that estate agency — independently picking up calls, making decisions, and working to produce profit for the agency.
• Agentic Workflow: The Agentic Workflow is the structured process by which AI Agents operate and achieve their goals. It’s akin to the runbook an employee follows at estate agency. For example, when they pick up a call, there’s a procedure to understand the client’s needs before providing a suitable response.

In this post, we will focus on the agentic workflow because it forms the core of a successful AI Agent.

The Components of Agentic Workflow and its Design Patterns

The workflow orchestrates a set of task nodes that respond to incoming requests. From a software engineering standpoint, a good agentic workflow — particularly one that leverages LLMs — should be:

1. Observable: Engineers should be able to see what is happening within the workflow at every step.
2. Flexible: Workflow components should be decoupled, allowing developers to modify or replace them easily.
3. Restorable: Because LLM calls and tool integrations can be expensive or occasionally fail, we want to be able to restore the workflow from the point of failure to save costs and simplify debugging.

From a software architecture standpoint, three key patterns help meet these requirements:

• Event-Driven Architecture: The workflow operates on a publish/subscribe (pub/sub) model, ensuring a decoupled structure. This design supports scalability, resilience, and modularity — critical qualities for modern systems.
• Event Sourcing Pattern: Every decision that a workflow component makes is recorded in an event store. This store forms an immutable log of all events — an invaluable resource for both observability and restorability. It also serves as the “memory” layer for the workflow.

• Command Pattern: By decoupling orchestration (which determines what needs to be done) from execution (the actual task), the system becomes significantly more reusable, testable, and flexible. For example, an LLM Command encapsulates the invocation of an LLM tool, such as OpenAI, Bedrock, Ollama, or others, based on the provided command context.

At the heart of event driven architecture is a workflow constructed from nodes and topics:

• Nodes: These are the functional units of the system. A node executes a specific tool through command in response to an event.
• Topics: These act as communication channels. Nodes produce messages to topics and consume messages from topics, enabling seamless communication.

Based on the pub/sub event driven architecture, a node publishes a message to a topic upon completing a command. Any node subscribed to that topic consumes the message and, when applicable, executes its corresponding command. This decoupled design forms the foundation for the system’s scalability, resilience, and modularity, enabling seamless communication and robust workflows.

Based on all above, here is the workflow design

Memory — A “Side Effect“ of Event Sourcing Pattern

In this post, “memory” refers specifically to how the agent retains the history of its conversation with the caller — distinct from the memory mechanisms typically found in transformer architectures.

One of the most powerful features of event sourcing is its immutable log of events, which effectively serves as a memory layer. Through this log, nodes can retrieve context-relevant information and maintain a coherent conversation across multiple interactions.:

• Instant Memory (Per Request): A node can look up relevant context from events generated within the current request. Each node in the directed graph aggregates the data from all of its ancestor nodes — i.e., any nodes from which it can be reached by following the graph’s edges backward — so that when the node executes, it has the complete set of data from every path leading into it.

• Short-Term Memory (Per Conversation): In conversational AI (e.g., chatbots), the system can maintain context by referencing the sequence of events that occurred in the current conversation.
• Long-term memory (or a knowledge base) can be generated from events too, and it’s another great topic we will address in a future post.

Note that the event store serves as the source of truth for the agent’s memory. Since each event is recorded at the tool level, an agent can assemble a relevant history of events to feed into an LLM node.

We see that adopting this design not only accelerates business growth through agile agent development, but also streamlines the engineering process by treating the agent as a standard service (Multi-Agent-as-a-Service — A Senior Engineer’s Overview) within a broader software ecosystem. By aligning with an established SDLC (software development life cycle), teams can iterate quickly, integrate new requirements seamlessly, and maintain a robust, scalable solution that meets evolving market demands. And the good news is, we’re already developing this agent framework. Here’s a quick look at what we’ve accomplished so far.

Build a ReAct Agent with Binome Agent Framework

The ReAct agent is a well-known agent design pattern. For a detailed explanation, refer to our post — AI Agent Workflow Design Patterns — An Overview. Below is a short code snippet illustrating how you might build a ReAct Agent using Binome Agent Framework:

# Create workflow builder
workflow_builder = EventDrivenWorkflow.Builder().name(
    "ReActAssistantWorkflow"
)

# Create thought result topic
thought_result_topic = Topic(name="thought_result")
# Create observation result topic
observation_result_topic = Topic(name="observation_result")
"""
Build thought node, which either agent_input_topic or observation_result_topic
can trigger this node, then publish to thought_result_topic. 
Given inputs, this node would use OpenAI to provide "thought".
"""
thought_node = (
    LLMNode.Builder()
    .name("ThoughtNode")
    .subscribe(
        SubscriptionBuilder()
        .subscribed_to(agent_input_topic)
        .or_()
        .subscribed_to(observation_result_topic)
        .build()
    )
    .command(
        LLMResponseCommand.Builder()
        .llm(
            OpenAITool.Builder()
            .name("ThoughtLLMTool")
            .api_key(self.api_key)
            .model(self.model)
            .system_message("""
You are an AI assistant tasked with analyzing the user's question and considering the provided observation to determine the next logical step required to answer the question. 
Your response should describe what would be the most effective action to take based on the information gathered.
If the information is sufficient to answer the question, return the answer with confirmation the answer is ready.
""")
            .build()
        )
        .build()
    )
    .publish_to(thought_result_topic)
    .build()
)
# Add thougt node to workflow.
workflow_builder.node(thougt_node)
# Create action result topics, each of them has its own publish condition.
action_result_search_topic = Topic(
    name="action_search_result",
    condition=lambda msgs: msgs[-1].function_call is not None,
)
action_result_finish_topic = Topic(
    name="action_finish_result",
    condition=lambda msgs: msgs[-1].content is not None
    and msgs[-1].content.strip() != "",
)
"""
Create action node, which subscribed to thought_result_topic, and publish to
two action result topics. 
This node uses OpenAI to decide finish the react loop or continue search the 
website to find the user's question. 
"""
action_node = (
    LLMNode.Builder()
    .name("ActionNode")
    .subscribe(thought_result_topic)
    .command(
        LLMResponseCommand.Builder()
        .llm(
            OpenAITool.Builder()
            .name("ActionLLMTool")
            .api_key(self.api_key)
            .model(self.model)
            .system_message("""
You are an AI assistant responsible for executing actions based on a given plan to retrieve information. 
Specify the appropriate action to take, such as performing a search query or accessing a specific resource, to gather the necessary data.
If answer is ready, return **FINISH REACT**. 
""")
            .build()
        )
        .build()
    )
    .publish_to(action_result_search_topic)
    .publish_to(action_result_finish_topic)
    .build()
)
# Add action node to workflow
workflow_builder.node(action_node)
# Create search result topic
search_function_result_topic = Topic(name="search_function_result")
# Create search function node, which subscribed to the action_result_search_topic
# This node will call search function
search_function_node = (
    LLMFunctionCallNode.Builder()
    .name("SearchFunctionNode")
    .subscribe(action_result_search_topic)
    .command(
        FunctionCallingCommand.Builder().function_tool(self.search_tool).build()
    )
    .publish_to(search_function_result_topic)
    .build()
)
# Add search function node to the workflow. 
# The function information will be regiested to action node.
workflow_builder.node(search_function_node)
"""
Create observation node, which subscribed to the search function result topic, 
and publish to observation result topic. 
This node use OpenAI to generate "observation" from user input, thought -> action results.
"""
observation_node = (
    LLMNode.Builder()
    .name("ObservationNode")
    .subscribe(search_function_result_topic)
    .command(
        LLMResponseCommand.Builder()
        .llm(
            OpenAITool.Builder()
            .name("ObservationLLMTool")
            .api_key(self.api_key)
            .model(self.model)
            .system_message("""
You are an AI assistant that records and reports the results obtained from executed actions. 
After performing an action, provide a clear and concise summary of the findings relevant to the user's question.
""")
            .build()
        )
        .build()
    )
    .publish_to(observation_result_topic)
    .build()
)
# Add observation node to workflow
workflow_builder.node(observation_node)
"""
Create the summaries node, which use OpenAI to summarize all the results from all above
This node will produce the message to agent_output_topic, which will be consumed by agent
and then respond to user. 
"""
summaries_node = (
    LLMNode.Builder()
    .name("SummariesNode")
    .subscribe(action_result_finish_topic)
    .command(
        LLMResponseCommand.Builder()
        .llm(
            OpenAITool.Builder()
            .name("SummariesLLMTool")
            .api_key(self.api_key)
            .model(self.model)
            .system_message("""
You are an AI assistant tasked with summarizing the findings from previous observations to provide a clear and accurate answer to the user's question. 
Ensure the summary directly addresses the query based on the information gathered.
""")
            .build()
        )
        .build()
    )
    .publish_to(agent_output_topic)
    .build()
)

# Add summaries_node to workflow
workflow_builder.node(summaries_node)

# Finally build the workflow
workflow = workflow_builder.build()

Below is the output workflow visualisation. Orange boxes represent topics, while navy-blue boxes represent nodes.

Observability is crucial for any AI system. Below is an example of how we use Phoenix by Arize to monitor real-time metrics and gain insights into our agents. The following screenshots illustrate the input of a “summarise” node, which includes historical data generated by various nodes. In sequence, the data points are: system message, user input, thought node, action node, search tool node, observation node, thought node, action node, and finally, the summarise node.

We hope you enjoy our work from above “glimpse”. And we are excited to announce that we plan to open-source our Agent Framework soon. Currently, we are polishing the codebase and adding more documentation, so stay tuned for future updates!

Conclusion

By combining event-driven architecture, event sourcing, and the command pattern, we create an agentic workflow that is robust, transparent, and easy to extend — key qualities for any modern AI system. This approach not only addresses practical challenges such as observability, scalability, and state restoration, but it also fosters agile development by treating AI Agents as standard services within a larger software ecosystem.

As we continue to refine and open-source our agent framework, we look forward to sharing more detailed examples, best practices, and real-world applications. Stay tuned for our upcoming posts, where we will dive deeper into advanced topics like long-term memory, conversation strategies, and more.

Design LLM-Based Agents: Key Principles — Part 2 was originally published in Binome on Medium, where people are continuing the conversation by highlighting and responding to this story.

In our previous post, we introduced AI Agent Design Patterns. Now, we are in the process of implementing an agent framework based on that design. At the same time, we are actively researching various workflow design patterns. Special thanks to Lychee Zhong for compiling these patterns with clear and informative diagrams.

Before diving deeper, let’s revisit the concept of an LLM-based AI agent. As described by the nvidia blog, LLM-powered agents are systems that utilize a large language model (LLM) to reason through problems, create actionable plans, and execute those plans using a set of tools. Simply put, these agents combine advanced reasoning capabilities, memory, and task execution.

The evolution of LLM-based AI agent workflow design patterns began in late 2022 and continues to evolve and innovate.

This is one of the earliest papers on LLM Agents. Although it may seem basic by today’s standards, at the time, ChatGPT hadn’t been released yet, and the idea of teaching LLMs to use tools was groundbreaking.

In the following section, we summarize the patterns we’ve learned and believe will be applicable to real-world business needs.

Introduction

Currently, there are numerous workflow designs. To summarize them based on their focus, we’ve categorized the workflow design patterns we’ve learnt so far into two groups: Reflection-focused and Planning-focused.

Reflection-focused

Reflection allows agents to learn from experience, improving adaptability and resilience. These agents emphasize introspection and learning from past experiences. They analyze previous actions and outcomes to refine future behaviour. By evaluating their performance, they identify mistakes and successes, allowing for continuous improvement. This reflective process enables the agent to adapt its strategies over time, leading to more effective problem-solving. [LangChain Blog]

• Basic Reflection: Reflecting and learning from the steps.
• Reflexion: Enhancing the next steps of the Agent through reinforcement learning.
• Tree search: TOT + reinforcement learning-based reflection.
• Self-Discover: Reasoning within the task.

Planning-focused

Planning enables agents to approach tasks methodically, increasing efficiency and effectiveness. These agents prioritize the development of structured plans before taking action. They decompose complex tasks into manageable sub-tasks and sequence them logically to achieve specific goals. By formulating detailed plans, these agents can anticipate potential challenges and allocate resources efficiently, resulting in more organized and goal-directed behaviour. [https://arxiv.org/pdf/2402.02716]

• Plan & Solve: Plan → Task list → RePlan.
• LLM compiler: Plan → Action in parallel → Joint execution.
• REWOO: Plan (including dependencies) → Action (depends on the previous step).
• Storm: Search for outline → Search each topic in the outline → Summarize into a long text.

When reviewing these design patterns, we see the workflow as an orchestrator. Each node can represent an LLM task, a function call, and other tasks like Retrieval-Augmented Generation (RAG) task, which we typically treat as another type of function call. This concept is one of the main drivers behind developing our own agent. We’ve designed the workflow as a flexible task orchestrator, allowing developers to create various workflows to tackle different problems.

In this post, we will explore two specific workflow design patterns: the ReAct pattern and the Plan-Solve pattern. For the remaining patterns, we’ll provide a brief summary to explain their motivation and use cases.

ReAct Pattern

The ReAct principle is straightforward and reflects a core aspect of human intelligence: “actions guided by verbal reasoning.” After each action, there’s an internal “Observation” or self-reflection: “What did I just do? Have I achieved my goal?” This enables the agent to retain short-term memory. Before ReAct, reasoning and action were treated as separate processes.

For example, imagine you ask someone to find a pen on your desk, and you give step-by-step instructions (similar to the Chain-of-Thought prompting strategy):

1. First, check the pen holder.
2. Then, look in the drawer.
3. Lastly, check behind the computer monitor.

Without ReAct, regardless of where the pen is found, the person would follow all the steps, checking each location (Action).

With ReAct, the process would look like this:

1. Action 1: First, check the pen holder;
2. Observation 1: The pen is not in the pen holder, so move to the next step;
3. Action 2: Then, check the drawer;
4. Observation 2: The pen is in the drawer;
5. Action 3: Take the pen from the drawer.

ReAct Implementation

After reviewing several open-source codes, let’s focus on the simplest one for analysis. As you explore it, you’ll notice that, at its essence, all Agent design patterns revolve around translating human thinking and management strategies into structured prompts. These prompts guide the large model to plan, invoke tools for execution, and continuously refine its approach through iteration.

The code logic is outlined in the diagram below (take a close look):

• Generate the prompt: First, the predefined ReAct prompt template (formatted as Question -> Thought -> Action -> Observation) is merged with the user’s question. The resulting prompt looks like this.

• Call the large model to generate Thought + Action: Next, send the few-shot prompt to the large model. If sent as is, the model will generate Thought, Action, and Observation responses. However, since the Action isn’t fully defined, we prevent the model from generating the Observation by using Stop.Observation, ensuring it stops after Thought and Action.

• Calling External Tools: Once the Action is obtained, the model can call external tools. First, check if the Action is “Finish.” If not, the model converts the Action into an API-compatible format using its function-calling feature. This fine-tunes the model for language-to-API conversion, though not all large models support function calling.
• Generating Observation: After the API interface returns the result, it is converted into natural language output to generate the Observation. Then the Observation, along with the previously generated Thought and Action, is input back into the model, repeating steps 2 and 3 until the Action is “Finish.”
• Final Output: The final Observation is converted into natural language and output to the user.

From this, we can see that implementing an Agent in a specific scenario requires customizing two key components:

1. The few-shot examples in the prompt template
2. The definition of external tools for function calling

The few-shot examples essentially mirror structured human thinking patterns. Reviewing prompt templates for different design patterns is a great way to understand Agent design. Once you grasp this approach, it can be applied to other design patterns similarly.

Plan and Solve Pattern

As the name suggests, this design pattern involves planning first and then executing. If ReAct is more suitable for tasks like “getting pen from the the desk” then Plan & Solve is better suited for tasks like “making a cup of flat white.” You need to plan, and the plan might change during the process (for example, if you open the fridge and find no milk, you would add “buy milk” as a new step in the plan).

Regarding the prompt template, the paper’s title makes it clear: Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models. In short, it’s about improving zero-shot capabilities. The following image shows some PS-Plan and Solve prompts provided in the author’s code.

Its architecture is composed as follows:

• Planner: Responsible for enabling the LLM to generate a multi-step plan to complete a large task. In the code, there are both a Planner and a Replanner. The Planner is responsible for generating the plan initially; the Replanner, on the other hand, comes into play after each individual task is completed, adjusting the plan based on the current progress. Therefore, the Replanner’s prompt includes not only the zero-shot input but also the goal, the original plan, and the status of completed steps.
• Executor: Receives the user’s query and the steps from the plan, then calls one or more tools to accomplish the task.

Other Workflow Design Patterns

Here we give the brief introduction of other design patterns listed above.

Reason without Observation (REWOO)

REWOO (Reason without Observation) is a variation on the observation process seen in ReAct. While ReAct follows the structure: Thought → Action → Observation, REWOO simplifies this by removing the explicit observation step. Instead, it implicitly embeds observation into the next execution unit. In practice, the next executor automatically observes the outcome of the previous step, streamlining the process.

LLMCompiler

In computer science, a compiler refers to the orchestration of tasks to optimize computational efficiency. The concept behind An LLM Compiler for Parallel Function Calling, as outlined in the original paper, is simple yet effective: it aims to enhance efficiency by enabling parallel function calls. For instance, if a user asks, “What’s the difference between AWS Glue and MWAA?”, the compiler would search for the AWS services definition simultaneously and combine the results, rather than handling each query sequentially.

Basic Reflection

Basic Reflection can be compared to a feedback loop between a student (the Generator) and a teacher (the Reflector). The student completes an assignment, the teacher provides feedback, and the student revises their work based on this feedback, repeating the cycle until the task is satisfactorily completed.

Reflexion

Reflexion builds on Basic Reflection, incorporating principles of reinforcement learning. Described in the paper Reflexion: Language Agents with Verbal Reinforcement Learning, this approach goes beyond simple feedback. It evaluates the response using external data and forces the model to address any redundancies or omissions, making the reflective process more robust and the output more refined.

Language Agent Tree Search (LATS)

LATS is detailed in the paper Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models. It combines several techniques, including Tree Search, ReAct, and Plan & Solve. LATS uses tree search to assess outcomes (drawing from reinforcement learning), while also integrating reflection to achieve optimal results. In essence, LATS can be represented by the following formula:

LATS = Tree Search + ReAct + Plan & Solve + Reflection + Reinforcement Learning.

In terms of prompt design, the difference between LATS and earlier methods such as Reflection, Plan & Solve, and ReAct is minimal. The key addition is the tree search evaluation step and the return of those evaluated results within the task context. Architecturally, LATS involves multiple rounds of Basic Reflection, with several Generators and Reflectors working collaboratively.

Self-Discovery

The core of Self-Discovery is to allow the large model to reflect at a more granular level. While Plan & Solve focuses on whether a task requires additional steps or adjustments, Self-Discovery goes further, encouraging reflection on the task itself. This involves evaluating each component of the task and the execution of those components.

Storm

Storm, outlined in the paper Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models, introduces a workflow for generating comprehensive articles from scratch, similar to a Wikipedia entry. The agent first uses external tools to search for information and generate an outline. Then, it produces content for each section based on the outline, making the process structured and efficient.

Conclusion

In this post, we explore various AI agent workflow design patterns, we categorize the patterns into Reflection-focused and Planning-focused, showcasing examples like the ReAct and Plan-Solve patterns. For developing purposes, these workflow patterns emphasize that workflows act as orchestrators for tasks within an agent. Each node in the workflow represents actions such as LLM tasks, function calls, or Retrieval-Augmented Generation (RAG). This structure enables the agent to plan, execute tasks, and iterate, simulating human reasoning.

We are currently developing our own agent based on these principles, utilizing workflows as flexible task orchestrators to handle different problems. More details on our agent implementation will be covered in the next post.

AI Agent Workflow Design Patterns — An Overview was originally published in Binome on Medium, where people are continuing the conversation by highlighting and responding to this story.

An Agent’s function calling execution capability heavily depends on the underlying model.

LLM Function Calling Overview

When it comes to function calling in large language models (LLMs), the ability of a model to accurately respond to a function call request based on input prompts is absolutely crucial. In our experiments, we’ve seen how testing the same prompt across different models can lead to significantly varied results, which really highlight the importance of choosing the right model.

UC Berkeley recently released a function calling leaderboard, where the top-performing GPT-4 model achieves an accuracy rate of 85.65% (as of 28 Aug 2024).

Berkeley Function Calling Leaderboard (BFCL) V4

That’s impressive, but let’s be honest — it’s still not where we need it to be for business requirements.

Unfortunately, developers have limited control over an LLM’s function calling capabilities. Even with precise definitions of API functions, parameter names, and descriptions — often well-documented by model providers — there are still challenges.

• OpenAI’s function calling guide: OpenAI Platform
• Google’s function calling guide: Google Cloud Function Calling
• Anthropic’s function calling guide: Tool use (function calling) — Anthropic

Given that execution accuracy remains less than perfect — topping at around 85.65% as per above — , this kind of inaccuracy is simply not acceptable for many business needs. That’s why it’s so important to have fallback strategies in place when function calling doesn’t go as planned. For example:

• Redirecting to human support in customer service scenarios.
• Translating API error messages into language that’s easy for users to understand and act on

In Chapter 1 of our proposed Multi-agent Framework, we talked about how an agent should be designed with a specific-enough task in mind to perform well. In this article, we — my colleague Eisha, and I — stick to that principle, and share our experiment with function calling, which we built as part of a Proof of Concept (PoC) for a recent collaboration project. We’ll also walk you through what we like to call the Agent Development Life Cycle, covering everything from gathering requirements to creating a ChatBot powered by an AI agent. We hope you’ll enjoy this journey with us.

Requirement & Design

The overall platform requirement was fairly substantial, we therefore attempted to break it into smaller domains and deliverables. One specific task was to build an API that allows users to find schools given a location, starting with UK postcodes. To make the agent’s task specific enough, we decided to use just one function. Here’s the design:

1. A dataset containing schools and their geolocation.
2. A function that takes the geolocation and finds schools in the dataset.
3. An AI Agent that handles user input for school search and calls the function to generate responses.

Implementation

We sourced datasets (such as schools data and postcode-to-coordinate data, from UK government public data repositories), then built a data pipeline to process the required data. We used SQLMesh for the data pipeline and DuckDB as the database. Here’s a glimpse of the pipeline lineage:

The function is straightforward: given a postcode, we convert it to coordinates, and given the coordinates, we find the closest schools and return the top N results.

from phi.tools import Toolkit
 ...
 class SchoolSearch(Toolkit):
    ...
    # Function to find nearest schools
    def find_nearest_schools(self, postcode: str, num_neighbors: int = 5) -> dict:
        """Runs a number of neighbor's schools which close to the postcode
        Args:
            postcode (str): the valid UK postcode
            num_neighbors (int): The number of schools will be returned, default to 5
        Returns:
            DataFrame: the output schools in a dict
        """
        logger.info(f'Function is called {postcode}, {num_neighbors}')
        file_dir = Path(__file__).parent
        db_path = file_dir / 'fact.duckdb'
        with db.connect(str(db_path.absolute())) as connection:
            # Get coordinates for the provided postcode
            user_coords = self.get_coordinates_from_postcode(postcode, connection)
            logger.info(f'coords {user_coords}')
            if user_coords is None or user_coords.empty:
                return f"Cannot find school given postcode {postcode}"# Handle case where postcode is not found
            user_lat = user_coords.iloc[0]['latitude']
            user_lon = user_coords.iloc[0]['longitude']
            # Get all school data
            df = self.get_all_school_data(connection)
            # Calculate distance for each school
            df['distance'] = df.apply(lambda row: self.haversine(user_lat, user_lon, row['latitude'], row['longitude']), axis=1)
            # Sort by distance and get the top N nearest schools
            nearest_schools = df.sort_values(by='distance').head(num_neighbors)
            return nearest_schools.to_json(orient='records')

To create the Agent, here we used PhiData

from phi.assistant import Assistant
from phi.llm.openai import OpenAIChat
...
assistant = Assistant(
                tools=[SchoolSearch()],
                llm = OpenAIChat(model='gpt-4o-mini', api_key='<api-key>')
                show_tool_calls=True
            )
...

We then built a FastAPI service around this agent and a simple Chatbot WebUI. Here’s a screenshot:

It does the job, though better prompt engineering would definitely improve responses. Now, let’s move on to the fun part — testing!

Testing and Eval

We’ll skip unit tests and jump straight into smoke testing. We’ve designed prompts and expected outputs like the following.

index,prompt,function_called,function_args_postcode,function_args_number_schools,expected_response
1,"Find like 10 schools near SW1A 1AA.",true,SW1A 1AA,10,"show nearby schools, the school number should be 10"
2,"What are the best schools around W1A.",false,,,ask user to provide the postcode instead
3,"Can you list 7 schools close to EC2N 2DB.",true,EC2N 2DB,7,"show nearby schools, the school number should be 7"
...
39,"Translate 'hello' to French.",false,,,explain to user that this agent is designed to show schools close to a postcode provided by user
40,"What's the time in New York right now.",false,,,explain to user that this agent is designed to show schools close to a postcode provided by user

The first 20 inputs focused on searching for schools, while the remaining 20 were unrelated dummy prompts. Here is the full list.

Given these prompts, we want to evaluate the agent’s performance. The agent should only respond to school search requests and reject irrelevant queries by providing helpful instructions. In smoke testing, we’ll also test different models like Openai GPT-4o, Llama 7B, and other open-source models that can run locally on my machine with Ollama. Here’s the list:

gpt-4o
gpt-4o-mini
llama3
llama3.1
llama3-groq-tool-use
phi3:medium
phi3.5
gemma2

We built an evaluation function to check function calling, and using gpt-4o-mini to check if response (Return usage to user and Correct Response) was as expected. Here’s the flow:

Then the result is like this

So can we improve the agent’s performance? Since we have limited control over the function call from the model, can we provide better instructions in the prompt? Here’s the updated agent description and instructions:

"description": "You help users find schools near a specific UK postcode. You process queries only when asked about schools around or near a UK postcode and responds with a list based on function call.",
"instructions": [
    'Process input only if it requests information about schools around or near a UK postcode',
    'Handle valid queries that contain a UK postcode or clearly indicate a need for school information based on location.',
    'If the query is unclear or lacks a valid UK postcode, prompt the user to provide more information.',
    'DO NOT use function calling if the query is unclear or lacks a valid UK postcode.',
    'Ensure the agent only processes relevant requests and guides users on its proper usage.'
],

And here is the results with prompt

And a bar chart to compare with and without prompt:

After analyzing the results ourselves, we discovered that the llama3-groq-tool-use model actually could perform better than llama3, llama3.1, phi3.5, and phi3. However, the LLM used in our evaluation function did not accurately assess the responses from the llama3-groq-tool-use model, indicating a need for further refinement in our evaluation LLM’s prompting strategy, which leads to the next round of Agent Development Life Cycle.

Since we modified some of the dependent code, it might be challenging to reproduce the setup. Therefore, we’ve built a Docker image. We simplified the code, exposed some variables, and created a docker-compose file, allowing you to try the agent we build with OpenAI GPT-4o-mini. You can run the docker compose and open a browser (http://localhost:8000/) to try the school search agent.

version: '3.8'
services:
  binome-school-search-agent:
    image: guanyili/binome-school-search-agent
    environment:
      ENV: docker
      OPENAI_API_KEY: <your-openai-api-key>
    ports:
      - "8000:8000"
    extra_hosts:
      - "host.docker.internal:host-gateway"

Summary

In this experiment, we detailed our journey from initial requirements to the creation of a fully functional AI Agent ready for demonstration. Throughout this process, we highlighted the critical importance of selecting the right model for function calling in AI agents. We also explored fallback strategies and refined prompt techniques to enhance the agent’s performance. While function calling accuracy is still an area requiring further development, this experiment establishes a strong foundation for building and improving AI agents capable of handling business tasks.

Additionally, we identified a key issue that needs to be addressed: the need to trace data flow within the agent. We prefer to not rely on the LangChain ecosystem (especially considering that LangSmith is not free), therefore we plan to tackle agent Observability in the upcoming post(s) using widely adopted software design patterns. Another important thing to address would be, how would things change in a multi-agent setup, i.e. where there is a need for orchestration and aggregation of values returned by agents (into one unified and cohesive response sent back to the users).

Stay tuned for more insights as we continue to publicly refine and advance our approach!

AI Agent Function Calling: What Really Matters was originally published in Binome on Medium, where people are continuing the conversation by highlighting and responding to this story.