<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Igor Dayen on Medium]]></title>
        <description><![CDATA[Stories by Igor Dayen on Medium]]></description>
        <link>https://medium.com/@igor.dayen?source=rss-81e4c14289d9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Usqke0KAotxILCmpUe7rIw.png</url>
            <title>Stories by Igor Dayen on Medium</title>
            <link>https://medium.com/@igor.dayen?source=rss-81e4c14289d9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 04:25:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@igor.dayen/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Embabel Agentic AI Framework Release 1.0.0: New Features]]></title>
            <link>https://medium.com/embabel/embabel-agentic-ai-framework-release-1-0-0-new-features-332f375d0fba?source=rss-81e4c14289d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/332f375d0fba</guid>
            <category><![CDATA[embabel]]></category>
            <category><![CDATA[agentic-ai-framework]]></category>
            <category><![CDATA[agentic-rag]]></category>
            <dc:creator><![CDATA[Igor Dayen]]></dc:creator>
            <pubDate>Mon, 20 Jul 2026 12:43:11 GMT</pubDate>
            <atom:updated>2026-07-30T23:53:22.080Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This blog is co-written with the Embabel Development Team.</em></p><p><strong>Embabel Agentic Framework has just reached a long-awaited milestone release, 1.0.0.</strong></p><p>You may take a look at <a href="https://medium.com/me/following-feed/publications/33b58d773744">Embabel&#39;s</a> blogs and at separate blogs published by Embabel founder <a href="https://medium.com/u/de5947672915">Rod Johnson</a>, creator of the Spring Framework.</p><p>Also, <a href="https://medium.com/u/2a4443d21452">Craig Walls</a> and <a href="https://medium.com/u/4fcb0571c65d">Jettro Coenradie</a> recently published a few <a href="http://thetalkingapp.medium.com/">blog posts</a> on <a href="http://github.com/embabel">Embabel</a>.</p><p>Please refer to the <a href="https://docs.embabel.com/embabel-agent/guide/1.0.0/index.html#reference.integrations">User Guide</a> for features, solutions, and APIs.</p><p>Please try <a href="http://hub.embabel.com/">Embabel HUB</a>, an Embabel-powered learning tool. You can use voice as well as a communication channel with HUB. Choose one of the built-in personalities, or make your own. HUB uses many of the features covered in this blog: <em>ToolishRag</em>, chat-store, ONNX local embeddings, and more.</p><p>Here are some of the recently added features and other features not fully covered in previous blogs.</p><h3>Structured Output</h3><ul><li><strong>Best-Effort (</strong><strong>createObjectIfPossible)</strong> - returns null instead of throwing when the LLM response cannot be parsed; the GOAP planner automatically replans around the missing output.<br>Use this when incomplete results are acceptable, and you want the planner to find an alternate path rather than fail the entire process. ¹</li><li><strong>Streaming</strong> — Reactive Flux-based streaming with three modes: typed objects, objects + thinking blocks (StreamingEvent&lt;T&gt;), and raw text chunks.<br>Use this when rendering progressive results in a UI or streaming responses to end users without waiting for the full object. ²</li></ul><h3>Native — Anthropic</h3><ul><li><strong>Prompt Caching</strong> — cache system prompt, tool definitions and/or conversation history; cache reads cost 10% of regular input tokens.<br>Use this when making repeated calls against the same large system prompt or document — cache hits reduce input token cost by 90%. ³</li></ul><h3>Native — Anthropic &amp; Google GenAI</h3><ul><li><strong>Native Thinking</strong> — PromptRunner.thinking() is the provider-neutral API for extended reasoning; the framework translates it to provider-native payloads: Anthropic sends ThinkingConfig(ThinkingType.ENABLED, budget), Google GenAI sets includeThoughts and thinkingBudget. Token budget controlled via LlmOptions.withThinking(Thinking.withTokenBudget(n)); supportsThinking() guards capability at runtime.<br>Use this when you need the model&#39;s reasoning enforced at the provider API level. ¹⁷</li></ul><h3>Native — OpenAI</h3><ul><li><strong>Structured JSON Output</strong> — automatically sends output schema via response_format: {type: json_schema}; API-level enforcement eliminates JSON parse failures. Conservative schema compatibility check with silent fallback to prompt-based extraction.<br>Use this when JSON parse failures are unacceptable, and schema compliance must be guaranteed at the provider API level. ⁴</li></ul><h3>Provider-neutral</h3><ul><li><strong>Thinking / Reasoning Support</strong> — uniform runner.thinking() API across Anthropic, OpenAI, Google GenAI and Ollama. Auto-detects dynamic XML tags (&lt;think&gt;, &lt;analysis&gt;, any custom tag), returns multiple ordered ThinkingBlocks, and preserves thinking blocks even when object parsing fails.<br>Use this when you want to observe or validate the model&#39;s reasoning process, irrespective of whether the call succeeds or fails. ⁵</li></ul><h3>Tool Loop Operations</h3><ul><li><strong>Tool Call Inspector &amp; Tool Loop Callbacks</strong> — two complementary hook levels: ToolCallInspector (beforeToolCall / afterToolCall) observes individual tool executions in both streaming and non-streaming modes; ToolLoopInspector (read-only) and ToolLoopTransformer (mutating) hook at every round of the multi-round loop (beforeLlmCall, afterLlmCall, afterToolResult, afterIteration). Built-in transformers include SlidingWindowTransformer and ToolResultTruncatingTransformer.<br>Use this when you need auditing, metrics, or cost tracking without modifying agent or tool logic. ⁶</li><li><strong>Thinking Across Tool Loop</strong> — thinking blocks accumulated from every LLM call in the multi-round tool loop, not just the final response; ThinkingResponse.thinkingBlocks contains reasoning from all iterations.<br>Use this when you need to understand the full reasoning chain across a multi-step agent execution, not just the final answer. ⁷</li><li><strong>Parallel Tool Loop</strong> — independent tool calls within a single LLM turn execute concurrently; enabled via a single YAML property.<br>Use this when tool calls within a turn are independent — e.g. fetching data from multiple sources simultaneously. ⁹</li></ul><h3>Tools</h3><ul><li><strong>Progressive / Folding Tools (</strong><strong>UnfoldingTool)</strong> - hides inner tools until the LLM invokes the facade; two-stage disclosure: short description always visible, childToolUsageNotes revealed only after the LLM commits. Supports flat, category-based, and exclusive modes.<br>Use this when a large toolset degrades model planning quality or hits token limits on the tool description list. ¹⁰</li><li><strong>ToolishRag</strong> — RAG exposed as typed tool calls (LlmReference + DelegatingTool); the model explicitly decides when and what to retrieve rather than having context silently injected.<br>Use this when you want the model to explicitly decide when and what to retrieve from a knowledge base or data pipeline, invoking it as a typed tool call rather than having documents silently injected into the prompt. ¹¹</li></ul><h3>Safety</h3><ul><li><strong>Guardrails</strong> — pre-LLM input validation (UserInputGuardRail) and post-response analysis (AssistantMessageGuardRail); CRITICAL Severity blocks the call; lower severities log only.<br>Use this when safety policies, PII checks, or content moderation must be enforced without coupling those concerns to agent or action logic. ¹²</li></ul><h3>Lifecycle</h3><ul><li><strong>Termination APIs</strong> — immediate (exception) and graceful (signal) termination, each with agent-scope and action-scope variants; sealed TerminationException base for joint catch blocks.<br>Use this when an action detects a condition (budget exhausted, invalid input, skippable record) that should stop the current work without crashing the entire agent process. ¹³</li></ul><h3>Observability</h3><ul><li><strong>Observability &amp; Budget Enforcement</strong> — LlmInvocationEvent carries per-call cost and token usage; ProcessOptions.withBudget() hard-stops the agent when cumulative spend or action count exceeds the ceiling.<br>Use this when you need hard spending limits and real-time cost visibility per agent run. ¹⁴</li><li><strong>Token Budget Estimates</strong> — TokenCountEstimator&lt;String&gt; for pre-call token estimation; attach as TokenBudgetGuardRail or read BeforeLlmCallContext.tokenEstimate inside ToolLoopInspector.<br>Use this when you want to prevent expensive LLM calls with oversized inputs before they happen, or enforce per-action context size limits. ¹⁵</li></ul><h3>BYOK (Bring Your Own Key)</h3><ul><li><strong>Provider Detection &amp; Key Validation</strong> — ByokFactory&lt;T&gt; validates a user-supplied API key with a low-cost probe call and returns a production LlmService on success, or throws InvalidApiKeyException on failure. detectProvider() races multiple factories concurrently using virtual threads and returns the first that validates - no need to know the provider upfront. Built-in factories: AnthropicModelFactory, OpenAiCompatibleModelFactory.openAi/deepSeek/mistral/gemini. Custom OpenAI-compatible providers supported via OpenAiCompatibleModelFactory.byok().<br>Use this when your product lets users supply their own API keys - eliminates per-user inference cost and separates usage cost from product value. ¹⁸</li></ul><h3>Provider SPI</h3><ul><li><strong>LlmMessageSender / LlmMessageStreamer</strong> — framework-agnostic SPI interfaces to plug in any LLM backend; non-streaming: Embabel owns the tool loop; streaming: tool execution is framework-managed, only ToolCallInspector observation possible.<br>Use this when you need custom integration with LLM provider APIs while retaining Embabel&#39;s full agent loop. ¹⁶</li></ul><h3>Skills</h3><ul><li><strong>Skills Engine</strong> — lazy-loading plugin system for composable agent capabilities. Skills are defined by a YAML frontmatter + Markdown body (SKILL.md) following the <a href="https://agentskills.io/specification">Agent Skills Specification</a>. The system prompt carries only name and description (~50-100 tokens per skill); activate fetches full instructions on demand. Scripts in the scripts/ subdirectory become LLM-callable tools. Load from local directories or GitHub repositories; filter with SkillFilter. Use asIndividualReferences() to expose each skill as its own top-level tool with per-loop repeat suppression.<br>Use this when you have a large, domain-specific instruction library that would overflow the context window if injected in full, or when skills are shared across agents and teams. ¹⁹</li></ul><h3>Security</h3><ul><li><strong>Secure MCP (</strong><strong>@SecureAgentTool)</strong> - Spring Security SpEL enforcement on @Action methods exposed as MCP tools. Annotate at method or class level; SecureAgentToolAspect intercepts via AOP and evaluates the expression against the current Authentication before the action body executes - same engine as @PreAuthorize. SecureAgentSecurityConfiguration (activated by the secured profile) adds stateless JWT Bearer token validation on all /sse/, /mcp/ and /message/ endpoints. Method-level annotation takes precedence over class-level.<br>Use this when your MCP server exposes agent tools to external clients,s and access must be scoped per-tool or per-agent by role or authority. ²⁰</li></ul><h3>Conversation Persistence</h3><ul><li><strong>Chat Store Support</strong> — ConversationFactory SPI with two storage strategies: IN_MEMORY (fast, ephemeral) and STORED (persistent). ConversationFactoryProvider resolves the right factory by ConversationStoreType at chatbot creation time. The InMemoryConversationFactory is built-in; persistent storage is provided by the <a href="https://github.com/embabel/embabel-chat-store">embabel-chat-store</a> module. Conversations can be created for named participants (user + agent) and reloaded by ID across sessions.<br>Use this when you need conversation history to survive server restarts, be queried for analytics, or be shared across multiple agent instances. ²¹</li></ul><h3>Embeddings</h3><ul><li><strong>ONNX Embedding Models</strong> — run embedding inference in-process via ONNX Runtime + DJL HuggingFace tokenizer; no external embedding API required. Default model: all-MiniLM-L6-v2 (384 dimensions). Auto-configured via OnnxEmbeddingAutoConfiguration when embabel.agent.platform.models.onnx.embeddings.enabled=true. Model and tokenizer files are downloaded from HuggingFace on first use and cached at~/.embabel/models; set. model-uri/tokenizer-uri to file:// URIs to use pre-downloaded files and avoid network access at startup.<br>Use this when you want zero-latency, zero-cost embeddings with no external dependency, or when your deployment cannot reach external embedding APIs. ²²</li></ul><h3>Code Examples</h3><h3>1. Structured Output Best-Effort</h3><pre>// Called on PromptRunner, not on context directly:<br>val person: Person? = context.promptRunner()<br>    .createObjectIfPossible(input.text, Person::class.java)<br>// Reified Kotlin shorthand:<br>val person: Person? = context.promptRunner()<br>    .createObjectIfPossible&lt;Person&gt;(input.text)</pre><h3>2. Structured Output Streaming</h3><pre>// Check streaming support and subscribe to partial results:<br>val runner = context.promptRunner()<br>if (runner.supportsStreaming()) {<br>    // Objects only:<br>    runner.streaming()<br>        .withPrompt(query)<br>        .createObjectStream(Report::class.java)  // Flux&lt;Report&gt;<br>        .subscribe { partial -&gt; updateUI(partial) }<br>    // Objects + thinking (StreamingEvent.Object / StreamingEvent.Thinking):<br>    runner.streaming()<br>        .withPrompt(query)<br>        .createObjectStreamWithThinking(Report::class.java)  // Flux&lt;StreamingEvent&lt;Report&gt;&gt;<br>        .subscribe { event -&gt;<br>            event.fold(<br>                left = { thinking -&gt; log.debug(thinking) },<br>                right = { obj -&gt; updateUI(obj) }<br>            )<br>        }<br>}</pre><h3>3. Anthropic Prompt Caching</h3><pre>// Cache system prompt and tools; cache hits cost 10% of regular input tokens:<br>val runner = context.promptRunner()<br>    .withLlm(LlmOptions().withAnthropicCaching(<br>        systemPrompt = true,<br>        tools = true,<br>        conversationHistory = false,<br>    ))<br>val summary = runner.createObject(largeDocument.content, Summary::class.java)</pre><h3>4. Native Structured JSON Output</h3><pre>// Default: native output used automatically for compatible OpenAI models - no code change required:<br>val result = context.promptRunner()<br>    .createObject(prompt, Analysis::class.java)<br>// Explicit control:<br>val runner = context.promptRunner()<br>    .withLlm(LlmOptions().withNativeStructuredOutput(NativeStructuredOutputMode.ENABLED))<br>val result = runner.createObject(prompt, Analysis::class.java)<br>// Disable (fall back to prompt-based JSON extraction):<br>val runner = context.promptRunner()<br>    .withLlm(LlmOptions().withNativeStructuredOutput(NativeStructuredOutputMode.DISABLED))<br>val result = runner.createObject(prompt, Analysis::class.java)</pre><h3>5. Thinking / Reasoning Support</h3><pre>// Check thinking support, extract blocks by type or tag, and control token budget:<br>val runner = context.promptRunner()<br>if (runner.supportsThinking()) {<br>    val response: ThinkingResponse&lt;ComplexAnalysis&gt; = runner<br>        .thinking()<br>        .createObject(query, ComplexAnalysis::class.java)<br>    // result is null and exception is set if parsing failed - thinking blocks still present:<br>    if (response.result != null) {<br>        process(response.result)<br>    } else {<br>        log.warn(&quot;Failed: ${response.exception?.message}&quot;)<br>    }<br>    // Multiple blocks extracted and ordered by position; filter by type or tag name:<br>    response.getThinkingByType(ThinkingTagType.TAG)    // &lt;think&gt;, &lt;analysis&gt;, dynamic tags<br>    response.getThinkingByTag(&quot;analysis&quot;)              // only &lt;analysis&gt; blocks<br>    response.thinkingBlocks.forEach { block -&gt;<br>        // block.tagType  - TAG | PREFIX | NO_PREFIX<br>        // block.tagValue - &quot;think&quot; / &quot;analysis&quot; / &quot;THINKING&quot; / &quot;plan&quot; (dynamic) / &quot;&quot;<br>        // block.content  - reasoning text with all markup stripped<br>        log.info(&quot;[${block.tagType}/${block.tagValue}] ${block.content.take(200)}&quot;)<br>    }<br>    // Explicit token budget (controls reasoning depth):<br>    context.promptRunner()<br>        .withLlm(LlmOptions().withThinking(Thinking.withTokenBudget(8000)))<br>        .thinking()<br>        .createObject(query, ComplexAnalysis::class.java)<br>}</pre><h3>6. Tool Call Inspector</h3><pre>// Observe individual tool calls before and after execution:<br>class AuditInspector : ToolCallInspector {<br>    override fun beforeToolCall(context: BeforeToolCallContext) {<br>        log.info(&quot;-&gt; ${context.toolCall.name}: ${context.toolCall.arguments}&quot;)<br>    }<br>    override fun afterToolCall(context: AfterToolCallContext) {<br>        log.info(&quot;&lt;- ${context.toolCall.name} (${context.durationMs}ms): ${context.result}&quot;)<br>    }<br>}<br>// Works in both streaming and non-streaming:<br>context.promptRunner()<br>    .withToolCallInspectors(AuditInspector())<br>    .createObject(prompt, Result::class.java)</pre><h3>7. Thinking Across Tool Loop</h3><pre>// Thinking blocks from ALL tool loop iterations are aggregated into the final response:<br>val runner = context.promptRunner()<br>if (runner.supportsThinking()) {<br>    val response: ThinkingResponse&lt;Analysis&gt; = runner<br>        .thinking()<br>        .createObject(query, Analysis::class.java)<br>    // thinkingBlocks contains reasoning from every LLM call in the loop, in order:<br>    response.thinkingBlocks.forEach { block -&gt;<br>        log.info(&quot;${block.tagType}: ${block.content.take(200)}&quot;)<br>    }<br>}</pre><h3>8. Tool Loop Callbacks</h3><pre>// Read-only inspector - observe without modifying:<br>class MetricsInspector : ToolLoopInspector {<br>    override fun beforeLlmCall(context: BeforeLlmCallContext) {<br>        log.info(&quot;Round ${context.iteration}, history=${context.history.size}&quot;)<br>    }<br>    override fun afterLlmCall(context: AfterLlmCallContext) {<br>        context.usage?.let { metrics.record(it.promptTokens, it.completionTokens) }<br>    }<br>    override fun afterToolResult(context: AfterToolResultContext) {<br>        log.debug(&quot;Tool ${context.toolCall.name}: ${context.resultAsString.length} chars&quot;)<br>    }<br>    override fun afterIteration(context: AfterIterationContext) {<br>        if (budgetExceeded()) throw TerminateActionException(&quot;Budget exceeded&quot;)<br>    }<br>}<br>// Mutating transformer - e.g. sliding window to bound context size:<br>val transformer = SlidingWindowTransformer(maxMessages = 20)<br>// Attach to PromptRunner:<br>context.promptRunner()<br>    .withToolLoopInspectors(MetricsInspector())<br>    .withToolLoopTransformers(transformer)<br>    .withToolCallInspectors(ToolCallLoggingInspector())<br>    .createObject(prompt, Result::class.java)</pre><h3>9. Parallel Tool Loop</h3><pre># Enable concurrent execution of independent tool calls within a single LLM turn:<br>embabel:<br>  agent:<br>    platform:<br>      toolloop:<br>        type: parallel<br>        parallel:<br>          per-tool-timeout: 30s<br>          batch-timeout: 60s</pre><h3>10. Progressive / Folding Tools</h3><pre>// Annotation-based (auto-detects categories from @LlmTool.category):<br>@UnfoldingTools(<br>    name = &quot;db_ops&quot;,<br>    description = &quot;Database operations. Invoke to see specific tools.&quot;,<br>    childToolUsageNotes = &quot;Use query for reads, insert for writes.&quot;<br>)<br>class DatabaseTools {<br>    @LlmTool(description = &quot;Execute a SQL query&quot;, category = &quot;read&quot;)<br>    fun query(sql: String): QueryResult { ... }<br>    @LlmTool(description = &quot;Insert a record&quot;, category = &quot;write&quot;)<br>    fun insert(table: String, data: String): InsertResult { ... }<br>}<br>val tool = UnfoldingTool.fromInstance(DatabaseTools())<br>context.promptRunner()<br>    .withTool(tool)<br>    .createObject(query, Output::class.java)<br>// Programmatic category-based:<br>val fileTool = UnfoldingTool.byCategory(<br>    name = &quot;file_ops&quot;,<br>    description = &quot;File operations.&quot;,<br>    toolsByCategory = mapOf(<br>        &quot;read&quot;  to listOf(readFileTool, listDirTool),<br>        &quot;write&quot; to listOf(writeFileTool, deleteTool),<br>    )<br>)</pre><h3>11. ToolishRag</h3><pre>// Expose a knowledge base as typed search tools for explicit LLM-driven retrieval:<br>val rag = ToolishRag(<br>    name = &quot;knowledge-search&quot;,<br>    description = &quot;Search the knowledge base for relevant information&quot;,<br>    searchOperations = knowledgeBase,<br>)<br>val answer = context.promptRunner()<br>    .withReference(rag)<br>    .createObject(question.text, Answer::class.java)</pre><h3>12. Guardrails</h3><pre>// Pre-call input validation and post-call response checks; CRITICAL severity blocks the call:<br>class PiiGuardRail : UserInputGuardRail {        // runs before LLM call<br>    override val name = &quot;PiiCheck&quot;<br>    override val description = &quot;Blocks PII&quot;<br>    override fun validate(input: String, blackboard: Blackboard) = ...<br>}<br>class ToxicityGuardRail : AssistantMessageGuardRail {  // runs after LLM call<br>    override val name = &quot;ToxicityCheck&quot;<br>    override val description = &quot;Checks response&quot;<br>    override fun validate(input: String, blackboard: Blackboard) = ...<br>}<br>// Attach per-call:<br>context.promptRunner()<br>    .withGuardRails(PiiGuardRail(), ToxicityGuardRail())<br>    .createObject(prompt, Response::class.java)<br>// Or register globally as a Spring bean:<br>@Component<br>class GlobalPiiGuardRail : UserInputGuardRail { ... }</pre><h3>13. Termination APIs</h3><pre>// Immediate - stops tool loop now:<br>throw TerminateAgentException(&quot;Budget exhausted&quot;)     // entire agent -&gt; TERMINATED<br>throw TerminateActionException(&quot;Skipping this step&quot;)  // current action only; planner replans<br>// Graceful - stops at next checkpoint:<br>processContext.terminateAgent(&quot;Shutdown requested&quot;)<br>processContext.terminateAction(&quot;Skip at next opportunity&quot;)<br>// Catch either immediate variant:<br>catch (e: TerminationException) { log.info(e.reason) }</pre><h3>14. Observability &amp; Budget Enforcement</h3><pre>// Per-call cost observation:<br>@Component<br>class CostTracker : AgenticEventListener {<br>    override fun onProcessEvent(event: AgentProcessEvent) {<br>        if (event is LlmInvocationEvent) {<br>            metrics.record(event.invocation.cost(), event.invocation.usage)<br>            val cumulativeCost = event.agentProcess.cost()<br>            if (cumulativeCost &gt; alertThreshold) notify()<br>        }<br>    }<br>}<br>// Budget enforcement - set on ProcessOptions at run time:<br>agentPlatform.runAgentWithInput(<br>    goal = goal,<br>    input = input,<br>    processOptions = ProcessOptions.DEFAULT.withBudget(<br>        Budget(cost = 0.10, actions = 20)<br>    )<br>)</pre><h3>15. Token Budget Estimates</h3><pre>// Estimate token count before the call; enforce as a guardrail or per-round inspector:<br>val estimator = TokenCountEstimator.heuristic()<br>val estimated: Int = estimator.estimate(prompt)<br>if (estimated &gt; tokenBudget)<br>    throw TerminateActionException(&quot;Prompt exceeds token budget ($estimated tokens)&quot;)<br>// Declarative guardrail:<br>context.promptRunner()<br>    .withGuardRails(TokenBudgetGuardRail(estimator, maxTokens = 8000))<br>    .createObject(largeInput, Result::class.java)<br>// Per-round estimate inside ToolLoopInspector:<br>class BudgetInspector : ToolLoopInspector {<br>    override fun beforeLlmCall(ctx: BeforeLlmCallContext) {<br>        ctx.tokenEstimate?.let {<br>            if (it &gt; maxTokens) throw TerminateActionException(&quot;Token budget exceeded: $it&quot;)<br>        }<br>    }<br>}</pre><h3>16. LlmMessageSender / LlmMessageStreamer</h3><pre>// Non-streaming: Embabel owns the tool loop; called once per iteration<br>@Component<br>class MyLlmSender : LlmMessageSender {<br>    override fun call(messages: List&lt;Message&gt;, tools: List&lt;Tool&gt;): LlmMessageResponse {<br>        val raw = myClient.complete(messages, tools)<br>        return LlmMessageResponse(<br>            message = raw.toMessage(),<br>            textContent = raw.text,<br>            usage = Usage(raw.inputTokens, raw.outputTokens),<br>        )<br>    }<br>}<br>// Streaming: framework manages tool execution; only observation possible<br>@Component<br>class MyLlmStreamer : LlmMessageStreamer {<br>    override fun stream(<br>        messages: List&lt;Message&gt;,<br>        tools: List&lt;Tool&gt;,<br>        toolCallInspectors: List&lt;ToolCallInspector&gt;,<br>    ): Flux&lt;String&gt; = myClient.stream(messages, tools)<br>}</pre><h3>17. Native Thinking</h3><pre>// Single API - framework translates to provider-native payloads:<br>// Anthropic -&gt; ThinkingConfig(ThinkingType.ENABLED, budget)<br>// Google GenAI -&gt; includeThoughts + thinkingBudget<br>val runner = context.promptRunner()<br>    .withLlm(LlmOptions().withThinking(Thinking.withTokenBudget(8000)))<br>if (runner.supportsThinking()) {<br>    val response: ThinkingResponse&lt;Analysis&gt; = runner<br>        .thinking()<br>        .createObject(query, Analysis::class.java)<br>    response.thinkingBlocks.forEach { block -&gt;<br>        log.info(&quot;[${block.tagType}/${block.tagValue}] ${block.content.take(200)}&quot;)<br>    }<br>}</pre><h3>18. BYOK</h3><pre>// Sign-up flow - provider unknown, race all candidates concurrently:<br>val service: LlmService&lt;*&gt; = detectProvider(<br>    AnthropicModelFactory(apiKey = userKey),<br>    OpenAiCompatibleModelFactory.openAi(userKey),<br>    OpenAiCompatibleModelFactory.deepSeek(userKey),<br>    OpenAiCompatibleModelFactory.mistral(userKey),<br>    OpenAiCompatibleModelFactory.gemini(userKey),<br>)<br>val detectedProvider = service.provider<br>// Settings flow - provider already known:<br>val service = detectProvider(AnthropicModelFactory(apiKey = userKey))<br>// Custom OpenAI-compatible provider:<br>val service = detectProvider(<br>    OpenAiCompatibleModelFactory.byok(<br>        baseUrl = &quot;https://api.myprovider.com&quot;,<br>        apiKey = userKey,<br>        validationModel = &quot;my-model-small&quot;,<br>        validationProvider = &quot;MyProvider&quot;,<br>    )<br>)</pre><h3>19. Skills Engine</h3><pre>// Load from local directory tree (each subdirectory is one skill):<br>val skills = Skills(name = &quot;my-skills&quot;, description = &quot;Available skills&quot;)<br>    .withLocalSkills(&quot;/path/to/skills&quot;)<br>// Load from GitHub:<br>val skills = Skills(name = &quot;my-skills&quot;, description = &quot;Available skills&quot;)<br>    .withGitHubSkills(owner = &quot;acme&quot;, repo = &quot;agent-skills&quot;, skillsPath = &quot;skills&quot;)<br>// Expose as individual per-skill tools (recommended - one tool per skill, lazy body):<br>context.promptRunner()<br>    .withReferences(skills.asIndividualReferences())<br>    .createObject(request, Response::class.java)<br>// Filter out skills that bundle scripts (e.g. in a sandboxed environment):<br>val readOnly = skills.withFilter(SkillFilter.WITHOUT_SCRIPTS)<br>// Attach a script execution engine so scripts become callable tools:<br>val skills = Skills(name = &quot;my-skills&quot;, description = &quot;Available skills&quot;)<br>    .withLocalSkills(&quot;/path/to/skills&quot;)<br>    .withScriptExecutionEngine(NodeJsExecutionEngine())</pre><h3>20. Secure MCP</h3><pre>// Method-level - fine-grained authority per action:<br>@SecureAgentTool(&quot;hasAuthority(&#39;payments:write&#39;)&quot;)<br>@AchievesGoal(description = &quot;Process a payment&quot;)<br>@Action<br>fun processPayment(request: PaymentRequest, context: OperationContext): PaymentResult { ... }<br>// Class-level - all actions in the agent share the same guard:<br>@SecureAgentTool(&quot;hasRole(&#39;ADMIN&#39;)&quot;)<br>@Agent(description = &quot;Admin operations&quot;)<br>class AdminAgent {<br>    @Action<br>    fun purgeCache(): CacheResult { ... }<br>}<br>// Policy bean expression with method parameter binding:<br>@SecureAgentTool(&quot;@paymentPolicy.canProcess(authentication, #request)&quot;)<br>@Action<br>fun submitTransfer(request: TransferRequest, context: OperationContext): TransferResult { ... }</pre><pre># Activate JWT validation by enabling the secured Spring profile:<br>spring:<br>  profiles:<br>    active: secured<br>  security:<br>    oauth2:<br>      resourceserver:<br>        jwt:<br>          issuer-uri: https://your-auth-server/.well-known/openid-configuration</pre><h3>21. Chat Store Support</h3><pre>// Inject the provider; select storage type per chatbot:<br>@Component<br>class MyChatbot(private val provider: ConversationFactoryProvider) {<br>    // In-memory: fast, ephemeral (no persistence across restarts):<br>    fun ephemeralSession(id: String): Conversation =<br>        provider.getFactory(ConversationStoreType.IN_MEMORY).create(id)<br>    // Stored: persists across restarts, reloads by ID:<br>    fun persistentSession(id: String, user: User, agent: User): Conversation {<br>        val factory = provider.getFactory(ConversationStoreType.STORED)<br>        return factory.load(id) ?: factory.createForParticipants(id, user, agent)<br>    }<br>}</pre><pre>&lt;!-- Add embabel-chat-store to pom.xml for persistent storage: --&gt;<br>&lt;dependency&gt;<br>    &lt;groupId&gt;com.embabel&lt;/groupId&gt;<br>    &lt;artifactId&gt;embabel-chat-store&lt;/artifactId&gt;<br>&lt;/dependency&gt;</pre><h3>22. ONNX Embedding Models</h3><pre># Model downloaded from HuggingFace and cached at ~/.embabel/models on first use:<br>embabel:<br>  agent:<br>    platform:<br>      models:<br>        onnx:<br>          embeddings:<br>            enabled: true<br>            model-name: all-MiniLM-L6-v2<br>            dimensions: 384<br>            cache-dir: ${user.home}/.embabel/models<br>            model-uri: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/onnx/model.onnx<br>            tokenizer-uri: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/tokenizer.json</pre><pre># For air-gapped or pre-downloaded models:<br>embabel:<br>  agent:<br>    platform:<br>      models:<br>        onnx:<br>          embeddings:<br>            model-uri: file:///opt/models/all-MiniLM-L6-v2/model.onnx<br>            tokenizer-uri: file:///opt/models/all-MiniLM-L6-v2/tokenizer.json</pre><p><em>The Embabel Agent Framework is open source. Check out the </em><a href="https://github.com/embabel/embabel-agent"><em>GitHub repository</em></a><em> for the implementation.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=332f375d0fba" width="1" height="1" alt=""><hr><p><a href="https://medium.com/embabel/embabel-agentic-ai-framework-release-1-0-0-new-features-332f375d0fba">Embabel Agentic AI Framework Release 1.0.0: New Features</a> was originally published in <a href="https://medium.com/embabel">Embabel</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Tool Loop: Where Agentic AI Actually Happens]]></title>
            <link>https://medium.com/embabel/the-tool-loop-where-agentic-ai-actually-happens-e4127908209e?source=rss-81e4c14289d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/e4127908209e</guid>
            <category><![CDATA[tool-loop]]></category>
            <category><![CDATA[transformer-inspector]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[embabel]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Igor Dayen]]></dc:creator>
            <pubDate>Thu, 23 Apr 2026 04:50:53 GMT</pubDate>
            <atom:updated>2026-04-23T04:50:53.322Z</atom:updated>
            <content:encoded><![CDATA[<h3>Acknowledgements</h3><p><em>This blog is co-authored with </em><a href="https://medium.com/@springrod"><em>Rod Johnson</em></a><em>, founder of Spring Framework and Embabel Agentic AI Framework.</em></p><p><em>Authors would like to express deep gratitude to Embabel Core Team for thorough content review.</em></p><p>Tool calling is critical part of agentic AI applications. We identify useful capabilities — for example, Java methods or MCP server operations — and expose them as tools to the LLM.</p><p>In fact, as LLMs are stateless, there is a tool calling dance underneath performed by whatever framework we use. The LLM says “I want to call this tool with these arguments”. The framework executes the tool, gets the result, adds it to the conversation history, and calls the LLM again. This loop continues until the LLM produces a final answer instead of more tool calls.</p><p>This innocent-looking loop is where the magic happens. It’s also where things can get messy in production.</p><p>What happens when:</p><ul><li>Your tool returns 50KB of JSON and you’re burning through your context window?</li><li>You need to log every tool call for debugging a production issue at 2 AM?</li><li>You want to run three independent API calls in parallel instead of waiting 9 seconds sequentially?</li></ul><blockquote><em>The tool loop is at the heart of agentic AI. Yet most frameworks treat it as an implementation detail, giving you little control over what happens inside.</em></blockquote><h3>What’s Actually Inside the Loop?</h3><p>Let’s break Tool Loop Iteration down:</p><ol><li>Prepare messages for LLM</li></ol><p>2. Call LLM</p><p>3. Process response</p><p>4. For each tool call</p><p>4.1. Execute tool → Get results</p><p>4.2 Add results to history</p><p>5. Tool calls remaining?</p><p>5.1. Yes, Loop back to 1</p><p>5.2. No, Final answer → Exit</p><p>Each step is an opportunity to observe or transform. However, most frameworks provide only limited support — often little more than basic logging hooks.</p><h3>Tool Loop JVM Landscape Today</h3><p>Consider the two most popular JVM frameworks:</p><p><strong>Spring AI</strong> recently added a ToolCallAdvisor with recursive advisors. But advisors are a single type that can both observe <em>and</em> modify - you never know what a given advisor does without reading its code. And although, being familiar from Spring AOP, the &quot;advisor&quot; mechanism is a questionable conceptual fit for this problem. Please refer to Jettro Coenradie&#39;s blog <a href="https://medium.com/@jettro.coenradie/reliable-ai-with-spring-ai-guardrails-observability-and-evaluations-f1fa08e67a4b">Reliable AI with Spring AI: Guardrails, Observability, and Evaluations</a>, which illustrates both the usage and limitations of the Spring AI advisory API.</p><p><strong>LangChain4j</strong> has tool callbacks in streaming mode such as onPartialToolCall, onCompleteToolCall and onToolExecuted(), and even parallel execution with executeToolsConcurrently(). However, framework does not provide with out-of-box <strong>transformers</strong>, which can unlock valuable possibilities for conversational history transformation. Below is a sample LangChain4j-based Transformer:</p><pre>class TransformingChatModel implements ChatModel {<br><br>    private final ChatModel delegate;<br>    private final ResponseTransformer transformer;<br><br>    TransformingChatModel(ChatModel delegate,<br>                          ResponseTransformer transformer) {<br>        this.delegate = delegate;<br>        this.transformer = transformer;<br>    }<br><br>    @Override<br>    public Response&lt;AiMessage&gt; generate(ChatRequest request) {<br><br>        Response&lt;AiMessage&gt; response = delegate.generate(request);<br><br>        String original = response.content().text();<br><br>        String transformed = transformer.transform(original);<br><br>        return Response.from(AiMessage.from(transformed));<br>    }<br>}</pre><h3>A Different Approach: Embabel’s Tool Loop</h3><p>Embabel manages its own tool loop. When you configure a prompt runner (see Arjen Poutsma’s blog <a href="https://medium.com/embabel/recent-improvements-to-embabels-promptrunner-b1bb07190405">Recent Improvements to Embabel’s PromptRunner</a>) to run with tools, you get a fully managed loop that handles LLM calls, tool execution, and history management. This tool loop is highly customizable, offering clear extension points with a separation between observation (inspectors) and transformation (transformers).</p><p><strong>Inspectors</strong> observe the loop without modifying it. Implement any or all of beforeLlmCall, afterLlmCall, afterToolResult, afterIteration. Inspectors are perfect for logging, metrics, debugging, and are read-only by design.</p><p><strong>Transformers</strong> <em>modify</em> data flowing through the loop. Use them to truncate large tool results, apply sliding window to conversation history, redact sensitive content. They change what the LLM sees.</p><p>One observes, one transforms. There’s no ambiguity about what a callback does.</p><pre>| Type        | Can observe | Can modify | Safe to add many |<br>|-------------|:-----------:|:----------:|:----------------:|<br>| Inspector   |      ✓      |     ✗      |        ✓         |<br>| Transformer |      ✓      |     ✓      |  Order matters   |</pre><p>Here are the two interfaces:</p><pre>public interface ToolLoopInspector {<br>    default void beforeLlmCall(BeforeLlmCallContext context) {<br>    }<br><br>    default void afterLlmCall(AfterLlmCallContext context) {<br>    }<br><br>    default void afterToolResult(AfterToolResultContext context) {<br>    }<br><br>    default void afterIteration(AfterIterationContext context) {<br>    }<br>}<br><br>public interface ToolLoopTransformer {<br>    List&lt;Message&gt; transformBeforeLlmCall(BeforeLlmCallContext context);<br><br>    String transformAfterToolResult(AfterToolResultContext context);<br>    // ...<br>}</pre><h3>Putting It Together</h3><p>Here’s the complete picture — tools, inspectors, and transformers working together, via Embabel’s elegant fluent API:</p><pre>// Execute with tools and callbacks<br>var result = ai.withDefaultLlm()<br>                .withTools(tools)<br>                .withToolLoopInspectors(callbackTracker, loggingInspector)<br>                .withToolLoopTransformers(truncatingTransformer, slidingWindowTransformer)<br>                .creating(RestaurantRecommendation.class)<br>                .fromPrompt(&quot;&quot;&quot;<br>                        I&#39;m looking for an Italian restaurant near the Upper East Side in NYC.<br>                        <br>                        You have access to these tools to fetch restaurant menus:<br>                        %s<br>                        <br>                        Please fetch the menus and recommend the best restaurant for a romantic dinner.<br>                        &quot;&quot;&quot;.formatted(String.join(&quot;, &quot;, toolNames)));</pre><p>Tools are executed in a managed loop. Inspectors observe every call. Transformers shape the data, while the developer can focus on business logic. Framework provides with simple out-of-box callbacks:</p><ul><li>ToolLoopLoggingInspector — logs calls details before and after LLM invocations, after Tool Execution, and after Tool Loop Iteration</li><li>ToolResultTruncatingTransformer — truncates tool call results</li><li>SlidingWindowTransformer — maintains a sliding window of messages to manage context size, while preserving conversation context system messages</li></ul><h4>Practical Example: Truncating Giant Results</h4><p>Suppose that your web scraping tool returns 30KB of HTML. Your LLM doesn’t need all of it. Here’s a transformer to help with that:</p><pre>var truncator = new ToolResultTruncatingTransformer(5000);</pre><p>Every tool result over 5KB is truncated, without scattered if statements in your tool code. (Of course you could apply different strategies for truncation.)</p><h3>Practical Example: LLM and Tools Invocations Counter</h3><p>Inspector below maintains counters of LLM and Tools invocations, and also maintains the list of invoked tool names.</p><pre> // Custom inspector that tracks callback invocations<br>    <br>    static class CallbackTracker implements ToolLoopInspector {<br>        final AtomicInteger beforeLlmCallCount = new AtomicInteger();<br>        final AtomicInteger afterLlmCallCount = new AtomicInteger();<br>        final AtomicInteger afterToolResultCount = new AtomicInteger();<br>        final List&lt;String&gt; toolsInvoked = new ArrayList&lt;&gt;();<br><br>        protected final Logger logger = LoggerFactory.getLogger(getClass());<br><br>        @Override<br>        public void beforeLlmCall(@NotNull BeforeLlmCallContext context) {<br>            beforeLlmCallCount.incrementAndGet();<br>            var threadName = Thread.currentThread().getName();<br>            logger.info(&quot;Before LLM Call Thread {}&quot;, threadName);<br>        }<br><br>        @Override<br>        public void afterLlmCall(@NotNull AfterLlmCallContext context) {<br>            afterLlmCallCount.incrementAndGet();<br>        }<br><br>        @Override<br>        public void afterToolResult(@NotNull AfterToolResultContext context) {<br>            afterToolResultCount.incrementAndGet();<br>            synchronized (toolsInvoked) {<br>                toolsInvoked.add(context.getToolCall().getName());<br>            }<br>        }<br>    }</pre><p>Tracking calls Inspector got designed for parallel excution environment, which can be configured via:</p><pre>embabel.agent.platform.toolloop.type=parallel</pre><h3>Benefits of the Embabel Tool Loop</h3><p>Embabel’s control of the tool loop not only lets you inspect and modify the tool calling history: It allows features such as unfolding tools, introducing progressive disclosure to tools and allowing a larger number of tools to be exposed than in other frameworks. See <a href="https://medium.com/@springrod/the-power-of-russian-dolls-rethinking-tool-discovery-1e79eba1543a">Rod Johnson’s blog on Unfolding Tools</a>.</p><p>The tool loop is agnostic to the LLM framework you use. You can use your choice of lower level frameworks, such as Spring AI or LangChain4j. This would be a subject of a separate blog, though.</p><h3>When Would You Use This?</h3><p><strong>Production observability</strong>: Log every LLM call and tool execution. Track token usage. Debug issues without adding println statements everywhere.</p><p><strong>Context management</strong>: Compress old messages. Truncate verbose tool results. Stay within token limits without manual intervention.</p><p><strong>Performance</strong>: Run independent tools in parallel. Set timeouts so one slow API doesn’t block everything.</p><p><strong>Migration</strong>: Moving from Spring AI to LangChain4j? Change one adapter class, keep everything else.</p><h3>Try It</h3><p>Working examples can be found in <a href="https://github.com/embabel/embabel-agent-experimental/tree/main/embabel-experimental-integration-tests">embabel-agent-experimental repository</a>:</p><ul><li>Basic Default tool loop with logging inspector, and history transformation</li><li>Parallel tool loop with the same (mentioned above) callbacks</li><li>LangChain4j and OpenAI integrations showing framework independence</li></ul><p>For the full tool programming model, see <a href="https://medium.com/@springrod/domain-tools-direct-access-zero-ceremony-9a3e8d4cf550">Rod Johnson’s blog Domain Tools: Direct Access, Zero Ceremony</a>.</p><h3>The Bigger Picture</h3><p>The tool loop is where your agent actually <em>does things</em>. It deserves first-class treatment:</p><ul><li>Clear extension points, not mysterious internals</li><li>Separation between observation and transformation</li><li>Parallel execution that handles real-world complexity</li><li>Independence from any specific LLM</li></ul><p>We believe that Embabel provides the clearest, most powerful abstractions to empower you to control tool calling without writing error-prone low level code.</p><p>Please join Embabel community on <a href="https://discord.com/invite/t6bjkyj93q">Discord</a> for updates, contributions are welcome.</p><p><em>The Embabel Agent Framework is open source. Check out the </em><a href="https://github.com/embabel/embabel-agent"><em>GitHub repository</em></a><em> for the implementation.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e4127908209e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/embabel/the-tool-loop-where-agentic-ai-actually-happens-e4127908209e">The Tool Loop: Where Agentic AI Actually Happens</a> was originally published in <a href="https://medium.com/embabel">Embabel</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>