Engineering an Enterprise Multi-Agent AI System: Enforcing Deterministic Truth via Kotlin, Ktor, and Gemini 2.0
Introduction: The Fallacy of Stochastic Fact-Generation
The single greatest flaw in modern Generative AI engineering is a structural reliance on Large Language Models (LLMs) to generate raw, underlying facts. If you ask a foundational model to compute complex mathematical structures—such as astronomical longitudes, planetary time cycles, or precise scoring matrices—it will confidently hallucinate.
To build Astro Jyotish, a high-concurrency, production-grade Vedic Astrology platform, I designed a Two-Tier Knowledge Framework built entirely on a ReAct (Reason + Act) multi-agent topology. The core architectural mission was uncompromising: Enforce Deterministic Truth vs. Stochastic Reasoning.
In this framework, the LLMs possess zero mathematical or regulatory sovereignty. They are strictly barred from calculating planetary charts or inventing rule interpretations; they act purely as the orchestrator and text synthesis engine ("the brain"), while native JVM engines and verified databases provide the absolute ground truth.
1. The Multi-Module Architecture Stack
To handle unpredictable concurrency demands without building a rigid if/else state tree, the platform uses a lightweight, asynchronous Kotlin-native stack:
ktor-app (API Gateway & SSE Server): A high-performance web gateway running on the Netty engine. It processes inbound requests, validates cryptographically signed identity tokens via custom Firebase Auth middleware, and streams real-time responses under a non-buffering text/event-stream contract using Server-Sent Events (SSE).
firebase-service (Agent Orchestration & Tool Layer): Powered by the official GA Google GenAI Java SDK, this module utilizes Gemini 2.0 Pro for top-tier routing/orchestration and Gemini 2.0 Flash for fast safety sweeps. It houses the custom coroutine execution loop and a custom framework-agnostic tool registry.
chart-service (Isolated Compute Module): A completely decoupled, stateless engine housing native swisseph-java JNI bindings for the C-based Swiss Ephemeris library.
Rather than using heavy WebSockets or blocking HTTP POST blocks that hide behind a loading spinner, the Ktor gateway streams structural metadata payloads (IREvidencePayload) across the wire exactly once to anchor confidence scores and native calculations on screen before the orchestrator begins typing real-time markdown text fragments down the pipe.
2. Deep-Tech Challenges from the Trenches
The Linux glibc vs. musl Container Trap
When pushing initial container images to Google Cloud Build, the automated quality evaluation gate violently short-circuited mid-flight, throwing low-level SIGABRT (Signal 6) crashes during gRPC and transport initialization.
The root cause was an environment mismatch. To minimize container size, the early build pipeline used an Alpine Linux-based JDK image. Alpine strips out the standard Linux C library (glibc) in favor of a minimalist translator called musl. However, the official Google GenAI SDK and Firebase Admin SDK initialize underlying gRPC Netty transport channels that rely on pre-compiled native .so binaries. These binaries expect standard glibc, and they completely broke down under Alpine’s minimalist environment, forcing the JVM to safely abort the program.
The Resolution: I shifted the continuous integration environment layers in cloudbuild.yaml to use standard Debian-based base images (eclipse-temurin). This guaranteed native glibc compatibility out of the box, allowing the gRPC compilation runtimes, Firebase Admin bindings, and JNI math processes to link cleanly at runtime.
Taming Tool States, Infinite Loops, and p95 Latency
In multi-agent systems, interactive pipelines introduce compounding latency bottlenecks as a supervisor coordinates data fetching, rule mapping, and citation lookups sequentially. Furthermore, if user queries are ambiguous, an agent risks entering an infinite recursive loop of tool calls, causing rapid token quota exhaustion.
The Resolution: I solved these production constraints by applying a two-fold optimization:
Coroutine Parallelization: I drastically lowered our p95 latency by parallelizing independent tool calling paths using Kotlin Coroutines and structured concurrency. Independent tool execution paths are fired simultaneously, allowing agents to ingest historical evidence instantly without stalling the active user stream.
Budget Deflection Gates: The custom AgentRunner routing loop tracks total agent-to-agent delegation turns and enforces a hard execution threshold of 7 maximum iterations. If an agent interaction fails to converge within this limit, a custom BudgetExhaustedException trips instantly, halting model calls and pushing a safe fallback payload down the wire to protect our token consumption budget.
Bypassing Firestore's 1MB Document Bottleneck
Early versions managed conversational context entirely in local JVM memory, resulting in total session loss during network fluctuations. However, accumulating raw message lists sequentially inside a single parent session document puts you on a direct crash course with Google Firestore's hard maximum limit of 1MB per document.
The Resolution: I refactored the persistence model to a User-Centric Identity Model anchored to cryptographically verified Firebase JWTs. I stripped message arrays entirely from the parent session envelope and stored individual chat events sequentially inside a nested sub-collection path: /users/{userId}/sessions/{sessionId}/messages/{messageId}. This completely bypasses the 1MB document storage bottleneck, allowing users to converse for years on a single thread without risking a serialization crash.
3. Continuous Quality Gating & Production Guardrails
A true Director of AI knows that a prompt modification in an orchestration layer can cause a cascading wave of regressions across specialist agents. To achieve continuous deployment immunity, no container image is pushed to Google Artifact Registry or deployed to Google Cloud Run unless it clears a hard automated quality gate.
The deployment pipeline automatically runs an offline JUnit Vertex AI Rapid Evaluation runner across a curated 20-case Golden Dataset of authoritative historical control profiles and adversarial safety traps. The pipeline evaluates three key metrics via an LLM-as-a-Judge mechanism to determine if a build is safe to deploy:
Groundedness (\ge 0.85): Asserts that the interpretation agents synthesize text based exclusively on the retrieved deterministic data payload and haven't fabricated rules.
Tool Choice Accuracy (\ge 0.85): Verifies that the supervisor agent consistently dispatches the proper tool arrays without dropping into runaway loops.
Safety Compliance (\ge 0.95): Confirms that defensive agents intercept and block forbidden or sensitive topics (e.g., questions regarding death or terminal illness).
If prompt updates or code refactors cause any metric to drop below these strict baselines, Google Cloud Build instantly terminates the rollout.
4. Real-Time Mobile Stream Rendering
The Android client application, built entirely in Jetpack Compose, operates as a real-time event-driven state machine utilizing an unbuffered OkHttp EventSource connection.
The client UI captures the force-serialized StreamEvent.Evidence event first. It instantly triggers an animated slide-in card displaying a linear progress gauge for the confidence score, an expandable grid displaying the precise planetary coordinates derived by the JNI engine, and classical source citations. Once the structural evidence card is pinned into place, the UI smoothly appends incoming textual tokens (StreamEvent.TextChunk) into a lazy list layout using a blinking cursor animation and intelligent autoscrolling metrics.
The moment the connection closes, an action panel linked to a /api/v1/feedback route displays. By passing back a hidden correlationId tracked across async coroutines via kotlinx-coroutines-slf4j, the user can flag a transaction, allowing developers to isolate and debug that specific multi-agent session instantly inside the centralized production cloud logging dashboard.
Conclusion
By treating calculations and business logic as isolated, deterministic software tools rather than text prompts, the platform successfully delivers tailored insights under an aggressive budget limit of $0.05 per user reading, with zero risk of hallucination.
🌐 Codebase State: 100% complete, fully traces multi-turn agent hops, and successfully deployed to Google Cloud Run.
Taming Multi-Agent Latency and State Loops: How I Leveraged Antigravity, Kotlin Coroutines, and Debian Containers to Build a…