<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Adarsh Singh Dikhit on Medium]]></title>
        <description><![CDATA[Stories by Adarsh Singh Dikhit on Medium]]></description>
        <link>https://medium.com/@to.adarshsingh?source=rss-73f6989daa12------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*FCnfVXsPtrU0ktRy</url>
            <title>Stories by Adarsh Singh Dikhit on Medium</title>
            <link>https://medium.com/@to.adarshsingh?source=rss-73f6989daa12------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 02:46:56 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@to.adarshsingh/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[MCP Drift: The Silent Threat to AI Agent Reliability]]></title>
            <link>https://medium.com/@to.adarshsingh/mcp-drift-the-silent-threat-to-ai-agent-reliability-774b34577994?source=rss-73f6989daa12------2</link>
            <guid isPermaLink="false">https://medium.com/p/774b34577994</guid>
            <category><![CDATA[drift-detection]]></category>
            <dc:creator><![CDATA[Adarsh Singh Dikhit]]></dc:creator>
            <pubDate>Tue, 16 Jun 2026 09:10:55 GMT</pubDate>
            <atom:updated>2026-06-16T09:10:55.807Z</atom:updated>
            <content:encoded><![CDATA[<h3>Abstract</h3><p>As Model Context Protocol (MCP) becomes the de facto standard for connecting AI agents to external tools and data sources, a new class of failure mode has emerged: <strong>MCP Drift</strong>. This article defines MCP Drift, explores its architectural root causes, examines how it manifests in production systems, and proposes detection and mitigation strategies. Engineers building agentic pipelines on MCP must understand drift to build resilient, observable systems.</p><h3>1. Introduction</h3><p>The Model Context Protocol (MCP), introduced by Anthropic in late 2024, provides a standardized interface between AI language models (clients/hosts) and external capability providers (servers). It defines how tools, resources, and prompts are declared, negotiated, and executed across an open JSON-RPC 2.0 transport layer.</p><p>Within months of widespread adoption, production teams began encountering a subtle but dangerous failure pattern: <strong>the contract between MCP client and server degrades silently over time</strong>. Tool schemas evolve, resource endpoints shift, server behavior changes — yet the AI agent continues operating on a stale understanding of what the server provides. We call this phenomenon <strong>MCP Drift</strong>.</p><p>Unlike hard failures that surface immediately, MCP Drift causes <strong>gradual behavioral degradation</strong>: agents confidently invoke tools with outdated parameters, receive unexpected outputs, and produce incorrect results — often without any explicit error.</p><h3>2. Background: How MCP Works</h3><p>Before defining drift, it is essential to understand the MCP lifecycle.</p><h3>2.1 Core Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1heEmjeUzhU8HllSEoDdeA.png" /></figure><h3>2.2 MCP Initialization Handshake</h3><p>At startup, the client and server perform a capability negotiation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cScFLifMVZ4CpGFSdHDFdA.png" /></figure><h3>2.3 Tool Schema Anatomy</h3><p>Each tool schema declares its full contract with the client:</p><pre>{<br>  &quot;name&quot;: &quot;search_database&quot;,<br>  &quot;description&quot;: &quot;Search records in the customer database&quot;,<br>  &quot;inputSchema&quot;: {<br>    &quot;type&quot;: &quot;object&quot;,<br>    &quot;properties&quot;: {<br>      &quot;query&quot;: { &quot;type&quot;: &quot;string&quot; },<br>      &quot;limit&quot;: { &quot;type&quot;: &quot;integer&quot;, &quot;default&quot;: 10 },<br>      &quot;filters&quot;: {<br>        &quot;type&quot;: &quot;object&quot;,<br>        &quot;properties&quot;: {<br>          &quot;region&quot;: { &quot;type&quot;: &quot;string&quot; },<br>          &quot;status&quot;: { &quot;enum&quot;: [&quot;active&quot;, &quot;inactive&quot;] }<br>        }<br>      }<br>    },<br>    &quot;required&quot;: [&quot;query&quot;]<br>  }<br>}</pre><p>Any change to this schema renamed fields, new required parameters, changed enum values that is not synchronized with the client creates drift.</p><h3>3. Defining MCP Drift</h3><p><strong>MCP Drift</strong> is the divergence between the semantic and structural contract a MCP client holds about a server’s capabilities and the server’s actual current behavior.</p><p>Drift can be decomposed into four types:</p><h3>3.1 Schema Drift</h3><p>The input or output schema of a tool changes without the client being aware.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D9S8tcfbWqKo7YraGPcGzA.png" /></figure><p><strong>Impact:</strong> Tool calls fail silently or return unexpected errors that the LLM misinterprets.</p><h3>3.2 Semantic Drift</h3><p>The tool’s name or description changes in meaning without a schema change.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uLtWBUQWwh8eDdXz6zcG6w.png" /></figure><p>The LLM invokes the same tool, passes valid parameters, but the side effects are completely different.</p><h3>3.3 Behavioral Drift</h3><p>The server’s internal logic changes: new rate limits, different pagination behavior, changed authentication requirements, altered output formats.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oiflFsbjFYWOp03uTKlA8Q.png" /></figure><h3>3.4 Capability Drift</h3><p>Tools appear or disappear from the server’s capability manifest without the client refreshing its list.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tzerXjyUfiUtMx_ORoNebA.png" /></figure><h3>4. Architecture of MCP Drift</h3><h3>4.1 Root Causes Taxonomy</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F6eeTrbxzSB0X9nB9mSOYQ.png" /></figure><h3>4.2 Drift Propagation Timeline</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GdZNUQrgsCJt7cxyfMrkMg.png" /></figure><h3>4.3 Multi-Server Drift (Federated MCP)</h3><p>Enterprise deployments commonly run multiple MCP servers simultaneously. A single drifted server can corrupt agent reasoning across the entire pipeline:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jPCUbHjUlPqh39hunzDDgg.png" /></figure><h3>5. Detection Strategies</h3><h3>5.1 Schema Hash Fingerprinting</h3><p>Assign a deterministic hash to each tool schema and compare on every session:</p><pre>import hashlib, json<br><br>def schema_fingerprint(tool_schema: dict) -&gt; str:<br>    &quot;&quot;&quot;Canonical JSON hash for a tool schema.&quot;&quot;&quot;<br>    canonical = json.dumps(tool_schema, sort_keys=True)<br>    return hashlib.sha256(canonical.encode()).hexdigest()[:16]<br><br>def detect_drift(cached_tools: list, live_tools: list) -&gt; list:<br>    cached_index = {t[&quot;name&quot;]: schema_fingerprint(t) for t in cached_tools}<br>    live_index   = {t[&quot;name&quot;]: schema_fingerprint(t) for t in live_tools}<br><br>    drift_report = []<br>    for name, live_hash in live_index.items():<br>        cached_hash = cached_index.get(name)<br>        if cached_hash is None:<br>            drift_report.append({&quot;tool&quot;: name, &quot;type&quot;: &quot;CAPABILITY_ADDED&quot;})<br>        elif cached_hash != live_hash:<br>            drift_report.append({&quot;tool&quot;: name, &quot;type&quot;: &quot;SCHEMA_DRIFT&quot;,<br>                                  &quot;cached&quot;: cached_hash, &quot;live&quot;: live_hash})<br><br>    for name in cached_index:<br>        if name not in live_index:<br>            drift_report.append({&quot;tool&quot;: name, &quot;type&quot;: &quot;CAPABILITY_REMOVED&quot;})<br><br>    return drift_report</pre><h3>5.2 Runtime Validation Layer</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GSUzhx9kp4UAaVOvcr9SKQ.png" /></figure><h3>5.3 Semantic Drift Detection via Embeddings</h3><p>Schema drift in descriptions requires semantic comparison. Use Claude to score description divergence:</p><pre>from anthropic import Anthropic<br><br>client = Anthropic()<br><br>def semantic_drift_score(old_desc: str, new_desc: str) -&gt; float:<br>    &quot;&quot;&quot;<br>    Use Claude to score semantic similarity between tool descriptions.<br>    Returns 0.0 (identical meaning) to 1.0 (completely different meaning).<br>    &quot;&quot;&quot;<br>    response = client.messages.create(<br>        model=&quot;claude-sonnet-4-6&quot;,<br>        max_tokens=100,<br>        messages=[{<br>            &quot;role&quot;: &quot;user&quot;,<br>            &quot;content&quot;: f&quot;&quot;&quot;Score how much the MEANING of this tool description has changed.<br>Old: &quot;{old_desc}&quot;<br>New: &quot;{new_desc}&quot;<br>Respond with only a float from 0.0 (same meaning) to 1.0 (completely different).<br>&quot;&quot;&quot;<br>        }]<br>    )<br>    return float(response.content[0].text.strip())</pre><h3>5.4 Drift Detection Pipeline Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pXcNYL3fkiMyAdBT5xP4xQ.png" /></figure><h3>6. Mitigation Architecture</h3><h3>6.1 Schema Registry with Versioned Snapshots</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6ZuzladoPUgOoqMmZsX8cA.png" /></figure><h3>6.2 Compatibility Policy Enforcement</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V4GtKrczp39KZCkQSycWNA.png" /></figure><h3>6.3 MCP Session Refresh Protocol</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RTxU6haR-9fgzljnoBvhRg.png" /></figure><h3>6.4 Circuit Breaker Pattern for MCP Tools</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c0QBPoM-YGbYLh-0nyu--g.png" /></figure><pre>from enum import Enum<br>from datetime import datetime, timedelta<br><br>class CircuitState(Enum):<br>    CLOSED    = &quot;closed&quot;     # Normal operation<br>    OPEN      = &quot;open&quot;       # Tool disabled due to drift<br>    HALF_OPEN = &quot;half_open&quot;  # Probing for recovery<br><br>class MCPDriftCircuitBreaker:<br>    def __init__(self, tool_name: str, threshold: int = 3):<br>        self.tool_name   = tool_name<br>        self.state       = CircuitState.CLOSED<br>        self.failures    = 0<br>        self.threshold   = threshold<br>        self.opened_at   = None<br>        self.retry_after = timedelta(minutes=5)<br><br>    def call(self, tool_fn, *args, **kwargs):<br>        if self.state == CircuitState.OPEN:<br>            if datetime.now() - self.opened_at &gt; self.retry_after:<br>                self.state = CircuitState.HALF_OPEN<br>            else:<br>                raise DriftCircuitOpenError(<br>                    f&quot;Tool &#39;{self.tool_name}&#39; suspended due to schema drift&quot;<br>                )<br>        try:<br>            result = tool_fn(*args, **kwargs)<br>            self._on_success()<br>            return result<br>        except (SchemaMismatchError, DriftError):<br>            self._on_failure()<br>            raise<br><br>    def _on_success(self):<br>        self.failures = 0<br>        self.state    = CircuitState.CLOSED<br><br>    def _on_failure(self):<br>        self.failures += 1<br>        if self.failures &gt;= self.threshold:<br>            self.state     = CircuitState.OPEN<br>            self.opened_at = datetime.now()</pre><h3>6.5 Full Mitigation Stack Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*bigO8aCE64073kaFlqS-jQ.png" /></figure><h3>7. Observability and Monitoring</h3><h3>7.1 Drift Metrics to Track</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*1l70XSGvFy0ILyhQ25S14g.png" /></figure><h3>7.2 Drift Monitoring Flow</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CueZGdpHBskC9F-RNUZTBw.png" /></figure><h3>7.3 Schema-First Development: Drift-Prone vs Drift-Resistant</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/439/1*q8HqtpqE8jFPK71LjPjjCw.png" /></figure><h3>8. Governance and Prevention</h3><h3>8.1 MCP Server Development Contracts</h3><p>Establish engineering contracts to prevent drift at the source:</p><p><strong>Server-Side Requirements:</strong></p><ul><li>All schema changes must increment a semantic version field in the server manifest</li><li>Breaking changes require a 48-hour deprecation notice via a tools/change_notify event</li><li>Tool descriptions must pass a semantic similarity gate (&gt;0.8 similarity) before deployment</li><li>Backward-incompatible changes must maintain the old tool under a versioned alias (e.g., search_v1, search_v2)</li></ul><p><strong>Client-Side Requirements:</strong></p><ul><li>Clients must not cache tool schemas beyond a configurable TTL (default: 15 minutes)</li><li>Clients must subscribe to server change notifications when available</li><li>Clients must implement graceful degradation for missing tools</li></ul><h3>8.2 Governance Lifecycle</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/551/1*pDV7jBjNvW-9pXRK_J5nKw.png" /></figure><h3>9. Real-World Drift Scenarios</h3><h3>Scenario 1: Database Migration Drift</h3><p>A team migrates their CRM from PostgreSQL to a new SaaS platform. The MCP server is updated overnight. The query parameter that previously accepted raw SQL now expects a structured filter object. The AI agent continues sending SQL strings — the server silently returns empty results, and the agent reports &quot;no customers found&quot; for queries that should return thousands of records.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kRRt4j2Phyt2uxYdubtErw.png" /></figure><p><strong>Drift Type:</strong> Schema Drift + Semantic Drift</p><h3>Scenario 2: Rate Limit Policy Change</h3><p>A search MCP server adds aggressive rate limiting after unexpected traffic spikes. The error response format changes from a tool error to an HTTP 429-mapped response. The agent interprets throttled responses as empty results rather than retriable errors, causing silent data loss.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*LmKogCgyNwrcXz2gCcizEw.png" /></figure><p><strong>Drift Type:</strong> Behavioral Drift</p><h3>Scenario 3: Multi-Tenant Capability Split</h3><p>A multi-tenant MCP deployment splits tools by tenant tier. Enterprise users gain access to bulk_export and analytics_query; free-tier users lose export_csv. An agent serving both tiers caches the enterprise schema and then attempts bulk exports for free-tier users, receiving authorization errors it was not designed to handle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4rCSrQ8VNAWkGDCXHUCTyQ.png" /></figure><p><strong>Drift Type:</strong> Capability Drift</p><h3>10. Conclusion</h3><p>MCP Drift is not a bug — it is an architectural reality of distributed, evolving systems. As MCP server ecosystems grow in complexity and as organizations deploy hundreds of specialized servers, the gap between what agents know and what servers do will inevitably widen.</p><p>The teams that build reliable AI agent infrastructure in 2026 and beyond will be those that treat MCP Drift as a first-class operational concern: instrumenting for it, designing against it, and recovering from it gracefully.</p><p>The key principles are:</p><ol><li><strong>Assume drift will happen</strong> — design for graceful degradation, not just happy paths</li><li><strong>Make schema state explicit</strong> — schema registries are not optional in production</li><li><strong>Detect early, fail safely</strong> — schema fingerprinting and circuit breakers are table stakes</li><li><strong>Treat descriptions as code</strong> — semantic drift is as dangerous as structural drift</li><li><strong>Version everything</strong> — MCP servers should be versioned like APIs, because they are</li></ol><p>As agentic systems take on higher-stakes tasks — executing transactions, managing infrastructure, communicating on behalf of organizations — the cost of silent drift grows proportionally. Building drift-aware MCP infrastructure today is an investment in the trust and reliability that autonomous AI systems will require tomorrow.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=774b34577994" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Frappe ERP Multi-Tenancy with AWS Cognito SSO Implementation Guide]]></title>
            <link>https://medium.com/@to.adarshsingh/frappe-erp-multi-tenancy-with-aws-cognito-sso-implementation-guide-ed336a3a5e2f?source=rss-73f6989daa12------2</link>
            <guid isPermaLink="false">https://medium.com/p/ed336a3a5e2f</guid>
            <dc:creator><![CDATA[Adarsh Singh Dikhit]]></dc:creator>
            <pubDate>Mon, 11 Aug 2025 20:49:16 GMT</pubDate>
            <atom:updated>2025-08-11T20:49:16.531Z</atom:updated>
            <content:encoded><![CDATA[<h3>Table of Contents</h3><ol><li>Overview</li><li>Architecture Design</li><li>Multi-Tenancy Setup</li><li>AWS Cognito Integration</li><li>Implementation Steps</li><li>Code Examples</li><li>Testing &amp; Deployment</li></ol><h3>Overview</h3><p>This guide demonstrates how to implement multi-tenancy in Frappe ERP with AWS Cognito SSO authentication. Multi-tenancy allows multiple organizations to use the same application instance while keeping their data isolated.</p><h3>Key Benefits</h3><ul><li><strong>Data Isolation</strong>: Each tenant’s data is completely separated</li><li><strong>Cost Efficiency</strong>: Single application instance serves multiple clients</li><li><strong>Centralized SSO</strong>: Users authenticate once across all tenant applications</li><li><strong>Scalability</strong>: Easy to onboard new tenants</li></ul><h3>Architecture Design</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lXEdS4Ep3r074Aft7dXQlQ.png" /></figure><h3>Component Flow</h3><ol><li><strong>User Access</strong>: User accesses tenant-specific subdomain</li><li><strong>Tenant Detection</strong>: Middleware identifies tenant from domain/header</li><li><strong>Authentication</strong>: AWS Cognito handles SSO authentication</li><li><strong>Site Selection</strong>: Frappe loads appropriate tenant database</li><li><strong>Data Access</strong>: User interacts with tenant-specific data</li></ol><h3>Multi-Tenancy Setup</h3><h3>1. Site Structure</h3><pre># Create multiple sites for different tenants<br>bench new-site tenant1.company.com<br>bench new-site tenant2.company.com  <br>bench new-site tenant3.company.com</pre><h3>2. Database Schema per Tenant</h3><p>Each tenant gets its own database:</p><ul><li>tenant1_erp</li><li>tenant2_erp</li><li>tenant3_erp</li></ul><h3>AWS Cognito Integration</h3><h3>1. Cognito User Pool Setup</h3><pre>{<br>  &quot;UserPoolId&quot;: &quot;us-east-1_XXXXXXXX&quot;,<br>  &quot;ClientId&quot;: &quot;your-app-client-id&quot;,<br>  &quot;Region&quot;: &quot;us-east-1&quot;,<br>  &quot;Domain&quot;: &quot;your-domain.auth.us-east-1.amazoncognito.com&quot;<br>}</pre><h3>2. User Pool Configuration</h3><ul><li><strong>Authentication Flow</strong>: ALLOW_USER_SRP_AUTH</li><li><strong>Attributes</strong>: email, custom:tenant_id</li><li><strong>Groups</strong>: Create groups for different tenants</li></ul><h3>Implementation Steps</h3><h3>Step 1: Install Required Dependencies</h3><pre># Install AWS SDK<br>pip install boto3<br>pip install python-jose[cryptography]<br># Install in Frappe environment<br>bench get-app https://github.com/your-repo/cognito_integration.git<br>bench install-app cognito_integration</pre><h3>Step 2: Configure Tenant Detection Middleware</h3><p>Create apps/cognito_integration/cognito_integration/middleware.py:</p><pre>import frappe<br>import boto3<br>from urllib.parse import urlparse<br>class TenantMiddleware:<br>    def __init__(self):<br>        self.cognito_client = boto3.client(&#39;cognito-idp&#39;, region_name=&#39;us-east-1&#39;)<br>    <br>    def process_request(self):<br>        # Get tenant from subdomain<br>        host = frappe.request.headers.get(&#39;Host&#39;, &#39;&#39;)<br>        tenant_id = self.extract_tenant_from_host(host)<br>        <br>        if tenant_id:<br>            frappe.local.tenant_id = tenant_id<br>            # Switch to tenant-specific site<br>            self.switch_tenant_site(tenant_id)<br>    <br>    def extract_tenant_from_host(self, host):<br>        # Extract tenant from subdomain: tenant1.company.com<br>        subdomain = host.split(&#39;.&#39;)[0]<br>        return subdomain if subdomain != &#39;www&#39; else None<br>    <br>    def switch_tenant_site(self, tenant_id):<br>        site_name = f&quot;{tenant_id}.company.com&quot;<br>        if site_name in frappe.utils.get_sites():<br>            frappe.init(site=site_name)<br>            frappe.connect()</pre><h3>Step 3: Cognito Authentication Handler</h3><p>Create apps/cognito_integration/cognito_integration/auth.py:</p><pre>import jwt<br>import boto3<br>import requests<br>from jose import jwk, jwt as jose_jwt<br>from jose.utils import base64url_decode<br>class CognitoAuth:<br>    def __init__(self, user_pool_id, client_id, region):<br>        self.user_pool_id = user_pool_id<br>        self.client_id = client_id<br>        self.region = region<br>        self.cognito_client = boto3.client(&#39;cognito-idp&#39;, region_name=region)<br>    <br>    def verify_token(self, token):<br>        try:<br>            # Get Cognito public keys<br>            keys_url = f&#39;https://cognito-idp.{self.region}.amazonaws.com/{self.user_pool_id}/.well-known/jwks.json&#39;<br>            response = requests.get(keys_url)<br>            keys = response.json()[&#39;keys&#39;]<br>            <br>            # Decode token header to get kid<br>            headers = jose_jwt.get_unverified_header(token)<br>            kid = headers[&#39;kid&#39;]<br>            <br>            # Find the correct key<br>            key = None<br>            for k in keys:<br>                if k[&#39;kid&#39;] == kid:<br>                    key = k<br>                    break<br>            <br>            if not key:<br>                return None<br>            <br>            # Construct public key<br>            public_key = jwk.construct(key)<br>            <br>            # Decode and verify token<br>            payload = jose_jwt.decode(<br>                token,<br>                public_key,<br>                algorithms=[&#39;RS256&#39;],<br>                audience=self.client_id,<br>                issuer=f&#39;https://cognito-idp.{self.region}.amazonaws.com/{self.user_pool_id}&#39;<br>            )<br>            <br>            return payload<br>            <br>        except Exception as e:<br>            frappe.log_error(f&quot;Token verification failed: {str(e)}&quot;)<br>            return None<br>    <br>    def get_user_info(self, access_token):<br>        try:<br>            response = self.cognito_client.get_user(AccessToken=access_token)<br>            return response<br>        except Exception as e:<br>            frappe.log_error(f&quot;Failed to get user info: {str(e)}&quot;)<br>            return None</pre><h3>Step 4: Custom Login Handler</h3><p>Create apps/cognito_integration/cognito_integration/login.py:</p><pre>import frappe<br>from frappe import _<br>from .auth import CognitoAuth<br>@frappe.whitelist(allow_guest=True)<br>def cognito_login():<br>    try:<br>        # Get token from request<br>        token = frappe.request.headers.get(&#39;Authorization&#39;, &#39;&#39;).replace(&#39;Bearer &#39;, &#39;&#39;)<br>        <br>        if not token:<br>            frappe.throw(_(&quot;No token provided&quot;))<br>        <br>        # Initialize Cognito auth<br>        cognito_auth = CognitoAuth(<br>            user_pool_id=frappe.conf.get(&#39;cognito_user_pool_id&#39;),<br>            client_id=frappe.conf.get(&#39;cognito_client_id&#39;),<br>            region=frappe.conf.get(&#39;cognito_region&#39;, &#39;us-east-1&#39;)<br>        )<br>        <br>        # Verify token<br>        payload = cognito_auth.verify_token(token)<br>        if not payload:<br>            frappe.throw(_(&quot;Invalid token&quot;))<br>        <br>        # Get user info<br>        email = payload.get(&#39;email&#39;)<br>        tenant_id = payload.get(&#39;custom:tenant_id&#39;)<br>        <br>        if not email:<br>            frappe.throw(_(&quot;Email not found in token&quot;))<br>        <br>        # Check if user exists, create if not<br>        user = get_or_create_user(email, tenant_id)<br>        <br>        # Login user<br>        frappe.local.login_manager.user = user.name<br>        frappe.local.login_manager.post_login()<br>        <br>        return {<br>            &quot;status&quot;: &quot;success&quot;,<br>            &quot;user&quot;: user.name,<br>            &quot;tenant&quot;: tenant_id<br>        }<br>        <br>    except Exception as e:<br>        frappe.log_error(f&quot;Cognito login failed: {str(e)}&quot;)<br>        frappe.throw(_(&quot;Login failed&quot;))<br>def get_or_create_user(email, tenant_id):<br>    user = None<br>    try:<br>        user = frappe.get_doc(&quot;User&quot;, email)<br>    except frappe.DoesNotExistError:<br>        # Create new user<br>        user = frappe.get_doc({<br>            &quot;doctype&quot;: &quot;User&quot;,<br>            &quot;email&quot;: email,<br>            &quot;first_name&quot;: email.split(&#39;@&#39;)[0],<br>            &quot;send_welcome_email&quot;: 0,<br>            &quot;tenant_id&quot;: tenant_id<br>        })<br>        user.flags.ignore_permissions = True<br>        user.insert()<br>        <br>        # Add user to tenant-specific role<br>        add_user_to_tenant_role(user, tenant_id)<br>    <br>    return user<br>def add_user_to_tenant_role(user, tenant_id):<br>    # Add user to tenant-specific role<br>    tenant_role = f&quot;Tenant {tenant_id} User&quot;<br>    <br>    # Create role if doesn&#39;t exist<br>    if not frappe.db.exists(&quot;Role&quot;, tenant_role):<br>        role = frappe.get_doc({<br>            &quot;doctype&quot;: &quot;Role&quot;,<br>            &quot;role_name&quot;: tenant_role,<br>            &quot;desk_access&quot;: 1<br>        })<br>        role.insert()<br>    <br>    # Add role to user<br>    user.add_roles(tenant_role)</pre><h3>Step 5: Frontend Integration</h3><p>Create JavaScript for frontend authentication:</p><pre>// cognito_integration/public/js/cognito_auth.js<br>class CognitoAuthManager {<br>    constructor() {<br>        this.userPool = new AmazonCognitoIdentity.CognitoUserPool({<br>            UserPoolId: frappe.boot.cognito_user_pool_id,<br>            ClientId: frappe.boot.cognito_client_id<br>        });<br>    }<br>    <br>    async authenticateUser(username, password) {<br>        return new Promise((resolve, reject) =&gt; {<br>            const authenticationData = {<br>                Username: username,<br>                Password: password,<br>            };<br>            <br>            const authenticationDetails = new AmazonCognitoIdentity.AuthenticationDetails(authenticationData);<br>            <br>            const userData = {<br>                Username: username,<br>                Pool: this.userPool,<br>            };<br>            <br>            const cognitoUser = new AmazonCognitoIdentity.CognitoUser(userData);<br>            <br>            cognitoUser.authenticateUser(authenticationDetails, {<br>                onSuccess: (result) =&gt; {<br>                    const accessToken = result.getAccessToken().getJwtToken();<br>                    this.loginToFrappe(accessToken);<br>                    resolve(result);<br>                },<br>                onFailure: (err) =&gt; {<br>                    reject(err);<br>                }<br>            });<br>        });<br>    }<br>    <br>    async loginToFrappe(token) {<br>        try {<br>            const response = await frappe.call({<br>                method: &#39;cognito_integration.login.cognito_login&#39;,<br>                headers: {<br>                    &#39;Authorization&#39;: `Bearer ${token}`<br>                }<br>            });<br>            <br>            if (response.message.status === &#39;success&#39;) {<br>                window.location.reload();<br>            }<br>        } catch (error) {<br>            console.error(&#39;Frappe login failed:&#39;, error);<br>        }<br>    }<br>}<br>// Initialize on page load<br>frappe.ready(() =&gt; {<br>    window.cognitoAuth = new CognitoAuthManager();<br>});</pre><h3>Step 6: Configuration</h3><p>Add to common_site_config.json:</p><pre>{<br>    &quot;cognito_user_pool_id&quot;: &quot;us-east-1_XXXXXXXX&quot;,<br>    &quot;cognito_client_id&quot;: &quot;your-app-client-id&quot;, <br>    &quot;cognito_region&quot;: &quot;us-east-1&quot;,<br>    &quot;multi_tenant_enabled&quot;: true<br>}</pre><h3>Step 7: Custom Login Page</h3><p>Create a custom login page that integrates with Cognito:</p><pre>&lt;!-- apps/cognito_integration/cognito_integration/www/login.html --&gt;<br>&lt;div class=&quot;login-container&quot;&gt;<br>    &lt;div class=&quot;login-form&quot;&gt;<br>        &lt;h2&gt;{{ _(&quot;Login to&quot;) }} {{ tenant_name }}&lt;/h2&gt;<br>        &lt;form id=&quot;cognito-login-form&quot;&gt;<br>            &lt;div class=&quot;form-group&quot;&gt;<br>                &lt;input type=&quot;email&quot; id=&quot;username&quot; placeholder=&quot;{{ _(&#39;Email&#39;) }}&quot; required&gt;<br>            &lt;/div&gt;<br>            &lt;div class=&quot;form-group&quot;&gt;<br>                &lt;input type=&quot;password&quot; id=&quot;password&quot; placeholder=&quot;{{ _(&#39;Password&#39;) }}&quot; required&gt;<br>            &lt;/div&gt;<br>            &lt;button type=&quot;submit&quot; class=&quot;btn btn-primary&quot;&gt;{{ _(&quot;Login&quot;) }}&lt;/button&gt;<br>        &lt;/form&gt;<br>    &lt;/div&gt;<br>&lt;/div&gt;<br>&lt;script&gt;<br>document.getElementById(&#39;cognito-login-form&#39;).addEventListener(&#39;submit&#39;, async (e) =&gt; {<br>    e.preventDefault();<br>    <br>    const username = document.getElementById(&#39;username&#39;).value;<br>    const password = document.getElementById(&#39;password&#39;).value;<br>    <br>    try {<br>        await window.cognitoAuth.authenticateUser(username, password);<br>    } catch (error) {<br>        frappe.msgprint(&#39;Login failed: &#39; + error.message);<br>    }<br>});<br>&lt;/script&gt;</pre><h3>Testing &amp; Deployment</h3><h3>1. Test Multi-Tenancy</h3><pre># Test different tenant sites<br>curl -H &quot;Host: tenant1.company.com&quot; http://localhost:8000/api/method/frappe.auth.get_logged_user<br>curl -H &quot;Host: tenant2.company.com&quot; http://localhost:8000/api/method/frappe.auth.get_logged_user</pre><h3>2. Test Cognito Integration</h3><pre># Test token verification<br>import requests<br>token = &quot;your-cognito-jwt-token&quot;<br>response = requests.post(<br>    &#39;http://localhost:8000/api/method/cognito_integration.login.cognito_login&#39;,<br>    headers={&#39;Authorization&#39;: f&#39;Bearer {token}&#39;}<br>)</pre><h3>3. Production Deployment</h3><ol><li><strong>DNS Setup</strong>: Configure subdomains to point to your Frappe server</li><li><strong>SSL Certificates</strong>: Setup SSL for all tenant domains</li><li><strong>Load Balancer</strong>: Configure load balancer with proper routing</li><li><strong>Monitoring</strong>: Setup monitoring for each tenant</li></ol><h3>4. Security Considerations</h3><ul><li><strong>Token Validation</strong>: Always verify JWT tokens server-side</li><li><strong>Data Isolation</strong>: Ensure complete data separation between tenants</li><li><strong>Rate Limiting</strong>: Implement rate limiting per tenant</li><li><strong>Audit Logging</strong>: Log all tenant activities</li></ul><h3>Troubleshooting</h3><h3>Common Issues</h3><ol><li><strong>Site Not Found</strong>: Ensure tenant site is created and accessible</li><li><strong>Token Verification Failed</strong>: Check Cognito configuration and JWT keys</li><li><strong>User Creation Failed</strong>: Verify user permissions and tenant role setup</li><li><strong>Database Connection</strong>: Ensure each tenant has proper database access</li></ol><h3>Debugging Tips</h3><pre># Enable debug logging<br>frappe.conf.developer_mode = 1<br>frappe.conf.log_level = &quot;DEBUG&quot;<br># Check tenant detection<br>print(f&quot;Current tenant: {frappe.local.tenant_id}&quot;)<br>print(f&quot;Current site: {frappe.local.site}&quot;)</pre><h3>Conclusion</h3><p>This implementation provides a robust multi-tenant Frappe ERP system with AWS Cognito SSO integration. The architecture ensures data isolation, centralized authentication, and scalability for growing businesses.</p><p>Key benefits achieved:</p><ul><li>Seamless SSO experience across all tenant applications</li><li>Complete data isolation between tenants</li><li>Centralized user management through AWS Cognito</li><li>Scalable architecture for adding new tenants</li></ul><p>For production deployment, ensure proper testing, monitoring, and security measures are in place.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ed336a3a5e2f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[New Relic AWS Integration Series: Complete Observability Guide — Part -2]]></title>
            <link>https://medium.com/@to.adarshsingh/new-relic-aws-integration-series-complete-observability-guide-part-2-910f847fe71d?source=rss-73f6989daa12------2</link>
            <guid isPermaLink="false">https://medium.com/p/910f847fe71d</guid>
            <dc:creator><![CDATA[Adarsh Singh Dikhit]]></dc:creator>
            <pubDate>Mon, 11 Aug 2025 20:36:24 GMT</pubDate>
            <atom:updated>2025-08-11T20:36:24.115Z</atom:updated>
            <content:encoded><![CDATA[<h3><a href="https://medium.com/@to.adarshsingh/new-relic-aws-integration-series-complete-observability-guide-d8c4223bfbc2?source=your_stories_page--------------------------------------------">New Relic AWS Integration Series: Complete Observability Guide</a> — Part -2</h3><p>Continue reading <a href="#part-4-eks-kubernetes-integration">Part 4: EKS Integration</a> for container monitoring…</p><h3>Complete Data Flow Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B89iIOx0sefk6_MfdWW_iA.png" /></figure><h3>Part 4: EKS (Kubernetes) Integration</h3><h3>Container and Orchestration Monitoring</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r048xNEN3O1sYtfuCA_dFQ.png" /></figure><p>Kubernetes monitoring requires a multi-layered approach to capture the full stack from infrastructure to applications. New Relic’s Kubernetes integration provides comprehensive visibility across all layers of your EKS environment.</p><h3>EKS Implementation</h3><h4>Installation via Helm Chart</h4><p><strong>1. Add New Relic Helm Repository</strong></p><pre># Add the New Relic Helm repository<br>helm repo add newrelic https://helm-charts.newrelic.com<br>helm repo update<br># Create namespace for New Relic components<br>kubectl create namespace newrelic</pre><p><strong>2. Configure Values File (values.yaml)</strong></p><pre># values.yaml for New Relic Kubernetes integration<br>global:<br>  licenseKey: &quot;YOUR_LICENSE_KEY&quot;<br>  cluster: &quot;eks-production-cluster&quot;<br>  <br>  # Custom attributes for all data<br>  customAttributes:<br>    environment: &quot;production&quot;<br>    team: &quot;platform&quot;<br>    cost_center: &quot;engineering&quot;<br># New Relic Infrastructure (DaemonSet)<br>newrelic-infrastructure:<br>  enabled: true<br>  <br>  # Resource limits<br>  resources:<br>    limits:<br>      memory: &quot;300Mi&quot;<br>      cpu: &quot;150m&quot;<br>    requests:<br>      memory: &quot;150Mi&quot;<br>      cpu: &quot;100m&quot;<br>  <br>  # Node selector for specific node groups<br>  nodeSelector:<br>    kubernetes.io/os: linux<br>  <br>  # Tolerations for master nodes<br>  tolerations:<br>    - key: &quot;node-role.kubernetes.io/master&quot;<br>      operator: &quot;Exists&quot;<br>      effect: &quot;NoSchedule&quot;<br># Kubernetes Events Integration<br>newrelic-k8s-events:<br>  enabled: true<br>  <br>  # Resource configuration<br>  resources:<br>    limits:<br>      memory: &quot;128Mi&quot;<br>      cpu: &quot;100m&quot;<br>    requests:<br>      memory: &quot;64Mi&quot;<br>      cpu: &quot;50m&quot;<br># Prometheus Metrics Integration<br>newrelic-prometheus:<br>  enabled: true<br>  <br>  # Scrape configuration<br>  config:<br>    # Scrape Kubernetes API server metrics<br>    - job_name: &#39;kubernetes-apiservers&#39;<br>      kubernetes_sd_configs:<br>        - role: endpoints<br>      scheme: https<br>      tls_config:<br>        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt<br>      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token<br>      relabel_configs:<br>        - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]<br>          action: keep<br>          regex: default;kubernetes;https<br>    <br>    # Scrape kubelet metrics<br>    - job_name: &#39;kubernetes-nodes&#39;<br>      kubernetes_sd_configs:<br>        - role: node<br>      scheme: https<br>      tls_config:<br>        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt<br>      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token<br>      relabel_configs:<br>        - action: labelmap<br>          regex: __meta_kubernetes_node_label_(.+)<br># Log Forwarding<br>newrelic-logging:<br>  enabled: true<br>  <br>  # Fluentbit configuration<br>  fluentBit:<br>    config:<br>      outputs: |<br>        [OUTPUT]<br>            Name newrelic<br>            Match *<br>            licenseKey ${LICENSE_KEY}<br>            endpoint https://log-api.newrelic.com/log/v1<br>            maxBufferSize 256000<br>            maxRecords 1024<br>      <br>      filters: |<br>        [FILTER]<br>            Name kubernetes<br>            Match kube.*<br>            Merge_Log On<br>            K8S-Logging.Parser On<br>            K8S-Logging.Exclude Off<br>        <br>        [FILTER]<br>            Name record_modifier<br>            Match *<br>            Record cluster_name ${CLUSTER_NAME}<br>            Record environment production<br># Pixie Integration (Advanced Observability)<br>pixie:<br>  enabled: true<br>  deployKey: &quot;YOUR_PIXIE_DEPLOY_KEY&quot;<br>  <br># Kube State Metrics<br>kube-state-metrics:<br>  enabled: true<br>  <br>  # Custom resource monitoring<br>  collectors:<br>    - deployments<br>    - replicasets<br>    - pods<br>    - services<br>    - nodes<br>    - persistentvolumes<br>    - persistentvolumeclaims<br>    - configmaps<br>    - secrets</pre><p><strong>3. Install via Helm</strong></p><pre># Install New Relic Kubernetes integration<br>helm upgrade --install newrelic-bundle newrelic/nri-bundle \<br>    --namespace newrelic \<br>    --values values.yaml \<br>    --set global.licenseKey=YOUR_LICENSE_KEY \<br>    --set global.cluster=eks-production-cluster<br># Verify installation<br>kubectl get pods -n newrelic<br>kubectl get daemonset -n newrelic</pre><h4>Manual YAML Deployment</h4><p><strong>1. Create RBAC Configuration</strong></p><pre># rbac.yaml<br>apiVersion: v1<br>kind: ServiceAccount<br>metadata:<br>  name: newrelic-infrastructure<br>  namespace: newrelic<br>---<br>apiVersion: rbac.authorization.k8s.io/v1<br>kind: ClusterRole<br>metadata:<br>  name: newrelic-infrastructure<br>rules:<br>- apiGroups: [&quot;&quot;]<br>  resources:<br>    - nodes<br>    - nodes/metrics<br>    - nodes/stats<br>    - nodes/proxy<br>    - pods<br>    - services<br>    - endpoints<br>    - persistentvolumes<br>    - persistentvolumeclaims<br>    - events<br>    - secrets<br>    - configmaps<br>    - namespaces<br>  verbs: [&quot;get&quot;, &quot;list&quot;, &quot;watch&quot;]<br>- apiGroups: [&quot;apps&quot;]<br>  resources:<br>    - deployments<br>    - replicasets<br>    - daemonsets<br>    - statefulsets<br>  verbs: [&quot;get&quot;, &quot;list&quot;, &quot;watch&quot;]<br>- apiGroups: [&quot;batch&quot;]<br>  resources:<br>    - jobs<br>    - cronjobs<br>  verbs: [&quot;get&quot;, &quot;list&quot;, &quot;watch&quot;]<br>- apiGroups: [&quot;metrics.k8s.io&quot;]<br>  resources:<br>    - nodes<br>    - pods<br>  verbs: [&quot;get&quot;, &quot;list&quot;]<br>---<br>apiVersion: rbac.authorization.k8s.io/v1<br>kind: ClusterRoleBinding<br>metadata:<br>  name: newrelic-infrastructure<br>roleRef:<br>  apiGroup: rbac.authorization.k8s.io<br>  kind: ClusterRole<br>  name: newrelic-infrastructure<br>subjects:<br>- kind: ServiceAccount<br>  name: newrelic-infrastructure<br>  namespace: newrelic</pre><p><strong>2. DaemonSet Configuration</strong></p><pre># daemonset.yaml<br>apiVersion: apps/v1<br>kind: DaemonSet<br>metadata:<br>  name: newrelic-infrastructure<br>  namespace: newrelic<br>  labels:<br>    app: newrelic-infrastructure<br>spec:<br>  selector:<br>    matchLabels:<br>      app: newrelic-infrastructure<br>  template:<br>    metadata:<br>      labels:<br>        app: newrelic-infrastructure<br>    spec:<br>      serviceAccountName: newrelic-infrastructure<br>      hostNetwork: true<br>      hostPID: true<br>      hostIPC: true<br>      containers:<br>      - name: newrelic-infrastructure<br>        image: newrelic/infrastructure-k8s:latest<br>        securityContext:<br>          privileged: true<br>        env:<br>        - name: NRIA_LICENSE_KEY<br>          valueFrom:<br>            secretKeyRef:<br>              name: newrelic-license<br>              key: license<br>        - name: NRI_KUBERNETES_CLUSTER_NAME<br>          value: &quot;eks-production-cluster&quot;<br>        - name: NRIA_DISPLAY_NAME<br>          valueFrom:<br>            fieldRef:<br>              fieldPath: spec.nodeName<br>        - name: NRIA_CUSTOM_ATTRIBUTES<br>          value: &#39;{&quot;environment&quot;:&quot;production&quot;,&quot;team&quot;:&quot;platform&quot;}&#39;<br>        - name: NRIA_PASSTHROUGH_ENVIRONMENT<br>          value: &quot;KUBERNETES_SERVICE_HOST,KUBERNETES_SERVICE_PORT,CLUSTER_NAME,CADVISOR_PORT,NRK8S_NODE_NAME,KUBE_STATE_METRICS_URL,KUBE_STATE_METRICS_POD_LABEL_SELECTOR,TIMEOUT,ETCD_TLS_SECRET_NAME,ETCD_TLS_SECRET_NAMESPACE,API_SERVER_SECURE_PORT,KUBE_STATE_METRICS_SCHEME,KUBE_STATE_METRICS_PORT,SCHEDULER_ENDPOINT_URL,ETCD_ENDPOINT_URL,CONTROLLER_MANAGER_ENDPOINT_URL,API_SERVER_ENDPOINT_URL,DISABLE_KUBE_STATE_METRICS,DISCOVERY_CACHE_TTL&quot;<br>        volumeMounts:<br>        - name: host-docker-socket<br>          mountPath: /var/run/docker.sock<br>        - name: host-fs<br>          mountPath: /host<br>          readOnly: true<br>        - name: host-proc<br>          mountPath: /host/proc<br>          readOnly: true<br>        - name: host-sys<br>          mountPath: /host/sys<br>          readOnly: true<br>        - name: host-etc<br>          mountPath: /host/etc<br>          readOnly: true<br>        - name: host-var<br>          mountPath: /host/var<br>          readOnly: true<br>        resources:<br>          limits:<br>            memory: &quot;300Mi&quot;<br>            cpu: &quot;150m&quot;<br>          requests:<br>            memory: &quot;150Mi&quot;<br>            cpu: &quot;100m&quot;<br>      volumes:<br>      - name: host-docker-socket<br>        hostPath:<br>          path: /var/run/docker.sock<br>      - name: host-fs<br>        hostPath:<br>          path: /<br>      - name: host-proc<br>        hostPath:<br>          path: /proc<br>      - name: host-sys<br>        hostPath:<br>          path: /sys<br>      - name: host-etc<br>        hostPath:<br>          path: /etc<br>      - name: host-var<br>        hostPath:<br>          path: /var<br>      tolerations:<br>      - key: &quot;node-role.kubernetes.io/master&quot;<br>        operator: &quot;Exists&quot;<br>        effect: &quot;NoSchedule&quot;<br>      - key: &quot;node-role.kubernetes.io/control-plane&quot;<br>        operator: &quot;Exists&quot;<br>        effect: &quot;NoSchedule&quot;</pre><h3>EKS APM Integration</h3><h4>Application Monitoring in Kubernetes</h4><p><strong>1. Java Application Example</strong></p><pre># java-app-deployment.yaml<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: java-web-app<br>  namespace: production<br>  labels:<br>    app: java-web-app<br>spec:<br>  replicas: 3<br>  selector:<br>    matchLabels:<br>      app: java-web-app<br>  template:<br>    metadata:<br>      labels:<br>        app: java-web-app<br>      annotations:<br>        # Enable automatic instrumentation<br>        newrelic.io/instrument: &quot;true&quot;<br>    spec:<br>      containers:<br>      - name: java-web-app<br>        image: myapp:latest<br>        ports:<br>        - containerPort: 8080<br>        env:<br>        # New Relic APM Configuration<br>        - name: NEW_RELIC_LICENSE_KEY<br>          valueFrom:<br>            secretKeyRef:<br>              name: newrelic-license<br>              key: license<br>        - name: NEW_RELIC_APP_NAME<br>          value: &quot;Java Web App - Production&quot;<br>        - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED<br>          value: &quot;true&quot;<br>        - name: NEW_RELIC_LABELS<br>          value: &quot;environment:production;cluster:eks-prod;namespace:production&quot;<br>        - name: JAVA_OPTS<br>          value: &quot;-javaagent:/opt/newrelic/newrelic.jar&quot;<br>        <br>        # Application-specific environment variables<br>        - name: CLUSTER_NAME<br>          value: &quot;eks-production-cluster&quot;<br>        - name: NODE_NAME<br>          valueFrom:<br>            fieldRef:<br>              fieldPath: spec.nodeName<br>        - name: POD_NAME<br>          valueFrom:<br>            fieldRef:<br>              fieldPath: metadata.name<br>        - name: NAMESPACE<br>          valueFrom:<br>            fieldRef:<br>              fieldPath: metadata.namespace<br>        <br>        volumeMounts:<br>        - name: newrelic-agent<br>          mountPath: /opt/newrelic<br>        <br>        resources:<br>          limits:<br>            memory: &quot;1Gi&quot;<br>            cpu: &quot;500m&quot;<br>          requests:<br>            memory: &quot;512Mi&quot;<br>            cpu: &quot;250m&quot;<br>        <br>        # Health checks<br>        livenessProbe:<br>          httpGet:<br>            path: /health<br>            port: 8080<br>          initialDelaySeconds: 30<br>          periodSeconds: 10<br>        readinessProbe:<br>          httpGet:<br>            path: /ready<br>            port: 8080<br>          initialDelaySeconds: 5<br>          periodSeconds: 5<br>      <br>      volumes:<br>      - name: newrelic-agent<br>        initContainer:<br>        - name: newrelic-java-agent<br>          image: newrelic/newrelic-java-init:latest<br>          volumeMounts:<br>          - name: newrelic-agent<br>            mountPath: /opt/newrelic<br>          env:<br>          - name: NEW_RELIC_LICENSE_KEY<br>            valueFrom:<br>              secretKeyRef:<br>                name: newrelic-license<br>                key: license</pre><p><strong>2. Node.js Application with Sidecar Pattern</strong></p><pre># nodejs-app-deployment.yaml<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: nodejs-api<br>  namespace: production<br>spec:<br>  replicas: 2<br>  selector:<br>    matchLabels:<br>      app: nodejs-api<br>  template:<br>    metadata:<br>      labels:<br>        app: nodejs-api<br>    spec:<br>      containers:<br>      # Main application container<br>      - name: nodejs-api<br>        image: node:16-alpine<br>        ports:<br>        - containerPort: 3000<br>        env:<br>        - name: NEW_RELIC_LICENSE_KEY<br>          valueFrom:<br>            secretKeyRef:<br>              name: newrelic-license<br>              key: license<br>        - name: NEW_RELIC_APP_NAME<br>          value: &quot;Node.js API - Production&quot;<br>        - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED<br>          value: &quot;true&quot;<br>        - name: NEW_RELIC_LABELS<br>          value: &quot;environment:production;service:api;version:v2&quot;<br>        <br>        command: [&quot;node&quot;]<br>        args: [&quot;-r&quot;, &quot;@newrelic/native-metrics&quot;, &quot;-r&quot;, &quot;newrelic&quot;, &quot;server.js&quot;]<br>        <br>        resources:<br>          limits:<br>            memory: &quot;512Mi&quot;<br>            cpu: &quot;250m&quot;<br>          requests:<br>            memory: &quot;256Mi&quot;<br>            cpu: &quot;100m&quot;<br>      <br>      # New Relic Infrastructure sidecar<br>      - name: newrelic-sidecar<br>        image: newrelic/infrastructure:latest<br>        env:<br>        - name: NRIA_LICENSE_KEY<br>          valueFrom:<br>            secretKeyRef:<br>              name: newrelic-license<br>              key: license<br>        - name: NRIA_DISPLAY_NAME<br>          value: &quot;nodejs-api-sidecar&quot;<br>        resources:<br>          limits:<br>            memory: &quot;100Mi&quot;<br>            cpu: &quot;50m&quot;<br>          requests:<br>            memory: &quot;50Mi&quot;<br>            cpu: &quot;25m&quot;</pre><h4>Custom Metrics and Events in Kubernetes</h4><p><strong>1. Kubernetes Event Monitoring</strong></p><pre># k8s-events-monitor.yaml<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: newrelic-k8s-events<br>  namespace: newrelic<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: newrelic-k8s-events<br>  template:<br>    metadata:<br>      labels:<br>        app: newrelic-k8s-events<br>    spec:<br>      serviceAccountName: newrelic-infrastructure<br>      containers:<br>      - name: newrelic-k8s-events<br>        image: newrelic/nri-kube-events:latest<br>        env:<br>        - name: NRI_LICENSE_KEY<br>          valueFrom:<br>            secretKeyRef:<br>              name: newrelic-license<br>              key: license<br>        - name: NRI_CLUSTER_NAME<br>          value: &quot;eks-production-cluster&quot;<br>        resources:<br>          limits:<br>            memory: &quot;128Mi&quot;<br>            cpu: &quot;100m&quot;<br>          requests:<br>            memory: &quot;64Mi&quot;<br>            cpu: &quot;50m&quot;</pre><p><strong>2. Custom Metrics via Prometheus</strong></p><pre># prometheus-custom-metrics.yaml<br>apiVersion: v1<br>kind: ServiceMonitor<br>metadata:<br>  name: custom-app-metrics<br>  namespace: production<br>  labels:<br>    app: custom-app-metrics<br>spec:<br>  selector:<br>    matchLabels:<br>      app: java-web-app<br>  endpoints:<br>  - port: metrics<br>    interval: 30s<br>    path: /actuator/prometheus<br>    scrapeTimeout: 10s<br>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: java-web-app-metrics<br>  namespace: production<br>  labels:<br>    app: java-web-app<br>spec:<br>  ports:<br>  - name: metrics<br>    port: 8081<br>    targetPort: 8081<br>  selector:<br>    app: java-web-app</pre><h3>Part 5: Additional AWS Services</h3><h3>RDS, S3, CloudFront, and More</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CNmlPpTcwzs2XqBreB4O-g.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=910f847fe71d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[New Relic AWS Integration Series: Complete Observability Guide]]></title>
            <link>https://medium.com/@to.adarshsingh/new-relic-aws-integration-series-complete-observability-guide-d8c4223bfbc2?source=rss-73f6989daa12------2</link>
            <guid isPermaLink="false">https://medium.com/p/d8c4223bfbc2</guid>
            <dc:creator><![CDATA[Adarsh Singh Dikhit]]></dc:creator>
            <pubDate>Mon, 11 Aug 2025 20:28:53 GMT</pubDate>
            <atom:updated>2025-08-11T20:28:53.575Z</atom:updated>
            <content:encoded><![CDATA[<h3>Table of Contents</h3><h3>Part 1: Architecture and Foundation</h3><ul><li><a href="#part-1-architecture-and-foundation">Understanding New Relic’s AWS Integration Architecture</a></li><li><a href="#core-components-and-data-flow">Core Components and Data Flow</a></li><li><a href="#planning-your-integration-strategy">Planning Your Integration Strategy</a></li></ul><h3>Part 2: AWS Lambda Integration</h3><ul><li><a href="#part-2-aws-lambda-integration">Serverless Monitoring with New Relic</a></li><li><a href="#lambda-implementation">Implementation and Configuration</a></li><li><a href="#lambda-optimization">Performance Optimization</a></li></ul><h3>Part 3: EC2 Integration</h3><ul><li><a href="#part-3-ec2-integration">Traditional Infrastructure Monitoring</a></li><li><a href="#ec2-implementation">Agent Installation and Configuration</a></li><li><a href="#ec2-dashboards">Custom Metrics and Dashboards</a></li></ul><h3>Part 4: EKS (Kubernetes) Integration</h3><ul><li><a href="#part-4-eks-kubernetes-integration">Container and Orchestration Monitoring</a></li><li><a href="#eks-implementation">Cluster-wide Observability</a></li><li><a href="#eks-apm">Application Performance in Kubernetes</a></li></ul><h3>Part 5: Additional AWS Services</h3><ul><li><a href="#part-5-additional-aws-services">RDS, S3, CloudFront, and More</a></li><li><a href="#cross-service-correlation">Cross-service Correlation</a></li><li><a href="#advanced-patterns">Advanced Monitoring Patterns</a></li></ul><h3>Part 6: Best Practices and Troubleshooting</h3><ul><li><a href="#part-6-best-practices">Optimization and Cost Management</a></li><li><a href="#troubleshooting">Common Issues and Solutions</a></li><li><a href="#enterprise-deployment">Enterprise-scale Deployments</a></li></ul><h3>Part 1: Architecture and Foundation</h3><h3>Understanding New Relic’s AWS Integration Architecture</h3><p>New Relic’s AWS integration provides comprehensive observability across your entire AWS infrastructure through multiple data collection methods and integration points. Understanding the architecture is crucial for effective implementation and troubleshooting.</p><h4>High-Level Architecture Overview</h4><pre>┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐<br>│   AWS Services  │    │  New Relic      │    │   New Relic     │<br>│                 │────│  Collector      │────│   Platform      │<br>│ • Lambda        │    │                 │    │                 │<br>│ • EC2           │    │ • Agents        │    │ • Dashboards    │<br>│ • EKS           │    │ • CloudWatch    │    │ • Alerts        │<br>│ • RDS           │    │ • API Polling   │    │ • AI Insights   │<br>│ • S3            │    │ • Log Shipping  │    │ • Query Builder │<br>└─────────────────┘    └─────────────────┘    └─────────────────┘</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AYquIAOZVwrl4-xP21TnFw.png" /></figure><h4>Core Components and Data Flow</h4><p><strong>1. Infrastructure Agent</strong></p><ul><li>Lightweight agent installed on EC2 instances</li><li>Collects system metrics, processes, and network data</li><li>Supports custom attributes and integrations</li><li>Automatic discovery of services</li></ul><p><strong>2. CloudWatch Integration</strong></p><ul><li>Polls AWS CloudWatch APIs for service metrics</li><li>No additional infrastructure required</li><li>Covers 50+ AWS services out of the box</li><li>Configurable polling intervals</li></ul><p><strong>3. Application Performance Monitoring (APM)</strong></p><ul><li>Language-specific agents for applications</li><li>Distributed tracing across services</li><li>Database query analysis</li><li>Error tracking and alerting</li></ul><p><strong>4. Logs Integration</strong></p><ul><li>CloudWatch Logs forwarding</li><li>Application logs correlation</li><li>Log parsing and analysis</li><li>Custom log attributes</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nJueu9uPPkxHmXFYSSBWJw.png" /></figure><h3>Planning Your Integration Strategy</h3><h4>Assessment Phase</h4><p><strong>1. Service Inventory</strong></p><ul><li>Catalog all AWS services in use</li><li>Identify critical applications and dependencies</li><li>Document current monitoring gaps</li><li>Assess compliance requirements</li></ul><p><strong>2. Data Requirements</strong></p><ul><li>Define key performance indicators (KPIs)</li><li>Establish baseline metrics</li><li>Determine alerting thresholds</li><li>Plan retention requirements</li></ul><p><strong>3. Access and Permissions</strong></p><ul><li>Set up AWS IAM roles and policies</li><li>Configure New Relic API keys</li><li>Establish cross-account access if needed</li><li>Document security requirements</li></ul><h4>Implementation Phases</h4><p><strong>Phase 1: Foundation (Week 1–2)</strong></p><ul><li>Set up New Relic account and basic AWS integration</li><li>Deploy CloudWatch integration for immediate visibility</li><li>Configure basic dashboards and alerts</li><li>Establish data pipeline validation</li></ul><p><strong>Phase 2: Deep Monitoring (Week 3–4)</strong></p><ul><li>Install infrastructure agents on critical EC2 instances</li><li>Deploy APM agents for key applications</li><li>Configure custom metrics and attributes</li><li>Set up log forwarding</li></ul><p><strong>Phase 3: Advanced Features (Week 5–6)</strong></p><ul><li>Implement distributed tracing</li><li>Configure AI-powered insights</li><li>Set up synthetic monitoring</li><li>Create custom dashboards and workflows</li></ul><p><strong>Phase 4: Optimization (Week 7–8)</strong></p><ul><li>Fine-tune alert policies</li><li>Optimize data retention and costs</li><li>Implement automation and infrastructure as code</li><li>Conduct team training and documentation</li></ul><h3>Part 2: AWS Lambda Integration</h3><h3>Serverless Monitoring with New Relic</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kDqVQGAzL7cJc4fyor8dOA.png" /></figure><p>AWS Lambda presents unique monitoring challenges due to its ephemeral, event-driven nature. New Relic provides specialized tools and approaches for comprehensive Lambda observability.</p><h4>Lambda Monitoring Challenges</h4><p><strong>Traditional Monitoring Limitations:</strong></p><ul><li>Short execution times (15 minutes max)</li><li>Cold start performance impact</li><li>Limited persistent storage</li><li>Dynamic scaling behavior</li><li>Distributed system complexity</li></ul><p><strong>New Relic Solutions:</strong></p><ul><li>Lightweight wrapper libraries</li><li>Distributed tracing across Lambda functions</li><li>Cold start analysis and optimization</li><li>Integration with AWS X-Ray</li><li>Custom event and metric collection</li></ul><h3>Lambda Implementation</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J9K5VsvWVRsfJjR8fb0sRA.png" /></figure><h4>Method 1: Layer-based Installation (Recommended)</h4><p><strong>1. Add New Relic Layer</strong></p><pre># Using AWS CLI<br>aws lambda update-function-configuration \<br>  --function-name your-function-name \<br>  --layers arn:aws:lambda:us-east-1:451483290750:layer:NewRelicPython39:58<br><br># Using Serverless Framework<br># serverless.yml<br>functions:<br>  myFunction:<br>    handler: handler.lambda_handler<br>    layers:<br>      - arn:aws:lambda:us-east-1:451483290750:layer:NewRelicPython39:58<br>    environment:<br>      NEW_RELIC_LAMBDA_HANDLER: handler.lambda_handler<br>      NEW_RELIC_LICENSE_KEY: ${env:NEW_RELIC_LICENSE_KEY}<br>      NEW_RELIC_ACCOUNT_ID: ${env:NEW_RELIC_ACCOUNT_ID}</pre><p><strong>2. Configure Environment Variables</strong></p><pre>NEW_RELIC_LAMBDA_HANDLER=handler.lambda_handler<br>NEW_RELIC_LICENSE_KEY=your_license_key_here<br>NEW_RELIC_ACCOUNT_ID=your_account_id<br>NEW_RELIC_TRUSTED_ACCOUNT_KEY=your_account_id<br>NEW_RELIC_DISTRIBUTED_TRACING_ENABLED=true<br>NEW_RELIC_EXTENSION_SEND_FUNCTION_LOGS=true</pre><p><strong>3. Modify Lambda Handler (Python Example)</strong></p><pre>import newrelic.agent<br><br># Original handler<br>def lambda_handler(event, context):<br>    # Your application logic here<br>    return {<br>        &#39;statusCode&#39;: 200,<br>        &#39;body&#39;: &#39;Hello from Lambda!&#39;<br>    }<br><br># Wrapped handler (automatically done by layer)<br>@newrelic.agent.lambda_handler()<br>def lambda_handler(event, context):<br>    # Add custom attributes<br>    newrelic.agent.add_custom_attribute(&#39;event_source&#39;, event.get(&#39;source&#39;, &#39;unknown&#39;))<br>    newrelic.agent.add_custom_attribute(&#39;user_id&#39;, event.get(&#39;userId&#39;))<br>    <br>    # Your application logic here<br>    result = process_event(event)<br>    <br>    # Record custom metrics<br>    newrelic.agent.record_custom_metric(&#39;Custom/ProcessedEvents&#39;, 1)<br>    <br>    return result<br><br>def process_event(event):<br>    # Simulate processing time<br>    import time<br>    time.sleep(0.1)<br>    <br>    # Add database transaction tracing<br>    with newrelic.agent.database_trace(&#39;SELECT&#39;, &#39;users&#39;, &#39;get_user&#39;):<br>        # Database operation<br>        pass<br>    <br>    return {<br>        &#39;statusCode&#39;: 200,<br>        &#39;body&#39;: f&#39;Processed event: {event.get(&quot;id&quot;, &quot;unknown&quot;)}&#39;<br>    }</pre><h4>Method 2: Manual Installation</h4><p><strong>1. Install New Relic Package</strong></p><pre># Create deployment package<br>mkdir lambda-package<br>cd lambda-package<br><br># Install New Relic<br>pip install newrelic -t .<br><br># Add your code<br>cp ../handler.py .</pre><p><strong>2. Configure newrelic.ini</strong></p><pre>[newrelic]<br>license_key = your_license_key_here<br>app_name = My Lambda Function<br>distributed_tracing.enabled = true<br>serverless_mode.enabled = true<br>[newrelic:serverless]<br># Serverless specific settings</pre><h4>Advanced Lambda Configuration</h4><p><strong>1. Custom Metrics and Events</strong></p><pre>import newrelic.agent<br>@newrelic.agent.lambda_handler()<br>def lambda_handler(event, context):<br>    # Record custom events<br>    newrelic.agent.record_custom_event(&#39;LambdaExecution&#39;, {<br>        &#39;functionName&#39;: context.function_name,<br>        &#39;requestId&#39;: context.aws_request_id,<br>        &#39;eventSource&#39;: event.get(&#39;Records&#39;, [{}])[0].get(&#39;eventSource&#39;),<br>        &#39;processingTime&#39;: 0  # Will be updated<br>    })<br>    <br>    # Business logic with timing<br>    start_time = time.time()<br>    result = business_logic(event)<br>    processing_time = time.time() - start_time<br>    <br>    # Update custom metric<br>    newrelic.agent.record_custom_metric(&#39;Custom/ProcessingTime&#39;, processing_time)<br>    <br>    return result</pre><p><strong>2. Error Handling and Alerting</strong></p><pre>import newrelic.agent<br>@newrelic.agent.lambda_handler()<br>def lambda_handler(event, context):<br>    try:<br>        return process_request(event)<br>    except ValidationError as e:<br>        # Record non-critical errors<br>        newrelic.agent.record_custom_event(&#39;ValidationError&#39;, {<br>            &#39;error&#39;: str(e),<br>            &#39;functionName&#39;: context.function_name,<br>            &#39;severity&#39;: &#39;warning&#39;<br>        })<br>        return {&#39;statusCode&#39;: 400, &#39;body&#39;: str(e)}<br>    except Exception as e:<br>        # Critical errors are automatically tracked<br>        newrelic.agent.record_exception()<br>        raise</pre><h3>Lambda Optimization</h3><h4>Cold Start Analysis</h4><p><strong>1. Monitor Cold Start Metrics</strong></p><pre>import os<br>import time<br>import newrelic.agent<br># Global variable to track cold starts<br>COLD_START = True<br>@newrelic.agent.lambda_handler()<br>def lambda_handler(event, context):<br>    global COLD_START<br>    <br>    # Record cold start metric<br>    if COLD_START:<br>        newrelic.agent.add_custom_attribute(&#39;coldStart&#39;, True)<br>        newrelic.agent.record_custom_metric(&#39;Custom/ColdStarts&#39;, 1)<br>        COLD_START = False<br>    else:<br>        newrelic.agent.add_custom_attribute(&#39;coldStart&#39;, False)<br>    <br>    # Your application logic<br>    return process_event(event)</pre><p><strong>2. Provisioned Concurrency Monitoring</strong></p><pre>@newrelic.agent.lambda_handler()<br>def lambda_handler(event, context):<br>    # Check if running on provisioned concurrency<br>    is_provisioned = os.environ.get(&#39;AWS_LAMBDA_INITIALIZATION_TYPE&#39;) == &#39;provisioned-concurrency&#39;<br>    <br>    newrelic.agent.add_custom_attribute(&#39;provisionedConcurrency&#39;, is_provisioned)<br>    <br>    if is_provisioned:<br>        newrelic.agent.record_custom_metric(&#39;Custom/ProvisionedExecutions&#39;, 1)<br>    else:<br>        newrelic.agent.record_custom_metric(&#39;Custom/OnDemandExecutions&#39;, 1)<br>    <br>    return process_event(event)</pre><h4>Performance Optimization Patterns</h4><p><strong>1. Connection Reuse</strong></p><pre>import newrelic.agent<br>import psycopg2<br>from functools import lru_cache<br><br># Initialize connections outside handler<br>@lru_cache(maxsize=1)<br>def get_db_connection():<br>    return psycopg2.connect(<br>        host=os.environ[&#39;DB_HOST&#39;],<br>        database=os.environ[&#39;DB_NAME&#39;],<br>        user=os.environ[&#39;DB_USER&#39;],<br>        password=os.environ[&#39;DB_PASSWORD&#39;]<br>    )<br><br>@newrelic.agent.lambda_handler()<br>def lambda_handler(event, context):<br>    # Reuse connection across invocations<br>    conn = get_db_connection()<br>    <br>    with newrelic.agent.database_trace(&#39;SELECT&#39;, &#39;orders&#39;, &#39;get_orders&#39;):<br>        with conn.cursor() as cur:<br>            cur.execute(&quot;SELECT * FROM orders WHERE user_id = %s&quot;, (event[&#39;userId&#39;],))<br>            results = cur.fetchall()<br>    <br>    return {&#39;statusCode&#39;: 200, &#39;body&#39;: json.dumps(results)}</pre><p><strong>2. Async Processing Patterns</strong></p><pre>import asyncio<br>import aiohttp<br>import newrelic.agent<br><br>@newrelic.agent.lambda_handler()<br>def lambda_handler(event, context):<br>    return asyncio.run(async_handler(event, context))<br><br>async def async_handler(event, context):<br>    # Parallel API calls<br>    async with aiohttp.ClientSession() as session:<br>        tasks = []<br>        for item in event.get(&#39;items&#39;, []):<br>            task = process_item_async(session, item)<br>            tasks.append(task)<br>        <br>        results = await asyncio.gather(*tasks)<br>    <br>    newrelic.agent.record_custom_metric(&#39;Custom/ProcessedItems&#39;, len(results))<br>    return {&#39;statusCode&#39;: 200, &#39;results&#39;: results}<br><br>@newrelic.agent.function_trace()<br>async def process_item_async(session, item):<br>    async with session.get(f&quot;https://api.example.com/items/{item[&#39;id&#39;]}&quot;) as response:<br>        return await response.json()</pre><h3>Part 3: EC2 Integration</h3><h3>Traditional Infrastructure Monitoring</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EvxEO_BvPBV1Qw6g-JxmvA.png" /></figure><p>EC2 instances represent the foundation of many AWS deployments, requiring comprehensive monitoring of both infrastructure and application performance. New Relic’s infrastructure agent provides deep visibility into EC2 instances while maintaining minimal performance overhead.</p><h4>EC2 Monitoring Scope</h4><p><strong>Infrastructure Metrics:</strong></p><ul><li>CPU, memory, and disk utilization</li><li>Network I/O and connections</li><li>Process monitoring and inventory</li><li>File system and storage metrics</li><li>System logs and events</li></ul><p><strong>Application Metrics:</strong></p><ul><li>APM agent integration</li><li>Custom application metrics</li><li>Service discovery and dependencies</li><li>Performance profiling</li><li>Error tracking and alerting</li></ul><h3>EC2 Implementation</h3><h4>Prerequisites and Planning</h4><p><strong>1. IAM Role Configuration</strong></p><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;ec2:DescribeInstances&quot;,<br>                &quot;ec2:DescribeInstanceStatus&quot;,<br>                &quot;ec2:DescribeVolumes&quot;,<br>                &quot;ec2:DescribeVolumeStatus&quot;,<br>                &quot;ec2:DescribeSnapshots&quot;,<br>                &quot;ec2:DescribeImages&quot;,<br>                &quot;ec2:DescribeSecurityGroups&quot;,<br>                &quot;ec2:DescribeNetworkInterfaces&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;<br>        }<br>    ]<br>}</pre><p><strong>2. Security Group Configuration</strong></p><p>bash</p><pre># Allow New Relic collector access<br># Outbound HTTPS (443) to New Relic collectors<br># Inbound access not required for standard monitoring</pre><h4>Installation Methods</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5TajM1v-9FctulF31KpyHg.png" /></figure><p><strong>Method 1: Automated Installation (Recommended)</strong></p><pre># Download and run installation script<br>curl -Ls https://download.newrelic.com/install/newrelic-cli/scripts/install.sh | bash<br><br># Run guided installation<br>sudo NEW_RELIC_API_KEY=YOUR_API_KEY NEW_RELIC_ACCOUNT_ID=YOUR_ACCOUNT_ID /usr/local/bin/newrelic install</pre><p><strong>Method 2: Manual Package Installation</strong></p><pre># Amazon Linux 2 / CentOS / RHEL<br>curl -o /etc/yum.repos.d/newrelic-infra.repo https://download.newrelic.com/infrastructure_agent/linux/yum/el/7/x86_64/newrelic-infra.repo<br>yum -q makecache -y --disablerepo=&#39;*&#39; --enablerepo=&#39;newrelic-infra&#39;<br>yum install newrelic-infra -y<br><br># Ubuntu / Debian<br>curl -s https://download.newrelic.com/infrastructure_agent/gpg/newrelic-infra.gpg | apt-key add -<br>printf &quot;deb https://download.newrelic.com/infrastructure_agent/linux/apt xenial main&quot; | tee -a /etc/apt/sources.list.d/newrelic-infra.list<br>apt-get update<br>apt-get install newrelic-infra -y</pre><p><strong>Method 3: Container-based Installation</strong></p><pre># Dockerfile for containerized applications<br>FROM ubuntu:20.04<br><br># Install New Relic infrastructure agent<br>RUN curl -s https://download.newrelic.com/infrastructure_agent/gpg/newrelic-infra.gpg | apt-key add - &amp;&amp; \<br>    printf &quot;deb https://download.newrelic.com/infrastructure_agent/linux/apt focal main&quot; | tee -a /etc/apt/sources.list.d/newrelic-infra.list &amp;&amp; \<br>    apt-get update &amp;&amp; \<br>    apt-get install -y newrelic-infra<br><br># Copy configuration<br>COPY newrelic-infra.yml /etc/newrelic-infra.yml<br><br># Your application<br>COPY . /app<br>WORKDIR /app<br><br>CMD [&quot;newrelic-infra&quot;, &quot;-config&quot;, &quot;/etc/newrelic-infra.yml&quot;]</pre><h4>Configuration Management</h4><p><strong>1. Basic Configuration (/etc/newrelic-infra.yml)</strong></p><pre># New Relic Infrastructure Configuration<br>license_key: YOUR_LICENSE_KEY<br><br># Custom attributes for organization<br>custom_attributes:<br>  environment: production<br>  team: platform<br>  cost_center: engineering<br>  application: web-service<br>  region: us-east-1<br><br># Display name in New Relic UI<br>display_name: web-server-prod-01<br><br># Logging configuration<br>log:<br>  level: info<br>  file: /var/log/newrelic-infra/newrelic-infra.log<br>  rotate:<br>    max_size: 100MB<br>    max_files: 5<br><br># Proxy configuration (if needed)<br># proxy: http://proxy.company.com:8080<br><br># Process sampling<br>enable_process_metrics: true<br>process_config:<br>  match:<br>    - name: java<br>      cmdline:<br>        - tomcat<br>    - name: nginx<br>    - name: mysql<br><br># Network monitoring<br>network_interface_filters:<br>  prefix:<br>    - eth<br>    - en<br>    - wlan</pre><p><strong>2. Advanced Configuration with Integrations</strong></p><pre># Integration configurations<br>integrations:<br>  - name: nri-nginx<br>    config:<br>      status_url: http://127.0.0.1/nginx_status<br>      status_module: discover<br>    <br>  - name: nri-mysql<br>    config:<br>      hostname: localhost<br>      port: 3306<br>      username: newrelic<br>      password: ${MYSQL_PASSWORD}<br>      database: performance_schema<br>      <br>  - name: nri-redis<br>    config:<br>      hostname: localhost<br>      port: 6379<br>      password: ${REDIS_PASSWORD}<br><br># Custom metric collection<br>custom_attributes:<br>  datacenter: us-east-1a<br>  instance_type: m5.large<br>  auto_scaling_group: web-servers-asg</pre><h4>Infrastructure as Code (Terraform)</h4><p><strong>1. EC2 Instance with New Relic</strong></p><pre># terraform/main.tf<br>resource &quot;aws_instance&quot; &quot;web_server&quot; {<br>  ami           = data.aws_ami.amazon_linux.id<br>  instance_type = &quot;t3.medium&quot;<br>  <br>  iam_instance_profile = aws_iam_instance_profile.newrelic_profile.name<br>  <br>  user_data = templatefile(&quot;${path.module}/user_data.sh&quot;, {<br>    newrelic_license_key = var.newrelic_license_key<br>    environment         = var.environment<br>    application_name    = var.application_name<br>  })<br>  <br>  tags = {<br>    Name        = &quot;web-server-${var.environment}&quot;<br>    Environment = var.environment<br>    Application = var.application_name<br>    Monitoring  = &quot;newrelic&quot;<br>  }<br>}<br><br># IAM role for New Relic<br>resource &quot;aws_iam_role&quot; &quot;newrelic_role&quot; {<br>  name = &quot;newrelic-infrastructure-role&quot;<br>  <br>  assume_role_policy = jsonencode({<br>    Version = &quot;2012-10-17&quot;<br>    Statement = [<br>      {<br>        Action = &quot;sts:AssumeRole&quot;<br>        Effect = &quot;Allow&quot;<br>        Principal = {<br>          Service = &quot;ec2.amazonaws.com&quot;<br>        }<br>      }<br>    ]<br>  })<br>}<br><br>resource &quot;aws_iam_role_policy_attachment&quot; &quot;newrelic_policy&quot; {<br>  role       = aws_iam_role.newrelic_role.name<br>  policy_arn = &quot;arn:aws:iam::aws:policy/ReadOnlyAccess&quot;<br>}<br><br>resource &quot;aws_iam_instance_profile&quot; &quot;newrelic_profile&quot; {<br>  name = &quot;newrelic-infrastructure-profile&quot;<br>  role = aws_iam_role.newrelic_role.name<br>}</pre><p><strong>2. User Data Script (user_data.sh)</strong>bash</p><pre>#!/bin/bash<br><br># Update system<br>yum update -y<br><br># Install New Relic Infrastructure agent<br>curl -o /etc/yum.repos.d/newrelic-infra.repo https://download.newrelic.com/infrastructure_agent/linux/yum/el/7/x86_64/newrelic-infra.repo<br>yum -q makecache -y --disablerepo=&#39;*&#39; --enablerepo=&#39;newrelic-infra&#39;<br>yum install newrelic-infra -y<br><br># Create configuration file<br>cat &gt; /etc/newrelic-infra.yml &lt;&lt; EOF<br>license_key: ${newrelic_license_key}<br>display_name: ${application_name}-$(ec2-metadata --instance-id | cut -d&#39; &#39; -f2)<br>custom_attributes:<br>  environment: ${environment}<br>  application: ${application_name}<br>  instance_id: $(ec2-metadata --instance-id | cut -d&#39; &#39; -f2)<br>  availability_zone: $(ec2-metadata --availability-zone | cut -d&#39; &#39; -f2)<br>  instance_type: $(ec2-metadata --instance-type | cut -d&#39; &#39; -f2)<br>EOF<br><br># Start and enable New Relic service<br>systemctl start newrelic-infra<br>systemctl enable newrelic-infra<br><br># Install application (example: nginx)<br>yum install nginx -y<br>systemctl start nginx<br>systemctl enable nginx</pre><h3>EC2 Dashboards</h3><h4>Custom Dashboard Creation</h4><p><strong>1. Infrastructure Overview Dashboard</strong></p><pre>{<br>  &quot;name&quot;: &quot;EC2 Infrastructure Overview&quot;,<br>  &quot;description&quot;: &quot;Comprehensive view of EC2 infrastructure metrics&quot;,<br>  &quot;pages&quot;: [<br>    {<br>      &quot;name&quot;: &quot;System Performance&quot;,<br>      &quot;widgets&quot;: [<br>        {<br>          &quot;title&quot;: &quot;CPU Utilization by Instance&quot;,<br>          &quot;visualization&quot;: {<br>            &quot;id&quot;: &quot;viz.line&quot;<br>          },<br>          &quot;rawConfiguration&quot;: {<br>            &quot;nrqlQueries&quot;: [<br>              {<br>                &quot;query&quot;: &quot;SELECT average(cpuPercent) FROM SystemSample WHERE entityName LIKE &#39;web-server%&#39; FACET entityName TIMESERIES&quot;<br>              }<br>            ]<br>          }<br>        },<br>        {<br>          &quot;title&quot;: &quot;Memory Usage&quot;,<br>          &quot;visualization&quot;: {<br>            &quot;id&quot;: &quot;viz.line&quot;<br>          },<br>          &quot;rawConfiguration&quot;: {<br>            &quot;nrqlQueries&quot;: [<br>              {<br>                &quot;query&quot;: &quot;SELECT average(memoryUsedPercent) FROM SystemSample WHERE entityName LIKE &#39;web-server%&#39; FACET entityName TIMESERIES&quot;<br>              }<br>            ]<br>          }<br>        }<br>      ]<br>    }<br>  ]<br>}</pre><p><strong>2. Application Performance Dashboard</strong></p><pre>{<br>  &quot;name&quot;: &quot;Application Performance - EC2&quot;,<br>  &quot;pages&quot;: [<br>    {<br>      &quot;name&quot;: &quot;Response Times and Throughput&quot;,<br>      &quot;widgets&quot;: [<br>        {<br>          &quot;title&quot;: &quot;Average Response Time&quot;,<br>          &quot;visualization&quot;: {<br>            &quot;id&quot;: &quot;viz.billboard&quot;<br>          },<br>          &quot;rawConfiguration&quot;: {<br>            &quot;nrqlQueries&quot;: [<br>              {<br>                &quot;query&quot;: &quot;SELECT average(duration) FROM Transaction WHERE appName = &#39;WebApp&#39; AND transactionType = &#39;Web&#39; SINCE 1 hour ago&quot;<br>              }<br>            ],<br>            &quot;thresholds&quot;: [<br>              {<br>                &quot;value&quot;: 500,<br>                &quot;alertSeverity&quot;: &quot;WARNING&quot;<br>              },<br>              {<br>                &quot;value&quot;: 1000,<br>                &quot;alertSeverity&quot;: &quot;CRITICAL&quot;<br>              }<br>            ]<br>          }<br>        }<br>      ]<br>    }<br>  ]<br>}</pre><h4>Automated Dashboard Deployment</h4><p><strong>1. Terraform Dashboard Configuration</strong></p><pre># terraform/dashboards.tf<br>resource &quot;newrelic_one_dashboard&quot; &quot;ec2_infrastructure&quot; {<br>  name = &quot;EC2 Infrastructure - ${var.environment}&quot;<br><br>  page {<br>    name = &quot;System Metrics&quot;<br><br>    widget_line {<br>      title  = &quot;CPU Utilization&quot;<br>      row    = 1<br>      column = 1<br>      width  = 6<br>      height = 3<br><br>      nrql_query {<br>        query = &quot;SELECT average(cpuPercent) FROM SystemSample WHERE environment = &#39;${var.environment}&#39; FACET entityName TIMESERIES AUTO&quot;<br>      }<br>    }<br><br>    widget_line {<br>      title  = &quot;Memory Usage&quot;<br>      row    = 1<br>      column = 7<br>      width  = 6<br>      height = 3<br><br>      nrql_query {<br>        query = &quot;SELECT average(memoryUsedPercent) FROM SystemSample WHERE environment = &#39;${var.environment}&#39; FACET entityName TIMESERIES AUTO&quot;<br>      }<br>    }<br><br>    widget_table {<br>      title  = &quot;Top Processes by CPU&quot;<br>      row    = 4<br>      column = 1<br>      width  = 12<br>      height = 4<br><br>      nrql_query {<br>        query = &quot;SELECT latest(cpuPercent) AS &#39;CPU %&#39;, latest(memoryResidentSizeBytes)/1000000 AS &#39;Memory (MB)&#39; FROM ProcessSample WHERE environment = &#39;${var.environment}&#39; FACET tuple(entityName, processDisplayName) ORDER BY cpuPercent DESC LIMIT 20&quot;<br>      }<br>    }<br>  }<br><br>  page {<br>    name = &quot;Network and Storage&quot;<br><br>    widget_line {<br>      title  = &quot;Network I/O&quot;<br>      row    = 1<br>      column = 1<br>      width  = 6<br>      height = 3<br><br>      nrql_query {<br>        query = &quot;SELECT average(networkReceiveBytesPerSecond), average(networkTransmitBytesPerSecond) FROM NetworkSample WHERE environment = &#39;${var.environment}&#39; TIMESERIES AUTO&quot;<br>      }<br>    }<br><br>    widget_line {<br>      title  = &quot;Disk Utilization&quot;<br>      row    = 1<br>      column = 7<br>      width  = 6<br>      height = 3<br><br>      nrql_query {<br>        query = &quot;SELECT average(diskUsedPercent) FROM StorageSample WHERE environment = &#39;${var.environment}&#39; FACET entityName, mountPoint TIMESERIES AUTO&quot;<br>      }<br>    }<br>  }<br>}</pre><p>Continue reading <a href="#part-4-eks-kubernetes-integration">Part 4: EKS Integration</a> for container monitoring…</p><p>This is Part 1–3 of the New Relic AWS Integration Series. The complete series covers Lambda, EC2, EKS, additional AWS services, and best practices for enterprise-scale deployments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d8c4223bfbc2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LoRA vs QLoRA: A Comprehensive Comparison]]></title>
            <link>https://medium.com/@to.adarshsingh/lora-vs-qlora-a-comprehensive-comparison-ff12117d669b?source=rss-73f6989daa12------2</link>
            <guid isPermaLink="false">https://medium.com/p/ff12117d669b</guid>
            <dc:creator><![CDATA[Adarsh Singh Dikhit]]></dc:creator>
            <pubDate>Fri, 01 Aug 2025 07:13:38 GMT</pubDate>
            <atom:updated>2025-08-01T07:13:38.263Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Low-Rank Adaptation (LoRA) and Quantized Low-Rank Adaptation (QLoRA) are two groundbreaking techniques in the field of large language model fine-tuning. Both methods address the challenge of efficiently adapting pre-trained models to specific tasks without the computational overhead of full parameter fine-tuning. This article provides an in-depth comparison of these two approaches, examining their methodologies, advantages, limitations, and practical applications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/561/1*7X4K7gG1FfzLUhdenFV--g.png" /></figure><h3>1. Core Methodology</h3><h3>LoRA (Low-Rank Adaptation)</h3><p>LoRA operates on the principle that the weight updates during fine-tuning have a low intrinsic rank. Instead of updating all parameters in a neural network layer, LoRA decomposes the weight update matrix into two smaller matrices: A and B. For a pre-trained weight matrix W₀, the adapted weight becomes W₀ + BA, where B ∈ ℝᵈˣʳ and A ∈ ℝʳˣᵏ, with r being the rank (typically much smaller than d and k).</p><p>This decomposition dramatically reduces the number of trainable parameters. For example, if the original weight matrix is 4096×4096, a full update would require 16.7 million parameters. With LoRA using rank r=16, only 131,072 parameters need to be trained (4096×16 + 16×4096).</p><h3>QLoRA (Quantized Low-Rank Adaptation)</h3><p>QLoRA builds upon LoRA by introducing quantization to further reduce memory requirements. The key innovation is storing the pre-trained model weights in 4-bit precision while performing LoRA fine-tuning. QLoRA uses a novel 4-bit NormalFloat (NF4) data type that is information-theoretically optimal for normally distributed weights.</p><p>The process involves three main components: 4-bit quantization of the frozen pre-trained model, Low-Rank Adapters for trainable parameters, and double quantization to further compress the quantization constants. This allows QLoRA to fine-tune models that would otherwise be impossible to fit in memory.</p><h3>2. Memory Efficiency</h3><h3>LoRA Memory Usage</h3><p>LoRA significantly reduces memory requirements compared to full fine-tuning, but the base model still requires substantial memory. For a 7B parameter model, LoRA typically requires around 24–28 GB of GPU memory, as the full precision weights of the base model must be loaded. The memory savings come primarily from the reduced number of trainable parameters and smaller optimizer states.</p><p>The memory footprint consists of the base model weights (unchanged), the low-rank adaptation matrices (small), gradients for the adaptation matrices only, and optimizer states for the reduced parameter set. This results in memory savings of 2–3x compared to full fine-tuning.</p><h3>QLoRA Memory Usage</h3><p>QLoRA achieves remarkable memory efficiency by quantizing the base model to 4-bit precision. The same 7B parameter model that requires 24–28 GB with LoRA can be fine-tuned with QLoRA using only 9–12 GB of GPU memory. This 65% reduction in memory usage is achieved through the 4-bit storage of frozen parameters.</p><p>The memory breakdown includes 4-bit quantized base model weights, 16-bit Low-Rank Adapter weights, quantization constants (also quantized), and optimizer states for adapters only. This dramatic reduction enables fine-tuning of larger models on consumer hardware.</p><h3>3. Computational Overhead</h3><h3>LoRA Computational Cost</h3><p>LoRA introduces minimal computational overhead during training and inference. The forward pass requires computing the low-rank adaptation (BA) and adding it to the original weights. This operation is computationally inexpensive due to the low rank constraint. The training speed is typically 90–95% of the base model’s training speed.</p><p>During inference, the adapted weights can be merged with the original weights (W₀ + BA), eliminating any computational overhead. Alternatively, the low-rank components can be kept separate, adding only a small matrix multiplication cost.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yKIx6ipZ3_9c0DDXRul8Jw.png" /></figure><h3>QLoRA Computational Cost</h3><p>QLoRA has slightly higher computational overhead due to the quantization and dequantization operations. During forward passes, 4-bit weights must be dequantized to 16-bit for computation, then the results are processed through the LoRA adapters. This process adds approximately 5–10% computational overhead compared to standard LoRA.</p><p>However, the dequantization is performed on-the-fly and only for the required computations, making it more efficient than storing full-precision weights. The backward pass computational cost remains similar to LoRA since gradients are only computed for the adapter parameters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G5rVIAxJaDIpvksyBstTfA.png" /></figure><h3>4. Model Performance and Quality</h3><h3>LoRA Performance Characteristics</h3><p>LoRA typically achieves 95–99% of full fine-tuning performance across various tasks while using only a fraction of the parameters. The performance depends heavily on the rank parameter (r) and which layers are adapted. Higher ranks generally provide better performance but with increased parameter count.</p><p>Research shows that LoRA works particularly well for tasks requiring specific domain knowledge or format adaptation. The method excels in instruction tuning, domain adaptation, and task-specific fine-tuning scenarios. Performance degradation is minimal for most natural language processing tasks.</p><h3>QLoRA Performance Characteristics</h3><p>QLoRA achieves comparable performance to LoRA while using significantly less memory. Extensive evaluations show that QLoRA can match LoRA’s performance on various benchmarks, including instruction following, conversational AI, and domain-specific tasks. The 4-bit quantization has minimal impact on the final model quality.</p><p>The key insight is that the quantization affects only the frozen pre-trained weights, while the trainable LoRA parameters remain in higher precision. This design choice preserves the learning capacity while reducing memory footprint.</p><h3>5. Hardware Requirements</h3><h3>LoRA Hardware Needs</h3><p>LoRA requires high-end GPUs with substantial memory for larger models. Fine-tuning a 7B model typically needs at least 24 GB of GPU memory, making it accessible primarily on A100, V100, or similar enterprise-grade hardware. For 13B models, multiple GPUs or 40GB+ memory cards are necessary.</p><p>The method is more accessible than full fine-tuning but still has significant hardware barriers for consumer users. Cloud computing or high-end workstations are typically required for practical use with larger models.</p><h3>QLoRA Hardware Accessibility</h3><p>QLoRA democratizes large model fine-tuning by making it possible on consumer hardware. A 7B model can be fine-tuned on a single RTX 3090 or RTX 4090 with 24GB memory. Even 13B models become accessible on high-end consumer GPUs or modest cloud instances.</p><p>This accessibility has significant implications for research and development, allowing smaller teams and individual researchers to experiment with large model fine-tuning without substantial computational resources.</p><h3>6. Training Speed and Convergence</h3><h3>LoRA Training Dynamics</h3><p>LoRA generally converges faster than full fine-tuning due to the reduced parameter space and focused updates. The low-rank constraint acts as a form of regularization, often leading to more stable training. Typical training times are 2–3x faster than full fine-tuning for the same number of steps.</p><p>The convergence characteristics depend on the rank selection and layer targeting strategy. Lower ranks converge faster but may have limited representational capacity, while higher ranks provide more flexibility at the cost of training time.</p><h3>QLoRA Training Dynamics</h3><p>QLoRA has similar convergence properties to LoRA, with the quantization having minimal impact on training dynamics. The training speed is slightly slower than LoRA due to quantization overhead, but the difference is typically marginal (5–10% slower).</p><p>The reduced memory pressure often allows for larger batch sizes, which can partially offset the computational overhead and lead to more stable gradient estimates during training.</p><h3>7. Flexibility and Customization</h3><h3>LoRA Flexibility</h3><p>LoRA offers excellent flexibility in terms of which layers to adapt and the rank selection for each layer. Different layers can have different ranks based on their importance for the target task. The method supports various adaptation strategies, including attention-only adaptation, full linear layer adaptation, or selective layer targeting.</p><p>The modular nature of LoRA allows for easy experimentation with different configurations and the ability to combine multiple LoRA adapters for different tasks.</p><h3>QLoRA Flexibility</h3><p>QLoRA inherits all the flexibility of LoRA while adding quantization options. Users can choose different quantization schemes, adjust the block size for quantization, and even use different precision levels for different parts of the model. The method supports the same layer selection and rank configuration options as LoRA.</p><p>The additional quantization parameters provide another dimension for optimization, though they require more expertise to tune effectively.</p><h3>8. Deployment and Inference</h3><h3>LoRA Deployment</h3><p>LoRA adapters are lightweight and easy to deploy. The adapter weights can be stored separately and loaded on top of the base model, enabling multiple task-specific adapters for a single base model. The adapters can also be merged with the base model for inference without overhead.</p><p>Deployment flexibility includes the ability to switch between different adapters dynamically and share base models across multiple applications with different adapter sets.</p><h3>QLoRA Deployment</h3><p>QLoRA deployment is slightly more complex due to the quantization requirements. The quantized base model and adapters must be properly configured for inference. However, modern frameworks provide good support for QLoRA deployment, making it increasingly straightforward.</p><p>The memory efficiency benefits extend to deployment, allowing larger models to be served on more modest hardware configurations.</p><h3>9. Use Cases and Applications</h3><h3>LoRA Applications</h3><p>LoRA is ideal for scenarios where memory is available but training efficiency is crucial. Common applications include research environments with access to high-end hardware, enterprise applications requiring quick adaptation to new domains, and scenarios where multiple model variants need to be maintained efficiently.</p><p>The method excels in instruction tuning, domain adaptation, style transfer, and task-specific fine-tuning where the base model capabilities need focused enhancement.</p><h3>QLoRA Applications</h3><p>QLoRA is particularly valuable for resource-constrained environments, educational settings, small research teams, and scenarios where cost efficiency is paramount. It enables experimentation and development on consumer hardware that would otherwise be impossible.</p><p>The democratization aspect makes QLoRA excellent for prototyping, proof-of-concept development, and scenarios where the slight performance trade-off is acceptable for the significant resource savings.</p><h3>10. Limitations and Considerations</h3><h3>LoRA Limitations</h3><p>LoRA’s primary limitation is the memory requirement for the base model, which still necessitates substantial hardware resources. The low-rank assumption may not hold for all types of adaptations, potentially limiting performance in certain scenarios. The method also requires careful tuning of rank parameters and layer selection strategies.</p><p>Performance can vary significantly based on the target task and how well it aligns with the low-rank adaptation paradigm. Some complex adaptations may require higher ranks, reducing the efficiency benefits.</p><h3>QLoRA Limitations</h3><p>QLoRA introduces quantization artifacts that may affect model quality in certain edge cases. The 4-bit quantization, while generally effective, may not be suitable for all model types or tasks requiring extreme precision. The additional complexity of quantization parameters requires more expertise to optimize effectively.</p><p>The method also has dependencies on specific quantization implementations, which may limit portability across different frameworks or hardware configurations.</p><h3>Conclusion</h3><p>Both LoRA and QLoRA represent significant advances in efficient fine-tuning techniques, each with distinct advantages and use cases. LoRA provides an excellent balance of efficiency and performance for users with adequate hardware resources, while QLoRA democratizes large model fine-tuning by dramatically reducing memory requirements.</p><p>The choice between LoRA and QLoRA primarily depends on available hardware resources, performance requirements, and deployment constraints. QLoRA’s memory efficiency makes it the preferred choice for resource-constrained scenarios, while LoRA may be favored when maximum performance is crucial and hardware resources are available.</p><p>As the field continues to evolve, both techniques are likely to see further improvements and optimizations, making efficient fine-tuning of large language models increasingly accessible to a broader range of users and applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ff12117d669b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RATH: Next-Generation Automated Data Analysis Tool — Architecture and AWS Deployment with Helm]]></title>
            <link>https://medium.com/@to.adarshsingh/rath-next-generation-automated-data-analysis-tool-architecture-and-aws-deployment-with-helm-a05c00909fd6?source=rss-73f6989daa12------2</link>
            <guid isPermaLink="false">https://medium.com/p/a05c00909fd6</guid>
            <dc:creator><![CDATA[Adarsh Singh Dikhit]]></dc:creator>
            <pubDate>Sun, 08 Jun 2025 20:12:11 GMT</pubDate>
            <atom:updated>2025-06-08T20:12:11.159Z</atom:updated>
            <content:encoded><![CDATA[<h3>RATH: Next-Generation Automated Data Analysis Tool — Architecture and AWS Deployment with Helm</h3><h3>Introduction</h3><p>RATH is an automated data analysis and visualization tool (auto-EDA) that is a free and open-source software licensed under the AGPL. It enables you to streamline your data analysis workflow and effortlessly discover insights from complex data sources, creating highly-customizable multi-dimensional Data Visualizations. RATH revolutionizes the exploratory data analysis workflow by leveraging its augmented analytic engine to automatically uncover patterns, insights, and causal relationships.</p><p>This comprehensive guide explores RATH’s architecture, capabilities, and provides detailed instructions for deploying RATH on AWS using Helm charts for scalable enterprise implementations.</p><h3>Key Features of RATH</h3><p><strong>Automated Exploratory Data Analysis (EDA)</strong>: RATH automatically analyzes your data to discover patterns, correlations, and anomalies without manual intervention, significantly reducing the time required for initial data exploration.</p><p><strong>AI-Powered Insights</strong>: The platform leverages artificial intelligence to identify meaningful relationships in data and suggest relevant visualizations based on data characteristics and patterns.</p><p><strong>Causal Analysis</strong>: Unlike traditional visualization tools, RATH includes causal inference capabilities that help identify cause-and-effect relationships in your data, providing deeper analytical insights.</p><p><strong>Multi-dimensional Visualization</strong>: Create complex, interactive visualizations that can handle multiple dimensions of data simultaneously, enabling comprehensive data exploration.</p><p><strong>Open Source Flexibility</strong>: Being open-source, RATH offers complete customization capabilities and can be modified to meet specific organizational requirements.</p><p><strong>Self-hosted Deployment</strong>: Organizations can deploy RATH on their own infrastructure, ensuring complete control over data privacy and security.</p><h3>RATH Architecture Overview</h3><p>Understanding RATH’s architecture is essential for successful deployment and optimization. The architecture consists of several interconnected components that work together to provide automated data analysis capabilities.</p><h3>Core Architecture Components</h3><p><strong>Frontend Layer</strong>: The frontend is built using modern web technologies, providing an intuitive user interface for data upload, visualization creation, and insight exploration. It includes React-based components for interactive dashboards and real-time data exploration.</p><p><strong>Analytics Engine</strong>: The core analytics engine is responsible for automated pattern discovery, statistical analysis, and causal inference. It includes machine learning algorithms for anomaly detection, clustering, and predictive analytics.</p><p><strong>Data Processing Layer</strong>: This layer handles data ingestion, cleaning, and transformation. It supports various data formats including CSV, JSON, databases, and streaming data sources.</p><p><strong>Visualization Engine</strong>: The visualization engine automatically generates appropriate charts and graphs based on data characteristics. It includes support for statistical plots, geographic visualizations, and interactive dashboards.</p><p><strong>API Layer</strong>: RESTful APIs enable integration with external systems and allow programmatic access to RATH’s capabilities for automated workflows and custom applications.</p><p><strong>Storage Layer</strong>: The storage layer manages data persistence, caching, and metadata storage. It supports various backend storage options including file systems, databases, and cloud storage services.</p><h3>Technical Architecture</h3><p><strong>Microservices Design</strong>: RATH follows a microservices architecture pattern, with different components running as independent services that communicate through well-defined APIs.</p><p><strong>Container-Native</strong>: The application is designed to run in containerized environments, making it ideal for Kubernetes deployments with benefits like scalability, portability, and easy management.</p><p><strong>Event-Driven Processing</strong>: The system uses event-driven architecture for real-time data processing and analysis, enabling responsive user interactions and efficient resource utilization.</p><p><strong>Scalable Computing</strong>: The analytics engine can scale horizontally to handle large datasets and complex analyses by distributing computational tasks across multiple instances.</p><h3>Implementation Strategy</h3><p>Successful RATH implementation requires careful planning considering organizational needs, data sources, infrastructure requirements, and user adoption strategies.</p><h3>Pre-Implementation Assessment</h3><p><strong>Data Landscape Analysis</strong>: Conduct a comprehensive inventory of data sources, formats, and volumes to understand integration requirements and potential challenges.</p><p><strong>Infrastructure Evaluation</strong>: Assess current infrastructure capabilities, security requirements, and scalability needs to determine the optimal deployment architecture.</p><p><strong>User Requirements Gathering</strong>: Identify different user personas, their analytical needs, and skill levels to plan appropriate training and onboarding strategies.</p><p><strong>Performance Requirements</strong>: Define performance expectations, concurrent user limits, and data processing requirements to size the infrastructure appropriately.</p><h3>Deployment Planning</h3><p><strong>Environment Strategy</strong>: Plan for development, staging, and production environments with appropriate data security and access controls for each environment.</p><p><strong>Data Integration</strong>: Design data pipelines and integration patterns for connecting RATH to existing data sources while maintaining data quality and security.</p><p><strong>Security Framework</strong>: Implement comprehensive security measures including authentication, authorization, data encryption, and audit logging.</p><p><strong>Backup and Recovery</strong>: Establish backup procedures for configurations, user data, and analytical results to ensure business continuity.</p><h3>Deploying RATH on AWS with Helm</h3><p>Deploying RATH on AWS using Helm charts provides several advantages including automated deployments, version control, easy rollbacks, and environment consistency. This section provides detailed instructions for a production-ready deployment.</p><h3>Prerequisites</h3><p>Before deploying RATH on AWS, ensure the following prerequisites are met:</p><p><strong>AWS Account</strong>: An active AWS account with appropriate IAM permissions for creating and managing EKS clusters, RDS instances, and other required services.</p><p><strong>Kubernetes Cluster</strong>: A running Amazon EKS cluster with sufficient compute resources. Recommended minimum configuration includes 3 worker nodes with 4 vCPU and 16GB RAM each.</p><p><strong>Helm Installation</strong>: Helm 3.x installed and configured to work with your Kubernetes cluster, including proper RBAC permissions.</p><p><strong>Container Registry</strong>: Access to a container registry (Amazon ECR recommended) for storing RATH container images.</p><p><strong>SSL Certificates</strong>: Valid SSL certificates for secure HTTPS access, either through AWS Certificate Manager or manually generated certificates.</p><p><strong>Domain Name</strong>: A registered domain name with DNS configured through Route 53 or your preferred DNS provider.</p><h3>AWS Infrastructure Setup</h3><p><strong>Amazon EKS Cluster Configuration</strong>: Create an EKS cluster with managed node groups optimized for data processing workloads. Configure nodes with instance types like m5.xlarge or c5.2xlarge for balanced compute and memory resources.</p><p><strong>Networking Setup</strong>: Configure VPC with public and private subnets across multiple availability zones. Set up security groups with appropriate inbound and outbound rules for web traffic and inter-service communication.</p><p><strong>Database Services</strong>: Set up Amazon RDS PostgreSQL instance for metadata storage with automated backups, multi-AZ deployment for high availability, and appropriate instance sizing based on expected usage.</p><p><strong>Storage Configuration</strong>: Configure Amazon EFS for shared storage across pods, including automated backups and appropriate performance modes based on workload requirements.</p><p><strong>Load Balancer Setup</strong>: Deploy Application Load Balancer with SSL termination, health checks, and integration with AWS Certificate Manager for automated certificate management.</p><h3>Helm Chart Structure and Configuration</h3><p>A comprehensive Helm chart for RATH includes several components and configuration files organized in a standard structure:</p><p><strong>Chart Metadata</strong>: The Chart.yaml file contains essential information about the chart including version, dependencies, and maintainer information.</p><p><strong>Default Values</strong>: The values.yaml file provides default configuration values that can be customized for different environments and deployment scenarios.</p><p><strong>Templates Directory</strong>: Contains Kubernetes manifest templates for deployments, services, ingress controllers, persistent volume claims, and configuration maps.</p><p><strong>Helper Templates</strong>: Includes utility templates for generating labels, names, and other common configurations to ensure consistency across resources.</p><h3>Sample Helm Configuration</h3><p>Here’s a comprehensive configuration example for deploying RATH:</p><pre># RATH Helm Values Configuration<br>global:<br>  domain: rath.company.com<br>  environment: production<br>  <br>replicaCount: 3<br><br>image:<br>  repository: kanaries/rath<br>  tag: &quot;latest&quot;<br>  pullPolicy: IfNotPresent<br><br>service:<br>  type: ClusterIP<br>  port: 3000<br>  targetPort: 3000<br><br>ingress:<br>  enabled: true<br>  className: &quot;alb&quot;<br>  annotations:<br>    kubernetes.io/ingress.class: alb<br>    alb.ingress.kubernetes.io/scheme: internet-facing<br>    alb.ingress.kubernetes.io/target-type: ip<br>    alb.ingress.kubernetes.io/ssl-redirect: &quot;443&quot;<br>    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:region:account:certificate/cert-id<br>  hosts:<br>    - host: rath.company.com<br>      paths:<br>        - path: /<br>          pathType: Prefix<br><br>database:<br>  type: postgresql<br>  host: rath-db.cluster-xyz.us-west-2.rds.amazonaws.com<br>  port: 5432<br>  name: rath_production<br>  user: rath_admin<br>  passwordSecret: rath-db-secret<br><br>storage:<br>  type: efs<br>  storageClass: efs-sc<br>  size: 100Gi<br>  accessMode: ReadWriteMany<br><br>resources:<br>  limits:<br>    cpu: 2000m<br>    memory: 4Gi<br>  requests:<br>    cpu: 1000m<br>    memory: 2Gi<br><br>autoscaling:<br>  enabled: true<br>  minReplicas: 2<br>  maxReplicas: 10<br>  targetCPUUtilizationPercentage: 70<br><br>nodeSelector:<br>  workload-type: analytics<br><br>securityContext:<br>  runAsNonRoot: true<br>  runAsUser: 1001<br>  fsGroup: 1001</pre><h3>Deployment Process</h3><p><strong>Environment Preparation</strong>: Create the Kubernetes namespace and apply necessary RBAC configurations for the RATH application and its components.</p><p><strong>Secret Management</strong>: Create Kubernetes secrets for database credentials, API keys, and other sensitive configuration data using kubectl or external secret management tools.</p><p><strong>Helm Chart Installation</strong>: Deploy RATH using Helm with environment-specific value overrides, ensuring proper validation and testing of configurations.</p><p><strong>Post-Deployment Configuration</strong>: Configure ingress rules, set up monitoring and logging, and perform initial application configuration through the web interface.</p><p><strong>Health Verification</strong>: Verify all components are running correctly, test external connectivity, and validate integration with data sources.</p><h3>Advanced Configuration Options</h3><p><strong>Data Source Integration</strong>: Configure connections to various data sources including databases, file systems, and APIs with appropriate authentication and security settings.</p><p><strong>Performance Tuning</strong>: Optimize resource allocation, configure caching strategies, and tune garbage collection settings for optimal performance.</p><p><strong>Security Hardening</strong>: Implement network policies, pod security policies, and service mesh integration for enhanced security.</p><p><strong>Monitoring Integration</strong>: Set up comprehensive monitoring using Prometheus, Grafana, and AWS CloudWatch for operational visibility.</p><h3>Monitoring and Operations</h3><p>Effective monitoring and operational procedures are crucial for maintaining optimal RATH performance and reliability in production environments.</p><h3>Monitoring Strategy</h3><p><strong>Application Metrics</strong>: Monitor key application metrics including response times, error rates, concurrent users, and data processing throughput.</p><p><strong>Infrastructure Metrics</strong>: Track resource utilization, network performance, storage capacity, and database performance metrics.</p><p><strong>Business Metrics</strong>: Monitor user engagement, report generation frequency, and data source connection health.</p><p><strong>Alert Configuration</strong>: Set up alerts for critical issues including service downtime, resource exhaustion, and security incidents.</p><h3>Operational Procedures</h3><p><strong>Regular Maintenance</strong>: Implement scheduled maintenance windows for updates, backups, and system optimization.</p><p><strong>Backup Procedures</strong>: Establish automated backup procedures for user data, configurations, and analytical results with tested recovery procedures.</p><p><strong>Update Management</strong>: Plan and execute regular updates for RATH software, dependencies, and underlying infrastructure components.</p><p><strong>Performance Optimization</strong>: Regularly review performance metrics and optimize configurations based on usage patterns and requirements.</p><h3>Troubleshooting Common Issues</h3><p><strong>Performance Issues</strong>: Address slow query performance through database optimization, resource scaling, and caching improvements.</p><p><strong>Connectivity Problems</strong>: Resolve data source connectivity issues through network configuration review and credential verification.</p><p><strong>Resource Constraints</strong>: Handle resource limitations through horizontal scaling, resource limit adjustments, and workload optimization.</p><p><strong>Data Quality Issues</strong>: Implement data validation procedures and error handling mechanisms for improved data quality.</p><h3>Security Considerations</h3><p>Security is paramount when deploying RATH in enterprise environments, especially when handling sensitive business data.</p><h3>Authentication and Authorization</h3><p><strong>Identity Provider Integration</strong>: Integrate with existing identity providers such as Active Directory, LDAP, or OAuth providers for seamless user authentication.</p><p><strong>Role-Based Access Control</strong>: Implement granular permissions based on user roles and responsibilities, ensuring users access only necessary data and features.</p><p><strong>API Security</strong>: Secure API endpoints with proper authentication tokens, rate limiting, and input validation to prevent unauthorized access.</p><p><strong>Session Management</strong>: Configure secure session management with appropriate timeout settings and secure cookie configurations.</p><h3>Data Protection</h3><p><strong>Encryption in Transit</strong>: Ensure all data communication is encrypted using TLS/SSL protocols with strong cipher suites.</p><p><strong>Encryption at Rest</strong>: Implement encryption for stored data including databases, file systems, and backup storage.</p><p><strong>Data Masking</strong>: Apply data masking techniques for sensitive information in non-production environments.</p><p><strong>Audit Logging</strong>: Maintain comprehensive audit logs for all user activities, data access, and system changes.</p><h3>Network Security</h3><p><strong>Network Segmentation</strong>: Implement proper network segmentation using VPCs, subnets, and security groups to isolate RATH components.</p><p><strong>Firewall Configuration</strong>: Configure firewalls and security groups to allow only necessary traffic between components.</p><p><strong>VPN Integration</strong>: Set up VPN connectivity for secure remote access to RATH administration interfaces.</p><p><strong>DDoS Protection</strong>: Implement DDoS protection using AWS Shield and WAF for internet-facing deployments.</p><h3>Integration Capabilities</h3><p>RATH’s integration capabilities enable it to work seamlessly within existing enterprise data ecosystems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oyM4kWv-2f1sbWBsTuLUmw.png" /></figure><h3>Data Source Connectors</h3><p><strong>Database Integration</strong>: Connect to various databases including PostgreSQL, MySQL, SQL Server, Oracle, and NoSQL databases like MongoDB and Elasticsearch.</p><p><strong>File System Integration</strong>: Process data from various file formats including CSV, JSON, Parquet, and Excel files stored in local or cloud storage systems.</p><p><strong>API Integration</strong>: Integrate with REST APIs and web services to access real-time data from external systems and applications.</p><p><strong>Streaming Data</strong>: Process streaming data from sources like Kafka, Kinesis, and other event streaming platforms for real-time analytics.</p><h3>Third-Party Tool Integration</h3><p><strong>Business Intelligence Tools</strong>: Integrate with existing BI tools and data warehouses to extend analytical capabilities.</p><p><strong>Data Pipeline Tools</strong>: Work with ETL/ELT tools like Apache Airflow, AWS Glue, and custom data processing pipelines.</p><p><strong>Collaboration Platforms</strong>: Integrate with collaboration tools like Slack, Microsoft Teams, and email systems for sharing insights and notifications.</p><p><strong>Export Capabilities</strong>: Export analysis results and visualizations to various formats for use in presentations, reports, and other applications.</p><h3>Best Practices and Recommendations</h3><h3>Development and Deployment Best Practices</h3><p><strong>Version Control</strong>: Maintain all configuration files, Helm charts, and custom code in version control systems with proper branching strategies.</p><p><strong>Environment Consistency</strong>: Use Infrastructure as Code (IaC) tools like Terraform to ensure consistent environments across development, staging, and production.</p><p><strong>Automated Testing</strong>: Implement automated testing for configurations, deployments, and application functionality to catch issues early.</p><p><strong>Documentation</strong>: Maintain comprehensive documentation for deployment procedures, configuration options, and operational procedures.</p><h3>Performance Optimization</h3><p><strong>Resource Planning</strong>: Right-size resources based on actual usage patterns and growth projections rather than over-provisioning.</p><p><strong>Caching Strategies</strong>: Implement appropriate caching at multiple levels including application cache, database query cache, and CDN for static assets.</p><p><strong>Database Optimization</strong>: Optimize database queries, implement proper indexing, and consider read replicas for improved performance.</p><p><strong>Monitoring-Driven Optimization</strong>: Use monitoring data to identify bottlenecks and optimize system performance continuously.</p><h3>Operational Excellence</h3><p><strong>Disaster Recovery</strong>: Implement comprehensive disaster recovery procedures with regular testing and documentation.</p><p><strong>Change Management</strong>: Establish proper change management procedures for configuration changes, updates, and maintenance activities.</p><p><strong>Capacity Planning</strong>: Monitor growth trends and plan for capacity increases before resource constraints impact performance.</p><p><strong>Security Reviews</strong>: Conduct regular security reviews and penetration testing to identify and address potential vulnerabilities.</p><h3>Cost Optimization</h3><p>Optimizing costs while maintaining performance and reliability is important for sustainable RATH deployments.</p><h3>Infrastructure Cost Optimization</h3><p><strong>Right-Sizing</strong>: Regularly review and adjust instance sizes based on actual resource utilization patterns.</p><p><strong>Reserved Instances</strong>: Use AWS Reserved Instances for predictable workloads to achieve significant cost savings.</p><p><strong>Spot Instances</strong>: Consider using Spot Instances for non-critical workloads and development environments.</p><p><strong>Auto-Scaling</strong>: Implement auto-scaling to automatically adjust resources based on demand, reducing costs during low-usage periods.</p><h3>Storage Cost Optimization</h3><p><strong>Data Lifecycle Management</strong>: Implement data lifecycle policies to automatically move older data to cheaper storage tiers.</p><p><strong>Compression</strong>: Use data compression techniques to reduce storage requirements and associated costs.</p><p><strong>Backup Optimization</strong>: Optimize backup retention policies and use incremental backups to reduce storage costs.</p><h3>Operational Cost Optimization</h3><p><strong>Automation</strong>: Automate routine tasks to reduce operational overhead and human error.</p><p><strong>Monitoring</strong>: Implement cost monitoring and alerting to track spending and identify cost optimization opportunities.</p><p><strong>Resource Tagging</strong>: Use comprehensive resource tagging for better cost allocation and tracking across different projects and departments.</p><h3>Future Considerations</h3><p>As RATH continues to evolve, several trends and developments may impact deployment strategies and capabilities.</p><h3>Technology Trends</h3><p><strong>AI/ML Integration</strong>: Enhanced integration with machine learning platforms and automated model deployment capabilities.</p><p><strong>Cloud-Native Features</strong>: Improved cloud-native capabilities including better integration with managed services and serverless computing.</p><p><strong>Real-Time Analytics</strong>: Enhanced real-time data processing capabilities for immediate insights and decision-making.</p><p><strong>Collaboration Features</strong>: Improved collaboration features for team-based analytics and knowledge sharing.</p><h3>Scalability Enhancements</h3><p><strong>Distributed Computing</strong>: Enhanced support for distributed computing frameworks for processing extremely large datasets.</p><p><strong>Edge Computing</strong>: Integration with edge computing platforms for analytics closer to data sources.</p><p><strong>Multi-Cloud Support</strong>: Improved support for multi-cloud deployments and hybrid cloud architectures.</p><h3>Conclusion</h3><p>RATH represents a significant advancement in automated data analysis and visualization, offering powerful capabilities that can transform how organizations approach exploratory data analysis. Its open-source nature, combined with AI-powered automation and comprehensive visualization capabilities, makes it an attractive alternative to traditional business intelligence tools.</p><p>Deploying RATH on AWS using Helm charts provides organizations with a scalable, maintainable, and cost-effective solution for enterprise analytics. The combination of AWS’s robust infrastructure services with Kubernetes orchestration creates a platform that can grow with organizational needs while maintaining high availability and performance.</p><p>The success of a RATH deployment depends on careful planning, proper architecture design, robust security implementation, and ongoing optimization. Organizations that invest in proper implementation and follow best practices will benefit from improved analytical capabilities, faster time-to-insight, and more data-driven decision-making processes.</p><p>As the data analytics landscape continues to evolve, tools like RATH that combine automation, AI, and open-source flexibility will become increasingly important for organizations seeking to maximize the value of their data assets. The investment in implementing RATH properly will pay dividends through improved analytical capabilities and more informed business decisions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a05c00909fd6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Vector Database Design for RAG System]]></title>
            <link>https://medium.com/@to.adarshsingh/vector-database-design-for-rag-system-894cfe30e1f0?source=rss-73f6989daa12------2</link>
            <guid isPermaLink="false">https://medium.com/p/894cfe30e1f0</guid>
            <dc:creator><![CDATA[Adarsh Singh Dikhit]]></dc:creator>
            <pubDate>Wed, 28 May 2025 18:24:32 GMT</pubDate>
            <atom:updated>2025-05-28T18:24:32.644Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*48IrzURk32nuHe_01sMfpA.jpeg" /></figure><h3>1. Architecture Overview</h3><h3>Core Components</h3><ul><li><strong>Vector Storage Engine</strong>: High-performance storage for embeddings</li><li><strong>Indexing Layer</strong>: ANN (Approximate Nearest Neighbor) indices for fast retrieval</li><li><strong>Metadata Management</strong>: Document metadata and filtering capabilities</li><li><strong>Query Processing Engine</strong>: Handles similarity search and hybrid queries</li><li><strong>API Layer</strong>: RESTful/gRPC interfaces for integration</li></ul><h3>2. Data Model</h3><h3>Document Structure</h3><pre>{<br>  &quot;id&quot;: &quot;doc_uuid_12345&quot;,<br>  &quot;vector&quot;: [0.1, -0.3, 0.8, ...], // 768/1536 dimensional embedding<br>  &quot;metadata&quot;: {<br>    &quot;source&quot;: &quot;document.pdf&quot;,<br>    &quot;chunk_id&quot;: &quot;chunk_001&quot;,<br>    &quot;page_number&quot;: 5,<br>    &quot;section&quot;: &quot;Introduction&quot;,<br>    &quot;timestamp&quot;: &quot;2024-01-15T10:30:00Z&quot;,<br>    &quot;content_type&quot;: &quot;text&quot;,<br>    &quot;language&quot;: &quot;en&quot;,<br>    &quot;author&quot;: &quot;John Doe&quot;,<br>    &quot;tags&quot;: [&quot;research&quot;, &quot;AI&quot;, &quot;ML&quot;]<br>  },<br>  &quot;content&quot;: &quot;Original text content for this chunk...&quot;,<br>  &quot;content_hash&quot;: &quot;sha256_hash_value&quot;<br>}</pre><h3>Index Configuration</h3><pre>indices:<br>  primary_vector_index:<br>    type: &quot;HNSW&quot;  # Hierarchical Navigable Small World<br>    dimensions: 1536<br>    metric: &quot;cosine&quot;<br>    ef_construction: 200<br>    m: 16<br>  <br>  metadata_indices:<br>    - field: &quot;source&quot;<br>      type: &quot;btree&quot;<br>    - field: &quot;timestamp&quot; <br>      type: &quot;btree&quot;<br>    - field: &quot;tags&quot;<br>      type: &quot;gin&quot;  # Generalized Inverted Index</pre><h3>3. Storage Layer Design</h3><h3>Partitioning Strategy</h3><ul><li><strong>Horizontal Partitioning</strong>: By timestamp/date ranges</li><li><strong>Vector Clustering</strong>: Group similar vectors for locality</li><li><strong>Metadata Sharding</strong>: Distribute based on source or content type</li></ul><h3>Storage Format</h3><pre>├── vectors/<br>│   ├── partition_2024_01/<br>│   │   ├── vectors.bin      # Binary vector data<br>│   │   ├── metadata.json   # Chunk metadata<br>│   │   └── index.hnsw      # HNSW index file<br>│   └── partition_2024_02/<br>├── metadata/<br>│   ├── documents.db        # Document-level metadata<br>│   └── chunks.db          # Chunk-level metadata<br>└── indices/<br>    ├── content_hash.idx<br>    └── timestamp.idx</pre><h3>4. Indexing Strategy</h3><h3>Multi-Index Approach</h3><ol><li><strong>Primary Vector Index (HNSW)</strong></li></ol><ul><li>Fast approximate similarity search</li><li>Configurable precision/speed tradeoffs</li><li>Support for incremental updates</li></ul><ol><li><strong>Secondary Indices</strong></li></ol><ul><li>B-tree indices for range queries on metadata</li><li>Inverted indices for tag-based filtering</li><li>Full-text search indices for hybrid retrieval</li></ul><h3>Index Maintenance</h3><ul><li><strong>Incremental Updates</strong>: Add new vectors without full rebuild</li><li><strong>Background Optimization</strong>: Periodic index reorganization</li><li><strong>Version Control</strong>: Index versioning for rollback capability</li></ul><h3>5. Query Processing</h3><h3>Query Types</h3><p><strong>Pure Vector Similarity</strong></p><pre>SELECT * FROM vectors <br>WHERE cosine_similarity(vector, query_vector) &gt; 0.8<br>ORDER BY similarity DESC<br>LIMIT 10</pre><p><strong>Hybrid Queries</strong></p><pre>SELECT * FROM vectors <br>WHERE cosine_similarity(vector, query_vector) &gt; 0.7<br>AND metadata-&gt;&gt;&#39;source&#39; = &#39;research_papers&#39;<br>AND timestamp &gt; &#39;2024-01-01&#39;<br>ORDER BY similarity DESC</pre><p><strong>Multi-Vector Queries</strong></p><ul><li>Support for multiple query vectors</li><li>Weighted combination of similarity scores</li></ul><h3>Query Optimization</h3><ul><li><strong>Query Planning</strong>: Choose optimal index based on filters</li><li><strong>Result Caching</strong>: Cache frequent query results</li><li><strong>Parallel Processing</strong>: Distribute queries across partitions</li></ul><h3>6. Scalability Considerations</h3><h3>Horizontal Scaling</h3><ul><li><strong>Distributed Architecture</strong>: Multiple nodes with data replication</li><li><strong>Consistent Hashing</strong>: Distribute data across nodes</li><li><strong>Query Routing</strong>: Route queries to appropriate nodes</li></ul><h3>Performance Optimization</h3><p><strong>Memory Management</strong>:</p><ul><li>Keep hot indices in memory</li><li>LRU cache for vectors and metadata</li><li>Configurable memory limits per partition</li></ul><p><strong>Compression</strong>:</p><ul><li>Vector quantization (PQ, SQ) for reduced memory</li><li>Metadata compression using columnar formats</li></ul><h3>7. Data Pipeline Integration</h3><h3>Ingestion Pipeline</h3><pre>Document → Chunking → Embedding → Vector DB<br>    ↓<br>Metadata Extraction → Validation → Storage</pre><h3>Real-time Updates</h3><ul><li><strong>Streaming Ingestion</strong>: Kafka/Pulsar integration</li><li><strong>Batch Processing</strong>: Scheduled bulk updates</li><li><strong>Change Detection</strong>: Monitor source documents for updates</li></ul><h3>8. API Design</h3><h3>Core Endpoints</h3><pre>POST /vectors/search<br>{<br>  &quot;vector&quot;: [0.1, -0.3, ...],<br>  &quot;filters&quot;: {<br>    &quot;source&quot;: &quot;documents&quot;,<br>    &quot;timestamp&quot;: {&quot;gte&quot;: &quot;2024-01-01&quot;}<br>  },<br>  &quot;limit&quot;: 10,<br>  &quot;threshold&quot;: 0.8<br>}<br><br>POST /vectors/insert<br>{<br>  &quot;documents&quot;: [<br>    {<br>      &quot;id&quot;: &quot;doc_1&quot;,<br>      &quot;vector&quot;: [...],<br>      &quot;metadata&quot;: {...},<br>      &quot;content&quot;: &quot;...&quot;<br>    }<br>  ]<br>}<br><br>DELETE /vectors/{id}<br>PUT /vectors/{id}<br>GET /vectors/{id}</pre><h3>9. Monitoring &amp; Observability</h3><h3>Key Metrics</h3><ul><li><strong>Query Latency</strong>: P50, P95, P99 response times</li><li><strong>Throughput</strong>: Queries per second, ingestion rate</li><li><strong>Index Health</strong>: Index size, fragmentation levels</li><li><strong>Memory Usage</strong>: Per-partition memory consumption</li></ul><h3>Logging</h3><ul><li>Query execution plans and performance</li><li>Index update operations</li><li>Error tracking and alerting</li></ul><h3>10. Security &amp; Access Control</h3><h3>Authentication &amp; Authorization</h3><ul><li>API key-based authentication</li><li>Role-based access control (RBAC)</li><li>Document-level permissions</li></ul><h3>Data Privacy</h3><ul><li>Encryption at rest and in transit</li><li>PII detection and masking</li><li>Audit logging for compliance</li></ul><h3>11. Backup &amp; Recovery</h3><h3>Backup Strategy</h3><ul><li><strong>Incremental Backups</strong>: Daily vector and metadata backups</li><li><strong>Index Snapshots</strong>: Periodic index state snapshots</li><li><strong>Cross-Region Replication</strong>: Disaster recovery setup</li></ul><h3>Recovery Procedures</h3><ul><li>Point-in-time recovery capabilities</li><li>Index rebuild from source data</li><li>Automated failover mechanisms</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=894cfe30e1f0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[QA Agentic Flow — Series -3]]></title>
            <link>https://medium.com/@to.adarshsingh/qa-agentic-flow-series-3-888392420808?source=rss-73f6989daa12------2</link>
            <guid isPermaLink="false">https://medium.com/p/888392420808</guid>
            <dc:creator><![CDATA[Adarsh Singh Dikhit]]></dc:creator>
            <pubDate>Sun, 25 May 2025 12:46:11 GMT</pubDate>
            <atom:updated>2025-05-25T12:46:11.169Z</atom:updated>
            <content:encoded><![CDATA[<h3>QA Agentic Flow — Series -3</h3><p><strong>9. Security &amp; Compliance</strong></p><ul><li>Authentication and authorization</li><li>Rate limiting implementation</li><li>Data encryption at rest</li><li>GDPR compliance features</li><li>Audit logging system</li><li>Security vulnerability scanning</li></ul><p><strong>10. Troubleshooting</strong></p><ul><li>Performance diagnostics tools</li><li>Common issues and solutions</li><li>Memory analysis utilities</li><li>Log analysis scripts</li><li>Health check automation</li><li>Emergency recovery procedures</li></ul><h3>9. Security &amp; Compliance</h3><h3>9.1 Security Architecture</h3><h4>API Security</h4><pre># security/auth.py<br>from functools import wraps<br>from flask import request, jsonify, current_app<br>import jwt<br>import redis<br>from datetime import datetime, timedelta<br><br>class SecurityManager:<br>    def __init__(self, app=None):<br>        self.redis_client = redis.Redis(<br>            host=&#39;redis&#39;, port=6379, db=1, <br>            decode_responses=True<br>        )<br>        if app:<br>            self.init_app(app)<br>    <br>    def init_app(self, app):<br>        self.app = app<br>        self.secret_key = app.config[&#39;JWT_SECRET_KEY&#39;]<br>        <br>    def generate_token(self, user_id: str, roles: list) -&gt; str:<br>        payload = {<br>            &#39;user_id&#39;: user_id,<br>            &#39;roles&#39;: roles,<br>            &#39;exp&#39;: datetime.utcnow() + timedelta(hours=24),<br>            &#39;iat&#39;: datetime.utcnow()<br>        }<br>        return jwt.encode(payload, self.secret_key, algorithm=&#39;HS256&#39;)<br>    <br>    def verify_token(self, token: str) -&gt; dict:<br>        try:<br>            # Check if token is blacklisted<br>            if self.redis_client.get(f&quot;blacklisted_token:{token}&quot;):<br>                raise jwt.InvalidTokenError(&quot;Token is blacklisted&quot;)<br>                <br>            payload = jwt.decode(token, self.secret_key, algorithms=[&#39;HS256&#39;])<br>            return payload<br>        except jwt.ExpiredSignatureError:<br>            raise jwt.InvalidTokenError(&quot;Token has expired&quot;)<br>        except jwt.InvalidTokenError:<br>            raise jwt.InvalidTokenError(&quot;Invalid token&quot;)<br>    <br>    def blacklist_token(self, token: str):<br>        # Add token to blacklist with expiration<br>        self.redis_client.setex(<br>            f&quot;blacklisted_token:{token}&quot;, <br>            86400, <br>            &quot;blacklisted&quot;<br>        )<br><br>def require_auth(roles=None):<br>    def decorator(f):<br>        @wraps(f)<br>        def decorated_function(*args, **kwargs):<br>            token = request.headers.get(&#39;Authorization&#39;)<br>            if not token or not token.startswith(&#39;Bearer &#39;):<br>                return jsonify({&#39;error&#39;: &#39;No valid token provided&#39;}), 401<br>            <br>            try:<br>                token = token.split(&#39; &#39;)[1]<br>                payload = current_app.security.verify_token(token)<br>                <br>                # Check roles if specified<br>                if roles and not any(role in payload.get(&#39;roles&#39;, []) for role in roles):<br>                    return jsonify({&#39;error&#39;: &#39;Insufficient permissions&#39;}), 403<br>                <br>                request.current_user = payload<br>                return f(*args, **kwargs)<br>            except jwt.InvalidTokenError as e:<br>                return jsonify({&#39;error&#39;: str(e)}), 401<br>        <br>        return decorated_function<br>    return decorator</pre><h4>Rate Limiting</h4><pre># security/rate_limit.py<br>from functools import wraps<br>from flask import request, jsonify<br>import redis<br>import time<br>from typing import Optional<br><br>class RateLimiter:<br>    def __init__(self, redis_client):<br>        self.redis = redis_client<br>    <br>    def is_allowed(self, key: str, limit: int, window: int) -&gt; tuple[bool, dict]:<br>        &quot;&quot;&quot;<br>        Check if request is allowed using sliding window algorithm<br>        Returns (allowed, info_dict)<br>        &quot;&quot;&quot;<br>        current_time = int(time.time())<br>        pipeline = self.redis.pipeline()<br>        <br>        # Remove old entries<br>        pipeline.zremrangebyscore(key, 0, current_time - window)<br>        <br>        # Count current requests<br>        pipeline.zcard(key)<br>        <br>        # Add current request<br>        pipeline.zadd(key, {str(current_time): current_time})<br>        <br>        # Set expiration<br>        pipeline.expire(key, window)<br>        <br>        results = pipeline.execute()<br>        <br>        current_requests = results[1]<br>        <br>        if current_requests &lt; limit:<br>            return True, {<br>                &#39;allowed&#39;: True,<br>                &#39;limit&#39;: limit,<br>                &#39;remaining&#39;: limit - current_requests - 1,<br>                &#39;reset_time&#39;: current_time + window<br>            }<br>        else:<br>            return False, {<br>                &#39;allowed&#39;: False,<br>                &#39;limit&#39;: limit,<br>                &#39;remaining&#39;: 0,<br>                &#39;reset_time&#39;: current_time + window<br>            }<br><br>def rate_limit(requests_per_minute: int = 60):<br>    def decorator(f):<br>        @wraps(f)<br>        def decorated_function(*args, **kwargs):<br>            # Get client identifier<br>            client_ip = request.environ.get(&#39;REMOTE_ADDR&#39;)<br>            user_id = getattr(request, &#39;current_user&#39;, {}).get(&#39;user_id&#39;)<br>            <br>            # Use user_id if authenticated, otherwise IP<br>            identifier = user_id or client_ip<br>            key = f&quot;rate_limit:{identifier}:{f.__name__}&quot;<br>            <br>            limiter = current_app.rate_limiter<br>            allowed, info = limiter.is_allowed(key, requests_per_minute, 60)<br>            <br>            if not allowed:<br>                response = jsonify({<br>                    &#39;error&#39;: &#39;Rate limit exceeded&#39;,<br>                    &#39;retry_after&#39;: info[&#39;reset_time&#39;] - int(time.time())<br>                })<br>                response.status_code = 429<br>                response.headers[&#39;Retry-After&#39;] = str(info[&#39;reset_time&#39;] - int(time.time()))<br>                return response<br>            <br>            # Add rate limit headers<br>            response = f(*args, **kwargs)<br>            if hasattr(response, &#39;headers&#39;):<br>                response.headers[&#39;X-RateLimit-Limit&#39;] = str(info[&#39;limit&#39;])<br>                response.headers[&#39;X-RateLimit-Remaining&#39;] = str(info[&#39;remaining&#39;])<br>                response.headers[&#39;X-RateLimit-Reset&#39;] = str(info[&#39;reset_time&#39;])<br>            <br>            return response<br>        <br>        return decorated_function<br>    return decorator</pre><h3>9.2 Data Protection</h3><h4>Encryption at Rest</h4><pre># security/encryption.py<br>from cryptography.fernet import Fernet<br>from cryptography.hazmat.primitives import hashes<br>from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC<br>import base64<br>import os<br><br>class DataEncryption:<br>    def __init__(self, password: str, salt: bytes = None):<br>        if salt is None:<br>            salt = os.urandom(16)<br>        <br>        kdf = PBKDF2HMAC(<br>            algorithm=hashes.SHA256(),<br>            length=32,<br>            salt=salt,<br>            iterations=100000,<br>        )<br>        key = base64.urlsafe_b64encode(kdf.derive(password.encode()))<br>        self.cipher = Fernet(key)<br>        self.salt = salt<br>    <br>    def encrypt(self, data: str) -&gt; str:<br>        &quot;&quot;&quot;Encrypt string data&quot;&quot;&quot;<br>        return self.cipher.encrypt(data.encode()).decode()<br>    <br>    def decrypt(self, encrypted_data: str) -&gt; str:<br>        &quot;&quot;&quot;Decrypt string data&quot;&quot;&quot;<br>        return self.cipher.decrypt(encrypted_data.encode()).decode()<br>    <br>    def encrypt_file(self, file_path: str, output_path: str):<br>        &quot;&quot;&quot;Encrypt file contents&quot;&quot;&quot;<br>        with open(file_path, &#39;rb&#39;) as file:<br>            file_data = file.read()<br>        <br>        encrypted_data = self.cipher.encrypt(file_data)<br>        <br>        with open(output_path, &#39;wb&#39;) as file:<br>            file.write(encrypted_data)<br><br># Database field encryption<br>class EncryptedField:<br>    def __init__(self, encryption_key: str):<br>        self.cipher = DataEncryption(encryption_key)<br>    <br>    def encrypt_value(self, value: str) -&gt; str:<br>        if value is None:<br>            return None<br>        return self.cipher.encrypt(value)<br>    <br>    def decrypt_value(self, encrypted_value: str) -&gt; str:<br>        if encrypted_value is None:<br>            return None<br>        return self.cipher.decrypt(encrypted_value)</pre><h3>9.3 Compliance Features</h3><h4>GDPR Compliance</h4><pre># compliance/gdpr.py<br>from datetime import datetime, timedelta<br>from typing import Dict, List, Optional<br>import json<br><br>class GDPRCompliance:<br>    def __init__(self, db_session):<br>        self.db = db_session<br>    <br>    def export_user_data(self, user_id: str) -&gt; Dict:<br>        &quot;&quot;&quot;Export all user data for GDPR compliance&quot;&quot;&quot;<br>        user_data = {<br>            &#39;user_profile&#39;: self._get_user_profile(user_id),<br>            &#39;task_history&#39;: self._get_user_tasks(user_id),<br>            &#39;api_usage&#39;: self._get_api_usage(user_id),<br>            &#39;preferences&#39;: self._get_user_preferences(user_id),<br>            &#39;export_timestamp&#39;: datetime.utcnow().isoformat()<br>        }<br>        return user_data<br>    <br>    def anonymize_user_data(self, user_id: str) -&gt; bool:<br>        &quot;&quot;&quot;Anonymize user data while preserving analytics&quot;&quot;&quot;<br>        try:<br>            # Replace identifiable information with anonymized values<br>            anonymized_id = f&quot;anon_{hash(user_id) % 1000000}&quot;<br>            <br>            # Update user profile<br>            self.db.execute(<br>                &quot;UPDATE users SET email = %s, name = %s, phone = NULL WHERE id = %s&quot;,<br>                (f&quot;{anonymized_id}@anonymized.local&quot;, f&quot;Anonymous User {anonymized_id}&quot;, user_id)<br>            )<br>            <br>            # Update task records<br>            self.db.execute(<br>                &quot;UPDATE tasks SET creator_id = %s WHERE creator_id = %s&quot;,<br>                (anonymized_id, user_id)<br>            )<br>            <br>            # Log anonymization<br>            self._log_data_action(user_id, &#39;anonymized&#39;)<br>            <br>            return True<br>        except Exception as e:<br>            self._log_error(f&quot;Anonymization failed for user {user_id}: {str(e)}&quot;)<br>            return False<br>    <br>    def delete_user_data(self, user_id: str) -&gt; bool:<br>        &quot;&quot;&quot;Complete data deletion for right to be forgotten&quot;&quot;&quot;<br>        try:<br>            # Delete in correct order to maintain referential integrity<br>            tables_to_clean = [<br>                &#39;user_preferences&#39;,<br>                &#39;api_usage_logs&#39;,<br>                &#39;task_history&#39;,<br>                &#39;user_sessions&#39;,<br>                &#39;users&#39;<br>            ]<br>            <br>            for table in tables_to_clean:<br>                self.db.execute(f&quot;DELETE FROM {table} WHERE user_id = %s&quot;, (user_id,))<br>            <br>            # Log deletion<br>            self._log_data_action(user_id, &#39;deleted&#39;)<br>            <br>            return True<br>        except Exception as e:<br>            self._log_error(f&quot;Data deletion failed for user {user_id}: {str(e)}&quot;)<br>            return False<br>    <br>    def _log_data_action(self, user_id: str, action: str):<br>        &quot;&quot;&quot;Log GDPR compliance actions&quot;&quot;&quot;<br>        self.db.execute(<br>            &quot;INSERT INTO gdpr_actions (user_id, action, timestamp) VALUES (%s, %s, %s)&quot;,<br>            (user_id, action, datetime.utcnow())<br>        )<br><br>#### Audit Logging<br><br>```python<br># compliance/audit.py<br>import json<br>from datetime import datetime<br>from enum import Enum<br>from typing import Dict, Any, Optional<br><br>class AuditEventType(Enum):<br>    USER_LOGIN = &quot;user_login&quot;<br>    USER_LOGOUT = &quot;user_logout&quot;<br>    DATA_ACCESS = &quot;data_access&quot;<br>    DATA_MODIFICATION = &quot;data_modification&quot;<br>    DATA_DELETION = &quot;data_deletion&quot;<br>    SYSTEM_CONFIG_CHANGE = &quot;system_config_change&quot;<br>    SECURITY_EVENT = &quot;security_event&quot;<br><br>class AuditLogger:<br>    def __init__(self, db_session):<br>        self.db = db_session<br>    <br>    def log_event(self, <br>                  event_type: AuditEventType,<br>                  user_id: Optional[str] = None,<br>                  resource_id: Optional[str] = None,<br>                  resource_type: Optional[str] = None,<br>                  action: str = None,<br>                  details: Dict[str, Any] = None,<br>                  ip_address: str = None,<br>                  user_agent: str = None):<br>        &quot;&quot;&quot;Log audit event&quot;&quot;&quot;<br>        <br>        audit_entry = {<br>            &#39;timestamp&#39;: datetime.utcnow().isoformat(),<br>            &#39;event_type&#39;: event_type.value,<br>            &#39;user_id&#39;: user_id,<br>            &#39;resource_id&#39;: resource_id,<br>            &#39;resource_type&#39;: resource_type,<br>            &#39;action&#39;: action,<br>            &#39;details&#39;: details or {},<br>            &#39;ip_address&#39;: ip_address,<br>            &#39;user_agent&#39;: user_agent,<br>            &#39;session_id&#39;: self._get_session_id()<br>        }<br>        <br>        # Store in database<br>        self.db.execute(&quot;&quot;&quot;<br>            INSERT INTO audit_logs <br>            (timestamp, event_type, user_id, resource_id, resource_type, <br>             action, details, ip_address, user_agent, session_id)<br>            VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s)<br>        &quot;&quot;&quot;, (<br>            audit_entry[&#39;timestamp&#39;],<br>            audit_entry[&#39;event_type&#39;],<br>            audit_entry[&#39;user_id&#39;],<br>            audit_entry[&#39;resource_id&#39;],<br>            audit_entry[&#39;resource_type&#39;],<br>            audit_entry[&#39;action&#39;],<br>            json.dumps(audit_entry[&#39;details&#39;]),<br>            audit_entry[&#39;ip_address&#39;],<br>            audit_entry[&#39;user_agent&#39;],<br>            audit_entry[&#39;session_id&#39;]<br>        ))<br>        <br>        # Also log to file for external SIEM systems<br>        with open(&#39;/app/logs/audit.log&#39;, &#39;a&#39;) as f:<br>            f.write(json.dumps(audit_entry) + &#39;\n&#39;)<br>    <br>    def _get_session_id(self) -&gt; str:<br>        from flask import session<br>        return session.get(&#39;session_id&#39;, &#39;anonymous&#39;)<br><br># Decorator for automatic audit logging<br>def audit_action(event_type: AuditEventType, resource_type: str = None):<br>    def decorator(func):<br>        @wraps(func)<br>        def wrapper(*args, **kwargs):<br>            from flask import request, g<br>            <br>            # Execute the function<br>            result = func(*args, **kwargs)<br>            <br>            # Log the audit event<br>            audit_logger = current_app.audit_logger<br>            audit_logger.log_event(<br>                event_type=event_type,<br>                user_id=getattr(g, &#39;current_user_id&#39;, None),<br>                resource_type=resource_type,<br>                action=func.__name__,<br>                details={<br>                    &#39;args&#39;: str(args),<br>                    &#39;kwargs&#39;: {k: str(v) for k, v in kwargs.items()},<br>                    &#39;result_status&#39;: &#39;success&#39; if result else &#39;failure&#39;<br>                },<br>                ip_address=request.environ.get(&#39;REMOTE_ADDR&#39;),<br>                user_agent=request.headers.get(&#39;User-Agent&#39;)<br>            )<br>            <br>            return result<br>        return wrapper<br>    return decorator</pre><h3>9.4 Security Scanning &amp; Vulnerability Management</h3><pre># security/scanner.py<br>import subprocess<br>import json<br>import requests<br>from typing import Dict, List<br>from datetime import datetime<br><br>class SecurityScanner:<br>    def __init__(self):<br>        self.scan_results = {}<br>    <br>    def scan_dependencies(self) -&gt; Dict:<br>        &quot;&quot;&quot;Scan Python dependencies for known vulnerabilities&quot;&quot;&quot;<br>        try:<br>            # Run safety check<br>            result = subprocess.run(<br>                [&#39;safety&#39;, &#39;check&#39;, &#39;--json&#39;],<br>                capture_output=True,<br>                text=True<br>            )<br>            <br>            if result.returncode == 0:<br>                vulnerabilities = json.loads(result.stdout) if result.stdout else []<br>            else:<br>                vulnerabilities = json.loads(result.stdout) if result.stdout else []<br>            <br>            scan_result = {<br>                &#39;timestamp&#39;: datetime.utcnow().isoformat(),<br>                &#39;scanner&#39;: &#39;safety&#39;,<br>                &#39;total_vulnerabilities&#39;: len(vulnerabilities),<br>                &#39;vulnerabilities&#39;: vulnerabilities,<br>                &#39;status&#39;: &#39;completed&#39;<br>            }<br>            <br>            self.scan_results[&#39;dependencies&#39;] = scan_result<br>            return scan_result<br>            <br>        except Exception as e:<br>            error_result = {<br>                &#39;timestamp&#39;: datetime.utcnow().isoformat(),<br>                &#39;scanner&#39;: &#39;safety&#39;,<br>                &#39;error&#39;: str(e),<br>                &#39;status&#39;: &#39;failed&#39;<br>            }<br>            self.scan_results[&#39;dependencies&#39;] = error_result<br>            return error_result<br>    <br>    def scan_container_image(self, image_name: str) -&gt; Dict:<br>        &quot;&quot;&quot;Scan Docker image for vulnerabilities using Trivy&quot;&quot;&quot;<br>        try:<br>            result = subprocess.run([<br>                &#39;trivy&#39;, &#39;image&#39;, &#39;--format&#39;, &#39;json&#39;, &#39;--quiet&#39;, image_name<br>            ], capture_output=True, text=True)<br>            <br>            if result.returncode == 0:<br>                scan_data = json.loads(result.stdout)<br>                vulnerabilities = []<br>                <br>                for target in scan_data.get(&#39;Results&#39;, []):<br>                    for vuln in target.get(&#39;Vulnerabilities&#39;, []):<br>                        vulnerabilities.append({<br>                            &#39;id&#39;: vuln.get(&#39;VulnerabilityID&#39;),<br>                            &#39;severity&#39;: vuln.get(&#39;Severity&#39;),<br>                            &#39;package&#39;: vuln.get(&#39;PkgName&#39;),<br>                            &#39;version&#39;: vuln.get(&#39;InstalledVersion&#39;),<br>                            &#39;fixed_version&#39;: vuln.get(&#39;FixedVersion&#39;),<br>                            &#39;title&#39;: vuln.get(&#39;Title&#39;),<br>                            &#39;description&#39;: vuln.get(&#39;Description&#39;)<br>                        })<br>                <br>                scan_result = {<br>                    &#39;timestamp&#39;: datetime.utcnow().isoformat(),<br>                    &#39;scanner&#39;: &#39;trivy&#39;,<br>                    &#39;image&#39;: image_name,<br>                    &#39;total_vulnerabilities&#39;: len(vulnerabilities),<br>                    &#39;critical&#39;: len([v for v in vulnerabilities if v[&#39;severity&#39;] == &#39;CRITICAL&#39;]),<br>                    &#39;high&#39;: len([v for v in vulnerabilities if v[&#39;severity&#39;] == &#39;HIGH&#39;]),<br>                    &#39;medium&#39;: len([v for v in vulnerabilities if v[&#39;severity&#39;] == &#39;MEDIUM&#39;]),<br>                    &#39;low&#39;: len([v for v in vulnerabilities if v[&#39;severity&#39;] == &#39;LOW&#39;]),<br>                    &#39;vulnerabilities&#39;: vulnerabilities,<br>                    &#39;status&#39;: &#39;completed&#39;<br>                }<br>                <br>                self.scan_results[&#39;container&#39;] = scan_result<br>                return scan_result<br>                <br>        except Exception as e:<br>            error_result = {<br>                &#39;timestamp&#39;: datetime.utcnow().isoformat(),<br>                &#39;scanner&#39;: &#39;trivy&#39;,<br>                &#39;image&#39;: image_name,<br>                &#39;error&#39;: str(e),<br>                &#39;status&#39;: &#39;failed&#39;<br>            }<br>            self.scan_results[&#39;container&#39;] = error_result<br>            return error_result<br>    <br>    def generate_security_report(self) -&gt; Dict:<br>        &quot;&quot;&quot;Generate comprehensive security report&quot;&quot;&quot;<br>        report = {<br>            &#39;timestamp&#39;: datetime.utcnow().isoformat(),<br>            &#39;summary&#39;: {<br>                &#39;total_scans&#39;: len(self.scan_results),<br>                &#39;completed_scans&#39;: len([r for r in self.scan_results.values() if r.get(&#39;status&#39;) == &#39;completed&#39;]),<br>                &#39;failed_scans&#39;: len([r for r in self.scan_results.values() if r.get(&#39;status&#39;) == &#39;failed&#39;]),<br>                &#39;total_vulnerabilities&#39;: sum([r.get(&#39;total_vulnerabilities&#39;, 0) for r in self.scan_results.values()]),<br>                &#39;critical_vulnerabilities&#39;: sum([r.get(&#39;critical&#39;, 0) for r in self.scan_results.values()]),<br>                &#39;high_vulnerabilities&#39;: sum([r.get(&#39;high&#39;, 0) for r in self.scan_results.values()])<br>            },<br>            &#39;scan_results&#39;: self.scan_results,<br>            &#39;recommendations&#39;: self._generate_recommendations()<br>        }<br>        <br>        return report<br>    <br>    def _generate_recommendations(self) -&gt; List[str]:<br>        &quot;&quot;&quot;Generate security recommendations based on scan results&quot;&quot;&quot;<br>        recommendations = []<br>        <br>        # Check dependency vulnerabilities<br>        if &#39;dependencies&#39; in self.scan_results:<br>            dep_scan = self.scan_results[&#39;dependencies&#39;]<br>            if dep_scan.get(&#39;total_vulnerabilities&#39;, 0) &gt; 0:<br>                recommendations.append(&quot;Update vulnerable dependencies to their latest secure versions&quot;)<br>                recommendations.append(&quot;Consider using dependency pinning and automated vulnerability monitoring&quot;)<br>        <br>        # Check container vulnerabilities<br>        if &#39;container&#39; in self.scan_results:<br>            container_scan = self.scan_results[&#39;container&#39;]<br>            if container_scan.get(&#39;critical&#39;, 0) &gt; 0:<br>                recommendations.append(&quot;Immediately address critical vulnerabilities in container images&quot;)<br>            if container_scan.get(&#39;total_vulnerabilities&#39;, 0) &gt; 50:<br>                recommendations.append(&quot;Consider using a minimal base image to reduce attack surface&quot;)<br>        <br>        # General recommendations<br>        recommendations.extend([<br>            &quot;Implement regular automated security scanning in CI/CD pipeline&quot;,<br>            &quot;Set up vulnerability alerts and monitoring&quot;,<br>            &quot;Establish incident response procedures for security vulnerabilities&quot;,<br>            &quot;Regular security training for development team&quot;<br>        ])<br>        <br>        return recommendations</pre><h3>10. Troubleshooting</h3><h3>10.1 Common Issues and Solutions</h3><h4>Performance Issues</h4><p><strong>Issue: High Response Times</strong></p><pre># diagnosis/performance.py<br>import time<br>import psutil<br>from typing import Dict, Any<br><br>class PerformanceDiagnostics:<br>    def diagnose_slow_responses(self) -&gt; Dict[str, Any]:<br>        &quot;&quot;&quot;Diagnose potential causes of slow API responses&quot;&quot;&quot;<br>        diagnostics = {<br>            &#39;timestamp&#39;: time.time(),<br>            &#39;system_metrics&#39;: self._get_system_metrics(),<br>            &#39;database_performance&#39;: self._check_database_performance(),<br>            &#39;cache_performance&#39;: self._check_cache_performance(),<br>            &#39;queue_status&#39;: self._check_queue_status()<br>        }<br>        <br>        # Generate recommendations<br>        diagnostics[&#39;recommendations&#39;] = self._generate_performance_recommendations(diagnostics)<br>        <br>        return diagnostics<br>    <br>    def _get_system_metrics(self) -&gt; Dict:<br>        &quot;&quot;&quot;Get current system resource usage&quot;&quot;&quot;<br>        return {<br>            &#39;cpu_percent&#39;: psutil.cpu_percent(interval=1),<br>            &#39;memory_percent&#39;: psutil.virtual_memory().percent,<br>            &#39;disk_usage&#39;: psutil.disk_usage(&#39;/&#39;).percent,<br>            &#39;load_average&#39;: psutil.getloadavg(),<br>            &#39;active_connections&#39;: len(psutil.net_connections())<br>        }<br>    <br>    def _check_database_performance(self) -&gt; Dict:<br>        &quot;&quot;&quot;Check database performance metrics&quot;&quot;&quot;<br>        # This would connect to your database and check performance<br>        return {<br>            &#39;active_connections&#39;: 15,  # Example values<br>            &#39;slow_queries&#39;: 3,<br>            &#39;lock_waits&#39;: 0,<br>            &#39;index_efficiency&#39;: 0.95<br>        }<br>    <br>    def _check_cache_performance(self) -&gt; Dict:<br>        &quot;&quot;&quot;Check Redis cache performance&quot;&quot;&quot;<br>        return {<br>            &#39;hit_rate&#39;: 0.85,<br>            &#39;memory_usage&#39;: 0.45,<br>            &#39;connected_clients&#39;: 12,<br>            &#39;commands_per_second&#39;: 1500<br>        }<br>    <br>    def _check_queue_status(self) -&gt; Dict:<br>        &quot;&quot;&quot;Check message queue status&quot;&quot;&quot;<br>        return {<br>            &#39;pending_tasks&#39;: 25,<br>            &#39;processing_tasks&#39;: 5,<br>            &#39;failed_tasks&#39;: 2,<br>            &#39;average_processing_time&#39;: 15.5<br>        }<br>    <br>    def _generate_performance_recommendations(self, diagnostics: Dict) -&gt; List[str]:<br>        &quot;&quot;&quot;Generate performance improvement recommendations&quot;&quot;&quot;<br>        recommendations = []<br>        <br>        system = diagnostics[&#39;system_metrics&#39;]<br>        if system[&#39;cpu_percent&#39;] &gt; 80:<br>            recommendations.append(&quot;High CPU usage detected - consider scaling horizontally&quot;)<br>        if system[&#39;memory_percent&#39;] &gt; 85:<br>            recommendations.append(&quot;High memory usage - check for memory leaks or increase memory limits&quot;)<br>        <br>        db = diagnostics[&#39;database_performance&#39;]<br>        if db[&#39;slow_queries&#39;] &gt; 5:<br>            recommendations.append(&quot;Multiple slow queries detected - review and optimize database queries&quot;)<br>        <br>        cache = diagnostics[&#39;cache_performance&#39;]<br>        if cache[&#39;hit_rate&#39;] &lt; 0.7:<br>            recommendations.append(&quot;Low cache hit rate - review caching strategy&quot;)<br>        <br>        queue = diagnostics[&#39;queue_status&#39;]<br>        if queue[&#39;pending_tasks&#39;] &gt; 50:<br>            recommendations.append(&quot;Queue backlog detected - consider adding more workers&quot;)<br>        <br>        return recommendations</pre><h4>Connection Issues</h4><p><strong>Issue: Database Connection Failures</strong></p><pre>#!/bin/bash<br># scripts/diagnose_db.sh<br><br>echo &quot;=== Database Connection Diagnostics ===&quot;<br><br># Check if PostgreSQL pod is running<br>echo &quot;Checking PostgreSQL pod status...&quot;<br>kubectl get pods -n qa-agentic-flow -l app=postgres<br><br># Check PostgreSQL service<br>echo &quot;Checking PostgreSQL service...&quot;<br>kubectl get svc -n qa-agentic-flow postgres-service<br><br># Test connection from within cluster<br>echo &quot;Testing database connection...&quot;<br>kubectl exec -n qa-agentic-flow deployment/qa-api -- pg_isready -h postgres-service -p 5432<br><br># Check database logs<br>echo &quot;Recent PostgreSQL logs...&quot;<br>kubectl logs -n qa-agentic-flow deployment/postgres --tail=20<br><br># Check connection pool status<br>echo &quot;Connection pool status...&quot;<br>kubectl exec -n qa-agentic-flow deployment/qa-api -- python -c &quot;<br>import psycopg2<br>try:<br>    conn = psycopg2.connect(&#39;postgresql://qa_user:qa_password@postgres-service:5432/qa_db&#39;)<br>    print(&#39;Database connection successful&#39;)<br>    conn.close()<br>except Exception as e:<br>    print(f&#39;Database connection failed: {e}&#39;)<br>&quot;</pre><h4>Memory and Resource Issues</h4><p><strong>Issue: Out of Memory Errors</strong></p><pre># diagnosis/memory.py<br>import gc<br>import tracemalloc<br>from typing import Dict, List, Tuple<br><br>class MemoryDiagnostics:<br>    def __init__(self):<br>        tracemalloc.start()<br>    <br>    def analyze_memory_usage(self) -&gt; Dict:<br>        &quot;&quot;&quot;Analyze current memory usage patterns&quot;&quot;&quot;<br>        snapshot = tracemalloc.take_snapshot()<br>        top_stats = snapshot.statistics(&#39;lineno&#39;)<br>        <br>        memory_analysis = {<br>            &#39;total_memory_blocks&#39;: len(snapshot.traces),<br>            &#39;total_memory_size&#39;: sum(stat.size for stat in top_stats),<br>            &#39;top_memory_consumers&#39;: [<br>                {<br>                    &#39;filename&#39;: stat.traceback.format()[0],<br>                    &#39;line_number&#39;: stat.traceback[0].lineno,<br>                    &#39;size_mb&#39;: round(stat.size / 1024 / 1024, 2),<br>                    &#39;count&#39;: stat.count<br>                }<br>                for stat in top_stats[:10]<br>            ],<br>            &#39;garbage_collection_stats&#39;: {<br>                &#39;generation_0&#39;: gc.get_count()[0],<br>                &#39;generation_1&#39;: gc.get_count()[1],<br>                &#39;generation_2&#39;: gc.get_count()[2],<br>                &#39;total_collections&#39;: sum(gc.get_stats()[i][&#39;collections&#39;] for i in range(3))<br>            }<br>        }<br>        <br>        return memory_analysis<br>    <br>    def suggest_memory_optimizations(self) -&gt; List[str]:<br>        &quot;&quot;&quot;Suggest memory optimization strategies&quot;&quot;&quot;<br>        suggestions = [<br>            &quot;Use generators instead of lists for large data processing&quot;,<br>            &quot;Implement proper connection pooling to avoid connection leaks&quot;,<br>            &quot;Use context managers for file and database operations&quot;,<br>            &quot;Consider using slots for frequently instantiated classes&quot;,<br>            &quot;Implement memory monitoring alerts&quot;,<br>            &quot;Use memory profiling tools like memory_profiler&quot;,<br>            &quot;Consider implementing data pagination for large responses&quot;<br>        ]<br>        return suggestions</pre><h3>10.2 Debugging Tools and Scripts</h3><h4>Log Analysis Script</h4><pre># scripts/analyze_logs.py<br>import json<br>import re<br>from collections import defaultdict, Counter<br>from datetime import datetime, timedelta<br>from typing import Dict, List<br><br>class LogAnalyzer:<br>    def __init__(self, log_file_path: str):<br>        self.log_file = log_file_path<br>        self.logs = []<br>        self._load_logs()<br>    <br>    def _load_logs(self):<br>        &quot;&quot;&quot;Load and parse JSON logs&quot;&quot;&quot;<br>        with open(self.log_file, &#39;r&#39;) as f:<br>            for line in f:<br>                try:<br>                    log_entry = json.loads(line.strip())<br>                    self.logs.append(log_entry)<br>                except json.JSONDecodeError:<br>                    continue<br>    <br>    def analyze_errors(self, hours: int = 24) -&gt; Dict:<br>        &quot;&quot;&quot;Analyze error patterns in the last N hours&quot;&quot;&quot;<br>        cutoff_time = datetime.now() - timedelta(hours=hours)<br>        <br>        error_logs = [<br>            log for log in self.logs <br>            if log.get(&#39;level&#39;) == &#39;ERROR&#39; and <br>            datetime.fromisoformat(log.get(&#39;timestamp&#39;, &#39;&#39;)) &gt; cutoff_time<br>        ]<br>        <br>        error_analysis = {<br>            &#39;total_errors&#39;: len(error_logs),<br>            &#39;error_by_module&#39;: Counter(log.get(&#39;module&#39;, &#39;unknown&#39;) for log in error_logs),<br>            &#39;error_by_function&#39;: Counter(log.get(&#39;function&#39;, &#39;unknown&#39;) for log in error_logs),<br>            &#39;common_error_messages&#39;: Counter(log.get(&#39;message&#39;, &#39;&#39;)[:100] for log in error_logs),<br>            &#39;error_timeline&#39;: self._create_error_timeline(error_logs),<br>            &#39;recent_errors&#39;: error_logs[-10:] if error_logs else []<br>        }<br>        <br>        return error_analysis<br>    <br>    def analyze_performance(self) -&gt; Dict:<br>        &quot;&quot;&quot;Analyze performance-related log entries&quot;&quot;&quot;<br>        perf_patterns = [<br>            &#39;slow query&#39;,<br>            &#39;timeout&#39;,<br>            &#39;high cpu&#39;,<br>            &#39;memory warning&#39;,<br>            &#39;connection pool&#39;<br>        ]<br>        <br>        perf_logs = []<br>        for log in self.logs:<br>            message = log.get(&#39;message&#39;, &#39;&#39;).lower()<br>            if any(pattern in message for pattern in perf_patterns):<br>                perf_logs.append(log)<br>        <br>        return {<br>            &#39;performance_issues&#39;: len(perf_logs),<br>            &#39;issue_types&#39;: Counter(<br>                next((pattern for pattern in perf_patterns <br>                     if pattern in log.get(&#39;message&#39;, &#39;&#39;).lower()), &#39;other&#39;)<br>                for log in perf_logs<br>            ),<br>            &#39;recent_performance_issues&#39;: perf_logs[-5:]<br>        }<br>    <br>    def _create_error_timeline(self, error_logs: List[Dict]) -&gt; Dict:<br>        &quot;&quot;&quot;Create hourly error timeline&quot;&quot;&quot;<br>        timeline = defaultdict(int)<br>        for log in error_logs:<br>            timestamp = datetime.fromisoformat(log.get(&#39;timestamp&#39;, &#39;&#39;))<br>            hour_key = timestamp.strftime(&#39;%Y-%m-%d %H:00&#39;)<br>            timeline[hour_key] += 1<br>        <br>        return dict(sorted(timeline.items()))<br>    <br>    def generate_report(self) -&gt; str:<br>        &quot;&quot;&quot;Generate comprehensive log analysis report&quot;&quot;&quot;<br>        error_analysis = self.analyze_errors()<br>        perf_analysis = self.analyze_performance()<br>        <br>        report = f&quot;&quot;&quot;<br># Log Analysis Report<br>Generated at: {datetime.now().isoformat()}<br><br>## Error Analysis (Last 24 hours)<br>- Total Errors: {error_analysis[&#39;total_errors&#39;]}<br>- Most Common Error Modules: {dict(error_analysis[&#39;error_by_module&#39;].most_common(5))}<br>- Most Common Error Functions: {dict(error_analysis[&#39;error_by_function&#39;].most_common(5))}<br><br>## Performance Analysis<br>- Total Performance Issues: {perf_analysis[&#39;performance_issues&#39;]}<br>- Issue Types: {dict(perf_analysis[&#39;issue_types&#39;])}<br><br>## Recommendations<br>&quot;&quot;&quot;<br>        <br>        if error_analysis[&#39;total_errors&#39;] &gt; 100:<br>            report += &quot;- High error rate detected - investigate immediately\n&quot;<br>        <br>        if &#39;timeout&#39; in perf_analysis[&#39;issue_types&#39;]:<br>            report += &quot;- Timeout issues detected - check service dependencies\n&quot;<br>        <br>        if &#39;memory warning&#39; in perf_analysis[&#39;issue_types&#39;]:<br>            report += &quot;- Memory warnings detected - monitor resource usage\n&quot;<br>        <br>        return report<br><br># Usage<br>if __name__ == &quot;__main__&quot;:<br>    analyzer = LogAnalyzer(&#39;/app/logs/qa_agentic.log&#39;)<br>    print(analyzer.generate_report())</pre><h4>Health Check Script</h4><pre># scripts/health_check.py<br>import requests<br>import psycopg2<br>import redis<br>import json<br>from typing import Dict, List<br>import time<br><br>class HealthChecker:<br>    def __init__(self, config: Dict):<br>        self.config = config<br>        self.results = {}<br>    <br>    def check_api_health(self) -&gt; Dict:<br>        &quot;&quot;&quot;Check API service health&quot;&quot;&quot;<br>        try:<br>            response = requests.get(<br>                f&quot;{self.config[&#39;api_url&#39;]}/health&quot;,<br>                timeout=5<br>            )<br>            <br>            return {<br>                &#39;status&#39;: &#39;healthy&#39; if response.status_code == 200 else &#39;unhealthy&#39;,<br>                &#39;response_time&#39;: response.elapsed.total_seconds(),<br>                &#39;status_code&#39;: response.status_code,<br>                &#39;details&#39;: response.json() if response.status_code == 200 else None<br>            }<br>        except Exception as e:<br>            return {<br>                &#39;status&#39;: &#39;unhealthy&#39;,<br>                &#39;error&#39;: str(e)<br>            }<br>    <br>    def check_database_health(self) -&gt; Dict:<br>        &quot;&quot;&quot;Check PostgreSQL database health&quot;&quot;&quot;<br>        try:<br>            conn = psycopg2.connect(self.config[&#39;database_url&#39;])<br>            cursor = conn.cursor()<br>            <br>            start_time = time.time()<br>            cursor.execute(&quot;SELECT 1&quot;)<br>            query_time = time.time() - start_time<br>            <br>            cursor.execute(&quot;SELECT count(*) FROM pg_stat_activity&quot;)<br>            active_connections = cursor.fetchone()[0]<br>            <br>            conn.close()<br>            <br>            return {<br>                &#39;status&#39;: &#39;healthy&#39;,<br>                &#39;query_time&#39;: query_time,<br>                &#39;active_connections&#39;: active_connections<br>            }<br>        except Exception as e:<br>            return {<br>                &#39;status&#39;: &#39;unhealthy&#39;,<br>                &#39;error&#39;: str(e)<br>            }<br>    <br>    def check_redis_health(self) -&gt; Dict:<br>        &quot;&quot;&quot;Check Redis cache health&quot;&quot;&quot;<br>        try:<br>            r = redis.Redis.from_url(https://medium.com/self.config[&#39;redis_url&#39;])<br>            <br>            start_time = time.time()<br>            r.ping()<br>            ping_time = time.time() - start_time<br>            <br>            info = r.info()<br>            <br>            return {<br>                &#39;status&#39;: &#39;healthy&#39;,<br>                &#39;ping_time&#39;: ping_time,<br>                &#39;connected_clients&#39;: info[&#39;connected_clients&#39;],<br>                &#39;used_memory&#39;: info[&#39;used_memory_human&#39;],<br>                &#39;hit_rate&#39;: info.get(&#39;keyspace_hits&#39;, 0) / max(info.get(&#39;keyspace_hits&#39;, 0) + info.get(&#39;keyspace_misses&#39;, 0), 1)<br>            }<br>        except Exception as e:<br>            return {<br>                &#39;status&#39;: &#39;unhealthy&#39;,<br>                &#39;error&#39;: str(e)<br>            }<br>    <br>    def run_all_checks(self) -&gt; Dict:<br>        &quot;&quot;&quot;Run all health checks&quot;&quot;&quot;<br>        self.results = {<br>            &#39;timestamp&#39;: time.time(),<br>            &#39;api&#39;: self.check_api_health(),<br>            &#39;database&#39;: self.check_database_health(),<br>            &#39;redis&#39;: self.check_redis_health()<br>        }<br>        <br>        # Determine overall health<br>        all_healthy = all(<br>            result.get(&#39;status&#39;) == &#39;healthy&#39; <br>            for result in self.results.values() <br>            if isinstance(result, dict) and &#39;status&#39; in result<br>        )<br>        <br>        self.results[&#39;overall_status&#39;] = &#39;healthy&#39; if all_healthy else &#39;unhealthy&#39;<br>        <br>        return self.results<br>    <br>    def generate_health_report(self) -&gt; str:<br>        &quot;&quot;&quot;Generate human-readable health report&quot;&quot;&quot;<br>        if not self.results:<br>            self.run_all_checks()<br>        <br>        report = f&quot;&quot;&quot;<br># System Health Report<br>Generated at: {time.ctime(self.results[&#39;timestamp&#39;])}<br>Overall Status: {self.results[&#39;overall_status&#39;].upper()}<br><br>## Component Status<br>&quot;&quot;&quot;<br>        <br>        for component, result in self.results.items():<br>            if component in [&#39;timestamp&#39;, &#39;overall_status&#39;]:<br>                continue<br>                <br>            status = result.get(&#39;status&#39;, &#39;unknown&#39;).upper()<br>            report += f&quot;- {component.title()}: {status}\n&quot;<br>            <br>            if result.get(&#39;error&#39;):<br>                report += f&quot;  Error: {result[&#39;error&#39;]}\n&quot;<br>            elif status == &#39;HEALTHY&#39;:<br>                if &#39;response_time&#39; in result:<br>                    report += f&quot;  Response Time: {result[&#39;response_time&#39;]:.3f}s\n&quot;<br>                if &#39;query_time&#39; in result:<br>                    report += f&quot;  Query Time: {result[&#39;query_time&#39;]:.3f}s\n&quot;<br>                if &#39;ping_time&#39; in result:<br>                    report += f&quot;  Ping Time: {result[&#39;ping_time&#39;]:.3f}s\n&quot;<br>        <br>        return report<br><br># Usage example<br>if __name__ == &quot;__main__&quot;:<br>    config = {<br>        &#39;api_url&#39;: &#39;http://qa-api-service&#39;,<br>        &#39;database_url&#39;: &#39;postgresql://qa_user:qa_password@postgres-service:5432/qa_db&#39;,<br>        &#39;redis_url&#39;: &#39;redis://redis-service:6379&#39;<br>    }<br>    <br>    checker = HealthChecker(config)<br>    results = checker.run_all_checks()<br>    print(checker.generate_health_report())<br>    <br>    # Exit with error code if unhealthy<br>    if results[&#39;overall_status&#39;] != &#39;healthy&#39;:<br>        exit(1)</pre><h3>10.3 Emergency Recovery Procedures</h3><h4>Database Recovery</h4><pre>#!/bin/bash<br># scripts/emergency_db_recovery.sh<br><br>set -e<br><br>NAMESPACE=&quot;qa-agentic-flow&quot;<br>BACKUP_PATH=&quot;/backups&quot;<br>TIMESTAMP=$(date +%Y%m%d_%H%M%S)<br><br>echo &quot;=== Emergency Database Recovery ===&quot;<br>echo &quot;Timestamp: $(date)&quot;<br><br># Step 1: Create emergency backup of current state<br>echo &quot;Creating emergency backup...&quot;<br>kubectl exec -n $NAMESPACE deployment/postgres -- pg_dump -U qa_user qa_db &gt; &quot;emergency_backup_${TIMESTAMP}.sql&quot;<br><br># Step 2: Check available backups<br>echo &quot;Available backups:&quot;<br>ls -la $BACKUP_PATH/*.sql | tail -10<br><br># Step 3: Restore from most recent backup<br>LATEST_BACKUP=$(ls -t $BACKUP_PATH/*.sql | head -1)<br>echo &quot;Restoring from: $LATEST_BACKUP&quot;<br><br># Stop application to prevent new connections<br>echo &quot;Scaling down application...&quot;<br>kubectl scale -n $NAMESPACE deployment/qa-api --replicas=0<br><br># Restore database<br>echo &quot;Restoring database...&quot;<br>kubectl exec -i -n $NAMESPACE deployment/postgres -- psql -U qa_user -d qa_db &lt; &quot;$LATEST_BACKUP&quot;<br><br># Verify restoration<br>echo &quot;Verifying restoration...&quot;<br>kubectl exec -n $NAMESPACE deployment/postgres -- psql -U qa_user -d qa_db -c &quot;SELECT COUNT(*) FROM information_schema.tables;&quot;<br><br># Restart application<br>echo &quot;Restarting application...&quot;<br>kubectl scale -n $NAMESPACE deployment/qa-api --replicas=3<br><br># Wait for pods to be ready<br>kubectl wait -n $NAMESPACE --for=condition=ready pod -l app=qa-api --timeout=300s<br><br>echo &quot;Database recovery completed successfully!&quot;</pre><h4>Service Recovery</h4><pre>#!/bin/bash<br># scripts/emergency_service_recovery.sh<br><br>NAMESPACE=&quot;qa-agentic-flow&quot;<br><br>echo &quot;=== Emergency Service Recovery ===&quot;<br><br># Step 1: Check current service status<br>echo &quot;Current service status:&quot;<br>kubectl get pods -n $NAMESPACE<br><br># Step 2: Identify failed services<br>FAILED_PODS=$(kubectl get pods -n $NAMESPACE --field-selector=status.phase!=Running -o name)<br><br>if [ -z &quot;$FAILED_PODS&quot; ]; then<br>    echo &quot;No failed pods detected&quot;<br>    exit 0<br>fi<br><br>echo &quot;Failed pods detected:&quot;<br>echo &quot;$FAILED_PODS&quot;<br><br># Step 3: Collect logs from failed pods<br>mkdir -p /tmp/emergency_logs<br>for pod in $FAILED_PODS; do<br>    POD_NAME=$(echo $pod | cut -d&#39;/&#39; -f2)<br>    echo &quot;Collecting logs from $POD_NAME...&quot;<br>    kubectl logs -n $NAMESPACE $POD_NAME &gt; &quot;/tmp/emergency_logs/${POD_NAME}_$(date +%Y%m%d_%H%M%S).log&quot; || true<br>done<br><br># Step 4: Force restart failed services<br>echo &quot;Restarting failed services...&quot;<br>for pod in $FAILED_PODS; do<br>    kubectl delete -n $NAMESPACE $pod<br>done<br><br># Step 5: Wait for services to recover<br>echo &quot;Waiting for services to recover...&quot;<br>kubectl wait -n $NAMESPACE --for=condition=ready pod -l app=qa-api --timeout=300s<br><br># Step 6: Verify service health<br>echo &quot;Verifying service health...&quot;<br>sleep 30<br>kubectl get pods -n $NAMESPACE<br><br># Step 7: Run health checks<br>kubectl exec -n $NAMESPACE deployment/qa-api -- python /app/scripts/health_check.py<br><br>echo &quot;Emergency service recovery completed!&quot;</pre><h3>Conclusion</h3><p>This comprehensive system architecture guide presents a robust, scalable, and production-ready infrastructure solution designed to meet the demands of modern distributed applications. The multi-layered approach, combining load balancing, API gateway management, container orchestration, message queuing, and a diversified data layer, provides the foundation for building resilient and high-performance systems.</p><h3>Key Takeaways</h3><p>The architecture delivers several critical benefits that make it suitable for enterprise-grade applications. The load balancer and API gateway layers ensure efficient traffic distribution and centralized request management, while the container orchestration platform provides the flexibility and scalability needed for dynamic workload management. The integration of message queuing systems enables asynchronous processing and loose coupling between components, improving overall system resilience.</p><p>The data layer strategy, incorporating PostgreSQL for transactional data, MongoDB for document storage, and Redis for caching, provides optimal performance across different data access patterns. This polyglot persistence approach allows each component to leverage the most appropriate storage technology for its specific requirements.</p><h3>Implementation Considerations</h3><p>Successful implementation of this architecture requires careful attention to configuration management, monitoring, and security practices. The modular design allows for incremental adoption and provides clear upgrade paths as system requirements evolve. Organizations should focus on establishing proper CI/CD pipelines, implementing comprehensive monitoring solutions, and maintaining strict security protocols throughout all layers.</p><h3>Future Scalability</h3><p>The proposed architecture is designed with growth in mind. The containerized approach and horizontal scaling capabilities ensure that the system can adapt to increasing loads and expanding feature requirements. The message queue infrastructure provides the foundation for event-driven architectures and real-time processing capabilities that can support advanced use cases as they emerge.</p><p>This architecture serves as a solid foundation for organizations looking to build reliable, scalable, and maintainable distributed systems that can support both current needs and future growth objectives.</p><p><strong>For technical consultation and implementation assistance, please contact: </strong><a href="/service/mailto:dikhitadarsh@gmail.com"><strong>dikhitadarsh@gmail.com</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=888392420808" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[QA Agentic Flow — Series -2]]></title>
            <link>https://medium.com/@to.adarshsingh/qa-agentic-flow-series-2-0beee78a14d7?source=rss-73f6989daa12------2</link>
            <guid isPermaLink="false">https://medium.com/p/0beee78a14d7</guid>
            <dc:creator><![CDATA[Adarsh Singh Dikhit]]></dc:creator>
            <pubDate>Sun, 25 May 2025 12:43:31 GMT</pubDate>
            <atom:updated>2025-05-25T12:43:31.082Z</atom:updated>
            <content:encoded><![CDATA[<h3>QA Agentic Flow — Series -2</h3><p><strong>5. Deployment Architecture</strong></p><ul><li>Infrastructure overview with diagrams</li><li>Kubernetes deployment configurations (integrated with your provided YAML files)</li><li>Container orchestration setup</li><li>Cloud provider specific deployments (AWS/Azure)</li></ul><p><strong>6. Configuration &amp; Setup</strong></p><ul><li>Multi-stage Dockerfile for production builds</li><li>Docker Compose for local development</li><li>Build and deployment scripts</li><li>Environment configuration files</li></ul><p><strong>7. API Documentation</strong></p><ul><li>Complete REST API reference</li><li>Authentication methods (JWT, API keys)</li><li>Core endpoints (agents, tasks, execution)</li><li>WebSocket API for real-time updates</li><li>Request/response examples</li></ul><p><strong>8. Monitoring &amp; Observability</strong></p><ul><li>Prometheus metrics configuration</li><li>Structured logging with JSON format</li><li>Distributed tracing with Jaeger</li><li>Custom alerting rules</li><li>ELK stack integration</li></ul><h3>5. Deployment Architecture</h3><h3>5.1 Infrastructure Overview</h3><p>The QA Agentic Flow system is designed for scalable, cloud-native deployment with the following architectural patterns:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*brWxGjyn-BgtQqwaqpmJ0A.png" /></figure><h3>5.2 Container Orchestration</h3><h4>Kubernetes Deployment Configuration</h4><pre># deployment.yaml<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: qa-agent-controller<br>  namespace: qa-agentic<br>spec:<br>  replicas: 3<br>  selector:<br>    matchLabels:<br>      app: qa-agent-controller<br>  template:<br>    metadata:<br>      labels:<br>        app: qa-agent-controller<br>    spec:<br>      containers:<br>      - name: agent-controller<br>        image: qa-agentic/controller:latest<br>        ports:<br>        - containerPort: 8080<br>        env:<br>        - name: DATABASE_URL<br>          valueFrom:<br>            secretKeyRef:<br>              name: qa-secrets<br>              key: database-url<br>        - name: REDIS_URL<br>          valueFrom:<br>            secretKeyRef:<br>              name: qa-secrets<br>              key: redis-url<br>        resources:<br>          requests:<br>            memory: &quot;256Mi&quot;<br>            cpu: &quot;250m&quot;<br>          limits:<br>            memory: &quot;512Mi&quot;<br>            cpu: &quot;500m&quot;<br>        livenessProbe:<br>          httpGet:<br>            path: /health<br>            port: 8080<br>          initialDelaySeconds: 30<br>          periodSeconds: 10<br>        readinessProbe:<br>          httpGet:<br>            path: /ready<br>            port: 8080<br>          initialDelaySeconds: 5<br>          periodSeconds: 5<br>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: qa-agent-controller-service<br>spec:<br>  selector:<br>    app: qa-agent-controller<br>  ports:<br>  - protocol: TCP<br>    port: 80<br>    targetPort: 8080<br>  type: LoadBalancer</pre><h4>Horizontal Pod Autoscaler</h4><pre># hpa.yaml<br>apiVersion: autoscaling/v2<br>kind: HorizontalPodAutoscaler<br>metadata:<br>  name: qa-agent-controller-hpa<br>spec:<br>  scaleTargetRef:<br>    apiVersion: apps/v1<br>    kind: Deployment<br>    name: qa-agent-controller<br>  minReplicas: 3<br>  maxReplicas: 20<br>  metrics:<br>  - type: Resource<br>    resource:<br>      name: cpu<br>      target:<br>        type: Utilization<br>        averageUtilization: 70<br>  - type: Resource<br>    resource:<br>      name: memory<br>      target:<br>        type: Utilization<br>        averageUtilization: 80</pre><h3>5.3 Cloud Provider Specific Deployments</h3><h4>AWS Deployment</h4><pre># aws-deployment.yaml<br>apiVersion: v1<br>kind: ConfigMap<br>metadata:<br>  name: aws-config<br>data:<br>  AWS_REGION: &quot;us-west-2&quot;<br>  SQS_QUEUE_URL: &quot;https://sqs.us-west-2.amazonaws.com/123456789/qa-agent-queue&quot;<br>  S3_BUCKET: &quot;qa-agentic-artifacts&quot;<br>---<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: qa-agent-aws<br>spec:<br>  template:<br>    spec:<br>      serviceAccountName: qa-agent-sa<br>      containers:<br>      - name: agent<br>        image: qa-agentic/agent:aws-latest<br>        envFrom:<br>        - configMapRef:<br>            name: aws-config<br>        - secretRef:<br>            name: aws-secrets</pre><h4>Azure Deployment</h4><pre># azure-deployment.yaml<br>apiVersion: v1<br>kind: ConfigMap<br>metadata:<br>  name: azure-config<br>data:<br>  AZURE_REGION: &quot;eastus&quot;<br>  SERVICE_BUS_NAMESPACE: &quot;qa-agentic-sb&quot;<br>  STORAGE_ACCOUNT: &quot;qaagenticstorage&quot;<br>---<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: qa-agent-azure<br>spec:<br>  template:<br>    spec:<br>      containers:<br>      - name: agent<br>        image: qa-agentic/agent:azure-latest<br>        envFrom:<br>        - configMapRef:<br>            name: azure-config<br>        - secretRef:<br>            name: azure-secrets</pre><h3>6. Configuration &amp; Setup</h3><h3>6.1 Environment Configuration</h3><h4>Production Configuration</h4><pre># config/production.yaml<br>server:<br>  port: 8080<br>  host: &quot;0.0.0.0&quot;<br>  <br>database:<br>  primary:<br>    host: &quot;${DB_HOST}&quot;<br>    port: 5432<br>    database: &quot;${DB_NAME}&quot;<br>    username: &quot;${DB_USER}&quot;<br>    password: &quot;${DB_PASSWORD}&quot;<br>    ssl_mode: &quot;require&quot;<br>    pool_size: 20<br>    max_connections: 100<br>  <br>  cache:<br>    redis:<br>      host: &quot;${REDIS_HOST}&quot;<br>      port: 6379<br>      password: &quot;${REDIS_PASSWORD}&quot;<br>      db: 0<br>      pool_size: 10<br>  <br>  document_store:<br>    mongodb:<br>      uri: &quot;${MONGODB_URI}&quot;<br>      database: &quot;qa_agentic&quot;<br>      collection: &quot;test_artifacts&quot;<br><br>messaging:<br>  broker: &quot;${MESSAGE_BROKER}&quot; # redis, rabbitmq, sqs, servicebus<br>  redis:<br>    host: &quot;${REDIS_HOST}&quot;<br>    port: 6379<br>    password: &quot;${REDIS_PASSWORD}&quot;<br>  rabbitmq:<br>    url: &quot;${RABBITMQ_URL}&quot;<br>    exchange: &quot;qa_agentic&quot;<br>    queue: &quot;agent_tasks&quot;<br>  aws_sqs:<br>    region: &quot;${AWS_REGION}&quot;<br>    queue_url: &quot;${SQS_QUEUE_URL}&quot;<br>  azure_servicebus:<br>    connection_string: &quot;${SERVICEBUS_CONNECTION_STRING}&quot;<br>    queue_name: &quot;agent-tasks&quot;<br><br>agents:<br>  max_concurrent: 10<br>  timeout_seconds: 300<br>  retry_attempts: 3<br>  retry_delay_seconds: 5<br>  <br>llm:<br>  provider: &quot;${LLM_PROVIDER}&quot; # openai, anthropic, azure, bedrock<br>  openai:<br>    api_key: &quot;${OPENAI_API_KEY}&quot;<br>    model: &quot;gpt-4-turbo&quot;<br>    max_tokens: 4096<br>  anthropic:<br>    api_key: &quot;${ANTHROPIC_API_KEY}&quot;<br>    model: &quot;claude-3-sonnet-20240229&quot;<br>    max_tokens: 4096<br>  azure_openai:<br>    endpoint: &quot;${AZURE_OPENAI_ENDPOINT}&quot;<br>    api_key: &quot;${AZURE_OPENAI_API_KEY}&quot;<br>    deployment_name: &quot;${AZURE_OPENAI_DEPLOYMENT}&quot;<br>  aws_bedrock:<br>    region: &quot;${AWS_REGION}&quot;<br>    model_id: &quot;anthropic.claude-3-sonnet-20240229-v1:0&quot;<br><br>security:<br>  jwt:<br>    secret: &quot;${JWT_SECRET}&quot;<br>    expiry_hours: 24<br>  api_keys:<br>    enabled: true<br>    header_name: &quot;X-API-Key&quot;<br>  rate_limiting:<br>    enabled: true<br>    requests_per_minute: 100<br>    burst_size: 20<br><br>logging:<br>  level: &quot;info&quot;<br>  format: &quot;json&quot;<br>  output: &quot;stdout&quot;<br>  <br>monitoring:<br>  metrics:<br>    enabled: true<br>    port: 9090<br>    path: &quot;/metrics&quot;<br>  tracing:<br>    enabled: true<br>    jaeger_endpoint: &quot;${JAEGER_ENDPOINT}&quot;<br>      health_checks:<br>      enabled: true<br>      endpoint: &quot;/health&quot;<br>      interval_seconds: 30</pre><h3>6.2 Docker Configuration</h3><h4>Multi-stage Dockerfile</h4><pre># Dockerfile<br>FROM python:3.11-slim as builder<br><br># Set working directory<br>WORKDIR /app<br><br># Install system dependencies<br>RUN apt-get update &amp;&amp; apt-get install -y \<br>    build-essential \<br>    curl \<br>    &amp;&amp; rm -rf /var/lib/apt/lists/*<br><br># Copy requirements and install Python dependencies<br>COPY requirements.txt .<br>RUN pip install --no-cache-dir --user -r requirements.txt<br><br># Production stage<br>FROM python:3.11-slim as production<br><br># Create non-root user<br>RUN groupadd -r qauser &amp;&amp; useradd -r -g qauser qauser<br><br># Set working directory<br>WORKDIR /app<br><br># Install runtime dependencies<br>RUN apt-get update &amp;&amp; apt-get install -y \<br>    curl \<br>    &amp;&amp; rm -rf /var/lib/apt/lists/*<br><br># Copy Python packages from builder stage<br>COPY --from=builder /root/.local /home/qauser/.local<br><br># Copy application code<br>COPY --chown=qauser:qauser . .<br><br># Make sure scripts are executable<br>RUN chmod +x /app/scripts/*.sh<br><br># Switch to non-root user<br>USER qauser<br><br># Set PATH to include user packages<br>ENV PATH=/home/qauser/.local/bin:$PATH<br><br># Health check<br>HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \<br>    CMD curl -f http://localhost:8000/health || exit 1<br><br># Expose port<br>EXPOSE 8000<br><br># Default command<br>CMD [&quot;python&quot;, &quot;-m&quot;, &quot;qa_agentic.main&quot;]</pre><h4>Docker Compose for Local Development</h4><pre># docker-compose.yml<br>version: &#39;3.8&#39;<br><br>services:<br>  postgres:<br>    image: postgres:14<br>    environment:<br>      POSTGRES_DB: qa_db<br>      POSTGRES_USER: qa_user<br>      POSTGRES_PASSWORD: qa_password<br>    ports:<br>      - &quot;5432:5432&quot;<br>    volumes:<br>      - postgres_data:/var/lib/postgresql/data<br>      - ./init.sql:/docker-entrypoint-initdb.d/init.sql<br>    healthcheck:<br>      test: [&quot;CMD-SHELL&quot;, &quot;pg_isready -U qa_user -d qa_db&quot;]<br>      interval: 10s<br>      timeout: 5s<br>      retries: 5<br><br>  redis:<br>    image: redis:7-alpine<br>    ports:<br>      - &quot;6379:6379&quot;<br>    volumes:<br>      - redis_data:/data<br>    healthcheck:<br>      test: [&quot;CMD&quot;, &quot;redis-cli&quot;, &quot;ping&quot;]<br>      interval: 10s<br>      timeout: 5s<br>      retries: 3<br><br>  elasticsearch:<br>    image: elasticsearch:8.8.0<br>    environment:<br>      - discovery.type=single-node<br>      - xpack.security.enabled=false<br>      - &quot;ES_JAVA_OPTS=-Xms512m -Xmx512m&quot;<br>    ports:<br>      - &quot;9200:9200&quot;<br>    volumes:<br>      - elasticsearch_data:/usr/share/elasticsearch/data<br>    healthcheck:<br>      test: [&quot;CMD-SHELL&quot;, &quot;curl -f http://localhost:9200/_cluster/health || exit 1&quot;]<br>      interval: 30s<br>      timeout: 10s<br>      retries: 5<br><br>  qa-api:<br>    build: .<br>    ports:<br>      - &quot;8000:8000&quot;<br>    environment:<br>      - OPENAI_API_KEY=${OPENAI_API_KEY}<br>      - REDIS_URL=redis://redis:6379<br>      - POSTGRES_URL=postgresql://qa_user:qa_password@postgres:5432/qa_db<br>      - ELASTICSEARCH_URL=http://elasticsearch:9200<br>      - LOG_LEVEL=INFO<br>    depends_on:<br>      postgres:<br>        condition: service_healthy<br>      redis:<br>        condition: service_healthy<br>      elasticsearch:<br>        condition: service_healthy<br>    volumes:<br>      - ./logs:/app/logs<br>    restart: unless-stopped<br><br>  nginx:<br>    image: nginx:alpine<br>    ports:<br>      - &quot;80:80&quot;<br>      - &quot;443:443&quot;<br>    volumes:<br>      - ./nginx.conf:/etc/nginx/nginx.conf<br>      - ./ssl:/etc/nginx/ssl<br>    depends_on:<br>      - qa-api<br>    restart: unless-stopped<br><br>volumes:<br>  postgres_data:<br>  redis_data:<br>  elasticsearch_data:</pre><h4>Build and Deployment Scripts</h4><pre>#!/bin/bash<br># scripts/build.sh<br><br>set -e<br><br># Configuration<br>IMAGE_NAME=&quot;qa-agentic-flow&quot;<br>TAG=${1:-latest}<br>REGISTRY=${REGISTRY:-&quot;your-registry.com&quot;}<br><br>echo &quot;Building Docker image: ${IMAGE_NAME}:${TAG}&quot;<br><br># Build the image<br>docker build -t ${IMAGE_NAME}:${TAG} .<br><br># Tag for registry<br>docker tag ${IMAGE_NAME}:${TAG} ${REGISTRY}/${IMAGE_NAME}:${TAG}<br><br># Push to registry<br>if [ &quot;$PUSH&quot; = &quot;true&quot; ]; then<br>    echo &quot;Pushing to registry...&quot;<br>    docker push ${REGISTRY}/${IMAGE_NAME}:${TAG}<br>fi<br><br>echo &quot;Build completed successfully!&quot;</pre><pre>#!/bin/bash<br># scripts/deploy.sh<br><br>set -e<br><br>NAMESPACE=&quot;qa-agentic-flow&quot;<br>ENVIRONMENT=${1:-development}<br><br>echo &quot;Deploying to ${ENVIRONMENT} environment...&quot;<br><br># Apply Kubernetes manifests based on the uploaded configuration<br>kubectl apply -f k8s/namespace.yaml<br>kubectl apply -f k8s/configmap.yaml<br>kubectl apply -f k8s/postgres.yaml<br>kubectl apply -f k8s/redis.yaml<br>kubectl apply -f k8s/elasticsearch.yaml<br>kubectl apply -f k8s/qa-api.yaml<br>kubectl apply -f k8s/ingress.yaml<br><br># Wait for deployment to be ready<br>kubectl rollout status deployment/qa-api -n ${NAMESPACE} --timeout=300s<br><br>echo &quot;Deployment completed successfully!&quot;</pre><h3>7. API Documentation</h3><h3>7.1 Authentication</h3><h4>API Key Authentication</h4><pre>GET /api/v1/agents<br>Authorization: Bearer your-jwt-token<br>X-API-Key: your-api-key<br>Content-Type: application/json</pre><h4>JWT Token Endpoint</h4><pre>POST /api/v1/auth/token<br>Content-Type: application/json<br><br>{<br>  &quot;username&quot;: &quot;admin&quot;,<br>  &quot;password&quot;: &quot;password&quot;<br>}</pre><p>Response:</p><pre>{<br>  &quot;access_token&quot;: &quot;eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...&quot;,<br>  &quot;token_type&quot;: &quot;bearer&quot;,<br>  &quot;expires_in&quot;: 3600<br>}</pre><h3>7.2 Core API Endpoints</h3><h4>Agent Management</h4><p><strong>Create Agent</strong></p><pre>POST /api/v1/agents<br>Content-Type: application/json<br>Authorization: Bearer {token}<br><br>{<br>  &quot;name&quot;: &quot;Test Generation Agent&quot;,<br>  &quot;type&quot;: &quot;test_generator&quot;,<br>  &quot;configuration&quot;: {<br>    &quot;framework&quot;: &quot;pytest&quot;,<br>    &quot;coverage_threshold&quot;: 80,<br>    &quot;test_types&quot;: [&quot;unit&quot;, &quot;integration&quot;]<br>  },<br>  &quot;capabilities&quot;: [<br>    &quot;code_analysis&quot;,<br>    &quot;test_generation&quot;,<br>    &quot;test_execution&quot;<br>  ]<br>}</pre><p>Response:</p><pre>{<br>  &quot;id&quot;: &quot;agent_123&quot;,<br>  &quot;name&quot;: &quot;Test Generation Agent&quot;,<br>  &quot;type&quot;: &quot;test_generator&quot;,<br>  &quot;status&quot;: &quot;active&quot;,<br>  &quot;created_at&quot;: &quot;2024-01-15T10:30:00Z&quot;,<br>  &quot;configuration&quot;: {<br>    &quot;framework&quot;: &quot;pytest&quot;,<br>    &quot;coverage_threshold&quot;: 80,<br>    &quot;test_types&quot;: [&quot;unit&quot;, &quot;integration&quot;]<br>  }<br>}</pre><p><strong>List Agents</strong></p><pre>GET /api/v1/agents?page=1&amp;limit=10&amp;status=active<br>Authorization: Bearer {token}</pre><p>Response:</p><pre>{<br>  &quot;agents&quot;: [<br>    {<br>      &quot;id&quot;: &quot;agent_123&quot;,<br>      &quot;name&quot;: &quot;Test Generation Agent&quot;,<br>      &quot;type&quot;: &quot;test_generator&quot;,<br>      &quot;status&quot;: &quot;active&quot;,<br>      &quot;last_activity&quot;: &quot;2024-01-15T11:45:00Z&quot;<br>    }<br>  ],<br>  &quot;pagination&quot;: {<br>    &quot;page&quot;: 1,<br>    &quot;limit&quot;: 10,<br>    &quot;total&quot;: 25,<br>    &quot;pages&quot;: 3<br>  }<br>}</pre><p><strong>Get Agent Details</strong></p><pre>GET /api/v1/agents/{agent_id}<br>Authorization: Bearer {token}</pre><p><strong>Update Agent</strong></p><pre>PUT /api/v1/agents/{agent_id}<br>Content-Type: application/json<br>Authorization: Bearer {token}<br><br>{<br>  &quot;configuration&quot;: {<br>    &quot;coverage_threshold&quot;: 85<br>  }<br>}</pre><p><strong>Delete Agent</strong></p><pre>DELETE /api/v1/agents/{agent_id}<br>Authorization: Bearer {token}</pre><h4>Task Management</h4><p><strong>Create Task</strong></p><pre>POST /api/v1/tasks<br>Content-Type: application/json<br>Authorization: Bearer {token}<br><br>{<br>  &quot;type&quot;: &quot;generate_tests&quot;,<br>  &quot;priority&quot;: &quot;high&quot;,<br>  &quot;agent_id&quot;: &quot;agent_123&quot;,<br>  &quot;parameters&quot;: {<br>    &quot;source_code&quot;: &quot;def calculate_sum(a, b):\n    return a + b&quot;,<br>    &quot;test_framework&quot;: &quot;pytest&quot;,<br>    &quot;coverage_requirements&quot;: {<br>      &quot;minimum&quot;: 90,<br>      &quot;branches&quot;: true<br>    }<br>  },<br>  &quot;timeout_seconds&quot;: 300<br>}</pre><p>Response:</p><pre>{<br>  &quot;task_id&quot;: &quot;task_456&quot;,<br>  &quot;status&quot;: &quot;queued&quot;,<br>  &quot;created_at&quot;: &quot;2024-01-15T12:00:00Z&quot;,<br>  &quot;estimated_completion&quot;: &quot;2024-01-15T12:05:00Z&quot;<br>}</pre><p><strong>Get Task Status</strong></p><pre>GET /api/v1/tasks/{task_id}<br>Authorization: Bearer {token}</pre><p>Response:</p><pre>{<br>  &quot;task_id&quot;: &quot;task_456&quot;,<br>  &quot;status&quot;: &quot;completed&quot;,<br>  &quot;progress&quot;: 100,<br>  &quot;created_at&quot;: &quot;2024-01-15T12:00:00Z&quot;,<br>  &quot;completed_at&quot;: &quot;2024-01-15T12:04:30Z&quot;,<br>  &quot;result&quot;: {<br>    &quot;tests_generated&quot;: 5,<br>    &quot;coverage_achieved&quot;: 95.2,<br>    &quot;test_files&quot;: [<br>      {<br>        &quot;filename&quot;: &quot;test_calculate_sum.py&quot;,<br>        &quot;content&quot;: &quot;import pytest\n\ndef test_calculate_sum_positive():\n    assert calculate_sum(2, 3) == 5&quot;<br>      }<br>    ]<br>  }<br>}</pre><p><strong>Cancel Task</strong></p><pre>DELETE /api/v1/tasks/{task_id}<br>Authorization: Bearer {token}</pre><h4>Test Execution</h4><p><strong>Execute Tests</strong></p><pre>POST /api/v1/execution/run<br>Content-Type: application/json<br>Authorization: Bearer {token}<br><br>{<br>  &quot;test_suite_id&quot;: &quot;suite_789&quot;,<br>  &quot;environment&quot;: &quot;staging&quot;,<br>  &quot;parallel&quot;: true,<br>  &quot;max_workers&quot;: 4,<br>  &quot;timeout_minutes&quot;: 30,<br>  &quot;filters&quot;: {<br>    &quot;tags&quot;: [&quot;smoke&quot;, &quot;regression&quot;],<br>    &quot;exclude_tags&quot;: [&quot;slow&quot;]<br>  }<br>}</pre><p><strong>Get Execution Results</strong></p><pre>GET /api/v1/execution/{execution_id}/results<br>Authorization: Bearer {token}</pre><p>Response:</p><pre>{<br>  &quot;execution_id&quot;: &quot;exec_101&quot;,<br>  &quot;status&quot;: &quot;completed&quot;,<br>  &quot;summary&quot;: {<br>    &quot;total_tests&quot;: 150,<br>    &quot;passed&quot;: 145,<br>    &quot;failed&quot;: 3,<br>    &quot;skipped&quot;: 2,<br>    &quot;duration_seconds&quot;: 125.5,<br>    &quot;coverage_percentage&quot;: 87.3<br>  },<br>  &quot;test_results&quot;: [<br>    {<br>      &quot;test_name&quot;: &quot;test_user_authentication&quot;,<br>      &quot;status&quot;: &quot;passed&quot;,<br>      &quot;duration_seconds&quot;: 0.5,<br>      &quot;assertions&quot;: 3<br>    },<br>    {<br>      &quot;test_name&quot;: &quot;test_invalid_login&quot;,<br>      &quot;status&quot;: &quot;failed&quot;,<br>      &quot;duration_seconds&quot;: 1.2,<br>      &quot;error_message&quot;: &quot;AssertionError: Expected 401, got 500&quot;,<br>      &quot;stack_trace&quot;: &quot;...&quot;<br>    }<br>  ]<br>}</pre><h3>7.3 WebSocket API</h3><h4>Real-time Updates</h4><pre>// WebSocket connection for real-time updates<br>const ws = new WebSocket(&#39;wss://qa-api.yourdomain.com/ws/tasks&#39;);<br><br>ws.onopen = function(event) {<br>    // Subscribe to specific task updates<br>    ws.send(JSON.stringify({<br>        action: &#39;subscribe&#39;,<br>        task_id: &#39;task_456&#39;<br>    }));<br>};<br><br>ws.onmessage = function(event) {<br>    const data = JSON.parse(event.data);<br>    console.log(&#39;Task update:&#39;, data);<br>    // {<br>    //   &quot;task_id&quot;: &quot;task_456&quot;,<br>    //   &quot;status&quot;: &quot;in_progress&quot;,<br>    //   &quot;progress&quot;: 45,<br>    //   &quot;message&quot;: &quot;Analyzing source code...&quot;<br>    // }<br>};</pre><h3>8. Monitoring &amp; Observability</h3><h3>8.1 Metrics Collection</h3><h4>Prometheus Configuration</h4><pre># prometheus.yml<br>global:<br>  scrape_interval: 15s<br>  evaluation_interval: 15s<br><br>rule_files:<br>  - &quot;qa_agentic_rules.yml&quot;<br><br>scrape_configs:<br>  - job_name: &#39;qa-agentic-api&#39;<br>    static_configs:<br>      - targets: [&#39;qa-api-service:9090&#39;]<br>    metrics_path: &#39;/metrics&#39;<br>    scrape_interval: 10s<br><br>  - job_name: &#39;postgres-exporter&#39;<br>    static_configs:<br>      - targets: [&#39;postgres-exporter:9187&#39;]<br><br>  - job_name: &#39;redis-exporter&#39;<br>    static_configs:<br>      - targets: [&#39;redis-exporter:9121&#39;]<br><br>alerting:<br>  alertmanagers:<br>    - static_configs:<br>        - targets:<br>          - alertmanager:9093</pre><h4>Custom Metrics</h4><pre># monitoring/metrics.py<br>from prometheus_client import Counter, Histogram, Gauge, start_http_server<br><br># Request metrics<br>REQUEST_COUNT = Counter(<br>    &#39;qa_agentic_requests_total&#39;,<br>    &#39;Total number of requests&#39;,<br>    [&#39;method&#39;, &#39;endpoint&#39;, &#39;status&#39;]<br>)<br><br>REQUEST_DURATION = Histogram(<br>    &#39;qa_agentic_request_duration_seconds&#39;,<br>    &#39;Request duration in seconds&#39;,<br>    [&#39;method&#39;, &#39;endpoint&#39;]<br>)<br><br># Agent metrics<br>ACTIVE_AGENTS = Gauge(<br>    &#39;qa_agentic_active_agents&#39;,<br>    &#39;Number of active agents&#39;<br>)<br><br>TASK_QUEUE_SIZE = Gauge(<br>    &#39;qa_agentic_task_queue_size&#39;,<br>    &#39;Number of tasks in queue&#39;<br>)<br><br>TEST_EXECUTION_TIME = Histogram(<br>    &#39;qa_agentic_test_execution_seconds&#39;,<br>    &#39;Test execution time in seconds&#39;,<br>    [&#39;test_type&#39;, &#39;framework&#39;]<br>)<br><br># Usage example<br>def track_request(method, endpoint):<br>    def decorator(func):<br>        def wrapper(*args, **kwargs):<br>            start_time = time.time()<br>            try:<br>                result = func(*args, **kwargs)<br>                REQUEST_COUNT.labels(<br>                    method=method, <br>                    endpoint=endpoint, <br>                    status=&#39;success&#39;<br>                ).inc()<br>                return result<br>            except Exception as e:<br>                REQUEST_COUNT.labels(<br>                    method=method, <br>                    endpoint=endpoint, <br>                    status=&#39;error&#39;<br>                ).inc()<br>                raise<br>            finally:<br>                REQUEST_DURATION.labels(<br>                    method=method, <br>                    endpoint=endpoint<br>                ).observe(time.time() - start_time)<br>        return wrapper<br>    return decorator</pre><h3>8.2 Logging Configuration</h3><h4>Structured Logging</h4><pre># logging/config.py<br>import logging<br>import json<br>from datetime import datetime<br>from typing import Dict, Any<br><br>class JSONFormatter(logging.Formatter):<br>    def format(self, record: logging.LogRecord) -&gt; str:<br>        log_entry = {<br>            &#39;timestamp&#39;: datetime.utcnow().isoformat(),<br>            &#39;level&#39;: record.levelname,<br>            &#39;logger&#39;: record.name,<br>            &#39;message&#39;: record.getMessage(),<br>            &#39;module&#39;: record.module,<br>            &#39;function&#39;: record.funcName,<br>            &#39;line&#39;: record.lineno<br>        }<br>        <br>        # Add extra fields<br>        if hasattr(record, &#39;user_id&#39;):<br>            log_entry[&#39;user_id&#39;] = record.user_id<br>        if hasattr(record, &#39;task_id&#39;):<br>            log_entry[&#39;task_id&#39;] = record.task_id<br>        if hasattr(record, &#39;agent_id&#39;):<br>            log_entry[&#39;agent_id&#39;] = record.agent_id<br>            <br>        # Add exception info if present<br>        if record.exc_info:<br>            log_entry[&#39;exception&#39;] = self.formatException(record.exc_info)<br>            <br>        return json.dumps(log_entry)<br><br># Configure logging<br>def setup_logging():<br>    logging.basicConfig(<br>        level=logging.INFO,<br>        format=&#39;%(message)s&#39;,<br>        handlers=[<br>            logging.StreamHandler(),<br>            logging.FileHandler(&#39;/app/logs/qa_agentic.log&#39;)<br>        ]<br>    )<br>    <br>    # Apply JSON formatter<br>    for handler in logging.root.handlers:<br>        handler.setFormatter(JSONFormatter())</pre><h4>Log Aggregation with ELK Stack</h4><pre># logging/filebeat.yml<br>filebeat.inputs:<br>- type: log<br>  enabled: true<br>  paths:<br>    - /app/logs/*.log<br>  fields:<br>    service: qa-agentic-flow<br>    environment: production<br>  fields_under_root: true<br>  json.keys_under_root: true<br>  json.add_error_key: true<br><br>output.elasticsearch:<br>  hosts: [&quot;elasticsearch:9200&quot;]<br>  index: &quot;qa-agentic-logs-%{+yyyy.MM.dd}&quot;<br><br>processors:<br>  - add_host_metadata:<br>      when.not.contains.tags: forwarded</pre><h3>8.3 Distributed Tracing</h3><h4>Jaeger Configuration</h4><pre># tracing/config.py<br>from jaeger_client import Config<br>from flask_opentracing import FlaskTracing<br>import opentracing<br><br>def init_tracer(service_name=&#39;qa-agentic-api&#39;):<br>    config = Config(<br>        config={<br>            &#39;sampler&#39;: {<br>                &#39;type&#39;: &#39;const&#39;,<br>                &#39;param&#39;: 1,<br>            },<br>            &#39;logging&#39;: True,<br>            &#39;reporter_batch_size&#39;: 1,<br>        },<br>        service_name=service_name,<br>        validate=True,<br>    )<br>    return config.initialize_tracer()<br><br># Usage in Flask app<br>tracer = init_tracer()<br>opentracing.tracer = tracer<br>flask_tracer = FlaskTracing(tracer, True, app)<br><br># Custom span creation<br>def trace_agent_execution(agent_id: str, task_id: str):<br>    def decorator(func):<br>        def wrapper(*args, **kwargs):<br>            with tracer.start_span(f&#39;agent_execution&#39;) as span:<br>                span.set_tag(&#39;agent_id&#39;, agent_id)<br>                span.set_tag(&#39;task_id&#39;, task_id)<br>                try:<br>                    result = func(*args, **kwargs)<br>                    span.set_tag(&#39;status&#39;, &#39;success&#39;)<br>                    return result<br>                except Exception as e:<br>                    span.set_tag(&#39;status&#39;, &#39;error&#39;)<br>                    span.set_tag(&#39;error_message&#39;, str(e))<br>                    raise<br>        return wrapper<br>    return decorator</pre><h3>8.4 Alerting Rules</h3><pre># alerts/qa_agentic_rules.yml<br>groups:<br>- name: qa_agentic_alerts<br>  rules:<br>  - alert: HighErrorRate<br>    expr: rate(qa_agentic_requests_total{status=&quot;error&quot;}[5m]) &gt; 0.1<br>    for: 2m<br>    labels:<br>      severity: warning<br>      service: qa-agentic<br>    annotations:<br>      summary: &quot;High error rate detected&quot;<br>      description: &quot;Error rate is {{ $value }} errors per second&quot;<br><br>  - alert: TaskQueueBacklog<br>    expr: qa_agentic_task_queue_size &gt; 100<br>    for: 5m<br>    labels:<br>      severity: critical<br>      service: qa-agentic<br>    annotations:<br>      summary: &quot;Task queue backlog detected&quot;<br>      description: &quot;Task queue has {{ $value }} pending tasks&quot;<br><br>  - alert: AgentTimeout<br>    expr: increase(qa_agentic_task_timeouts_total[10m]) &gt; 5<br>    for: 2m<br>    labels:<br>      severity: warning<br>      service: qa-agentic<br>    annotations:<br>      summary: &quot;Multiple agent timeouts detected&quot;<br>      description: &quot;{{ $value }} agent timeouts in the last 10 minutes&quot;<br><br>  - alert: DatabaseConnectionFailure<br>    expr: up{job=&quot;postgres-exporter&quot;} == 0<br>    for: 1m<br>    labels:<br>      severity: critical<br>      service: qa-agentic<br>    annotations:<br>      summary: &quot;Database connection failure&quot;<br>      description: &quot;PostgreSQL database is not responding&quot;</pre><h3>In this article, I will cover the following key sections:</h3><p><strong>5. Deployment Architecture</strong></p><p><strong>6. Configuration &amp; Setup</strong></p><p><strong>7. API Documentation</strong></p><p><strong>8. Monitoring &amp; Observability</strong></p><p>For the remaining sections, please refer to the other articles in this series:</p><p><strong>9. Security &amp; Compliance</strong></p><p><strong>10. Troubleshooting</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0beee78a14d7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[QA Agentic Flow — Series -1]]></title>
            <link>https://medium.com/@to.adarshsingh/qa-agentic-flow-series-1-3e76a581ec0a?source=rss-73f6989daa12------2</link>
            <guid isPermaLink="false">https://medium.com/p/3e76a581ec0a</guid>
            <dc:creator><![CDATA[Adarsh Singh Dikhit]]></dc:creator>
            <pubDate>Sun, 25 May 2025 12:23:54 GMT</pubDate>
            <atom:updated>2025-05-25T12:35:08.673Z</atom:updated>
            <content:encoded><![CDATA[<h3>QA Agentic Flow — Series -1</h3><h3>Table of Contents</h3><ol><li>Overview</li><li>System Architecture</li><li>Core Components</li><li>Implementation Code</li><li>Deployment Architecture</li><li>Configuration &amp; Setup</li><li>API Documentation</li><li>Monitoring &amp; Observability</li><li>Security &amp; Compliance</li><li>Troubleshooting</li></ol><h3>Overview</h3><p>The QA Agentic Flow is an intelligent quality assurance system that leverages AI agents to automate testing, bug detection, and quality validation processes. The system provides autonomous decision-making capabilities for comprehensive software quality management.</p><h3>Key Features</h3><ul><li><strong>Autonomous Test Generation</strong>: AI-driven test case creation and execution</li><li><strong>Intelligent Bug Detection</strong>: Advanced pattern recognition for quality issues</li><li><strong>Self-Healing Tests</strong>: Automatic test maintenance and updates</li><li><strong>Multi-Modal Testing</strong>: Support for UI, API, performance, and security testing</li><li><strong>Continuous Learning</strong>: Adaptive algorithms that improve over time</li><li><strong>Real-time Reporting</strong>: Live dashboards and notifications</li></ul><h3>System Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TO5c4TVWhD94C7ezqgiM1A.png" /></figure><h3>Component Interaction Flow</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zdlp4wMphHSWV8mr6p70QA.png" /></figure><h3>Core Components</h3><h3>1. Agent Orchestrator</h3><p>The central coordinator that manages AI agents and their interactions.</p><h3>2. AI Agents</h3><ul><li><strong>Test Generation Agent</strong>: Creates intelligent test cases</li><li><strong>Execution Agent</strong>: Runs tests across multiple environments</li><li><strong>Analysis Agent</strong>: Evaluates results and identifies patterns</li><li><strong>Reporting Agent</strong>: Generates insights and recommendations</li></ul><h3>3. Test Engine</h3><p>Core testing framework supporting multiple test types and protocols.</p><h3>4. ML Service</h3><p>Machine learning capabilities for pattern recognition and predictive analytics.</p><h3>Implementation Code</h3><h3>Agent Orchestrator</h3><pre># agent_orchestrator.py<br>import asyncio<br>import logging<br>from typing import Dict, List, Optional<br>from dataclasses import dataclass<br>from enum import Enum<br>import uuid<br>from datetime import datetime<br><br>class AgentType(Enum):<br>    TEST_GENERATION = &quot;test_generation&quot;<br>    EXECUTION = &quot;execution&quot;<br>    ANALYSIS = &quot;analysis&quot;<br>    REPORTING = &quot;reporting&quot;<br><br>class TaskStatus(Enum):<br>    PENDING = &quot;pending&quot;<br>    IN_PROGRESS = &quot;in_progress&quot;<br>    COMPLETED = &quot;completed&quot;<br>    FAILED = &quot;failed&quot;<br><br>@dataclass<br>class Task:<br>    id: str<br>    agent_type: AgentType<br>    payload: Dict<br>    status: TaskStatus<br>    created_at: datetime<br>    completed_at: Optional[datetime] = None<br>    result: Optional[Dict] = None<br>    error: Optional[str] = None<br><br>class AgentOrchestrator:<br>    def __init__(self, config: Dict):<br>        self.config = config<br>        self.agents = {}<br>        self.task_queue = asyncio.Queue()<br>        self.active_tasks = {}<br>        self.logger = logging.getLogger(__name__)<br>        <br>    async def initialize(self):<br>        &quot;&quot;&quot;Initialize all agents and services.&quot;&quot;&quot;<br>        await self._initialize_agents()<br>        await self._start_task_processor()<br>        <br>    async def _initialize_agents(self):<br>        &quot;&quot;&quot;Initialize AI agents.&quot;&quot;&quot;<br>        from agents.test_generation_agent import TestGenerationAgent<br>        from agents.execution_agent import ExecutionAgent<br>        from agents.analysis_agent import AnalysisAgent<br>        from agents.reporting_agent import ReportingAgent<br>        <br>        self.agents[AgentType.TEST_GENERATION] = TestGenerationAgent(self.config)<br>        self.agents[AgentType.EXECUTION] = ExecutionAgent(self.config)<br>        self.agents[AgentType.ANALYSIS] = AnalysisAgent(self.config)<br>        self.agents[AgentType.REPORTING] = ReportingAgent(self.config)<br>        <br>        for agent in self.agents.values():<br>            await agent.initialize()<br>    <br>    async def submit_task(self, agent_type: AgentType, payload: Dict) -&gt; str:<br>        &quot;&quot;&quot;Submit a new task to the orchestrator.&quot;&quot;&quot;<br>        task = Task(<br>            id=str(uuid.uuid4()),<br>            agent_type=agent_type,<br>            payload=payload,<br>            status=TaskStatus.PENDING,<br>            created_at=datetime.utcnow()<br>        )<br>        <br>        await self.task_queue.put(task)<br>        self.active_tasks[task.id] = task<br>        <br>        self.logger.info(f&quot;Task {task.id} submitted for agent {agent_type.value}&quot;)<br>        return task.id<br>    <br>    async def _start_task_processor(self):<br>        &quot;&quot;&quot;Start the task processing loop.&quot;&quot;&quot;<br>        asyncio.create_task(self._process_tasks())<br>    <br>    async def _process_tasks(self):<br>        &quot;&quot;&quot;Process tasks from the queue.&quot;&quot;&quot;<br>        while True:<br>            try:<br>                task = await self.task_queue.get()<br>                await self._execute_task(task)<br>            except Exception as e:<br>                self.logger.error(f&quot;Error processing task: {e}&quot;)<br>                await asyncio.sleep(1)<br>    <br>    async def _execute_task(self, task: Task):<br>        &quot;&quot;&quot;Execute a specific task.&quot;&quot;&quot;<br>        try:<br>            task.status = TaskStatus.IN_PROGRESS<br>            agent = self.agents[task.agent_type]<br>            <br>            result = await agent.execute(task.payload)<br>            <br>            task.status = TaskStatus.COMPLETED<br>            task.result = result<br>            task.completed_at = datetime.utcnow()<br>            <br>            self.logger.info(f&quot;Task {task.id} completed successfully&quot;)<br>            <br>        except Exception as e:<br>            task.status = TaskStatus.FAILED<br>            task.error = str(e)<br>            task.completed_at = datetime.utcnow()<br>            <br>            self.logger.error(f&quot;Task {task.id} failed: {e}&quot;)<br>    <br>    async def get_task_status(self, task_id: str) -&gt; Optional[Task]:<br>        &quot;&quot;&quot;Get the status of a specific task.&quot;&quot;&quot;<br>        return self.active_tasks.get(task_id)<br>    <br>    async def start_qa_flow(self, project_id: str, trigger_data: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Start a complete QA flow for a project.&quot;&quot;&quot;<br>        flow_id = str(uuid.uuid4())<br>        <br>        # Step 1: Generate test cases<br>        test_gen_task_id = await self.submit_task(<br>            AgentType.TEST_GENERATION,<br>            {<br>                &quot;project_id&quot;: project_id,<br>                &quot;trigger_data&quot;: trigger_data,<br>                &quot;flow_id&quot;: flow_id<br>            }<br>        )<br>        <br>        # Wait for test generation to complete<br>        await self._wait_for_task_completion(test_gen_task_id)<br>        test_gen_result = self.active_tasks[test_gen_task_id].result<br>        <br>        # Step 2: Execute tests<br>        exec_task_id = await self.submit_task(<br>            AgentType.EXECUTION,<br>            {<br>                &quot;project_id&quot;: project_id,<br>                &quot;test_cases&quot;: test_gen_result[&quot;test_cases&quot;],<br>                &quot;flow_id&quot;: flow_id<br>            }<br>        )<br>        <br>        # Wait for execution to complete<br>        await self._wait_for_task_completion(exec_task_id)<br>        exec_result = self.active_tasks[exec_task_id].result<br>        <br>        # Step 3: Analyze results<br>        analysis_task_id = await self.submit_task(<br>            AgentType.ANALYSIS,<br>            {<br>                &quot;project_id&quot;: project_id,<br>                &quot;execution_results&quot;: exec_result,<br>                &quot;flow_id&quot;: flow_id<br>            }<br>        )<br>        <br>        # Wait for analysis to complete<br>        await self._wait_for_task_completion(analysis_task_id)<br>        analysis_result = self.active_tasks[analysis_task_id].result<br>        <br>        # Step 4: Generate report<br>        report_task_id = await self.submit_task(<br>            AgentType.REPORTING,<br>            {<br>                &quot;project_id&quot;: project_id,<br>                &quot;analysis_results&quot;: analysis_result,<br>                &quot;flow_id&quot;: flow_id<br>            }<br>        )<br>        <br>        # Wait for reporting to complete<br>        await self._wait_for_task_completion(report_task_id)<br>        report_result = self.active_tasks[report_task_id].result<br>        <br>        return {<br>            &quot;flow_id&quot;: flow_id,<br>            &quot;status&quot;: &quot;completed&quot;,<br>            &quot;results&quot;: {<br>                &quot;test_generation&quot;: test_gen_result,<br>                &quot;execution&quot;: exec_result,<br>                &quot;analysis&quot;: analysis_result,<br>                &quot;report&quot;: report_result<br>            }<br>        }<br>    <br>    async def _wait_for_task_completion(self, task_id: str, timeout: int = 300):<br>        &quot;&quot;&quot;Wait for a task to complete with timeout.&quot;&quot;&quot;<br>        start_time = datetime.utcnow()<br>        <br>        while True:<br>            task = self.active_tasks.get(task_id)<br>            if not task:<br>                raise Exception(f&quot;Task {task_id} not found&quot;)<br>            <br>            if task.status in [TaskStatus.COMPLETED, TaskStatus.FAILED]:<br>                if task.status == TaskStatus.FAILED:<br>                    raise Exception(f&quot;Task {task_id} failed: {task.error}&quot;)<br>                return<br>            <br>            elapsed = (datetime.utcnow() - start_time).total_seconds()<br>            if elapsed &gt; timeout:<br>                raise Exception(f&quot;Task {task_id} timed out after {timeout} seconds&quot;)<br>            <br>            await asyncio.sleep(1)</pre><h3>Test Generation Agent</h3><pre># agents/test_generation_agent.py<br>import asyncio<br>import logging<br>from typing import Dict, List<br>import openai<br>from dataclasses import dataclass<br>import json<br><br>@dataclass<br>class TestCase:<br>    id: str<br>    name: str<br>    description: str<br>    type: str<br>    steps: List[Dict]<br>    expected_result: str<br>    priority: str<br>    tags: List[str]<br><br>class TestGenerationAgent:<br>    def __init__(self, config: Dict):<br>        self.config = config<br>        self.openai_client = openai.AsyncOpenAI(<br>            api_key=config.get(&quot;openai_api_key&quot;)<br>        )<br>        self.logger = logging.getLogger(__name__)<br>        <br>    async def initialize(self):<br>        &quot;&quot;&quot;Initialize the agent.&quot;&quot;&quot;<br>        self.logger.info(&quot;Test Generation Agent initialized&quot;)<br>        <br>    async def execute(self, payload: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Execute test generation task.&quot;&quot;&quot;<br>        project_id = payload[&quot;project_id&quot;]<br>        trigger_data = payload[&quot;trigger_data&quot;]<br>        <br>        # Analyze code changes<br>        code_analysis = await self._analyze_code_changes(trigger_data)<br>        <br>        # Generate test cases based on analysis<br>        test_cases = await self._generate_test_cases(code_analysis)<br>        <br>        # Store test cases<br>        await self._store_test_cases(project_id, test_cases)<br>        <br>        return {<br>            &quot;test_cases&quot;: [tc.__dict__ for tc in test_cases],<br>            &quot;analysis&quot;: code_analysis,<br>            &quot;count&quot;: len(test_cases)<br>        }<br>    <br>    async def _analyze_code_changes(self, trigger_data: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Analyze code changes to understand what needs testing.&quot;&quot;&quot;<br>        files_changed = trigger_data.get(&quot;files_changed&quot;, [])<br>        commit_message = trigger_data.get(&quot;commit_message&quot;, &quot;&quot;)<br>        <br>        prompt = f&quot;&quot;&quot;<br>        Analyze the following code changes and provide testing recommendations:<br>        <br>        Files Changed: {files_changed}<br>        Commit Message: {commit_message}<br>        <br>        Please provide:<br>        1. Areas that need testing<br>        2. Risk assessment<br>        3. Recommended test types<br>        4. Priority levels<br>        <br>        Format your response as JSON.<br>        &quot;&quot;&quot;<br>        <br>        response = await self.openai_client.chat.completions.create(<br>            model=&quot;gpt-4&quot;,<br>            messages=[<br>                {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a QA expert analyzing code changes.&quot;},<br>                {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}<br>            ],<br>            temperature=0.3<br>        )<br>        <br>        return json.loads(response.choices[0].message.content)<br>    <br>    async def _generate_test_cases(self, analysis: Dict) -&gt; List[TestCase]:<br>        &quot;&quot;&quot;Generate test cases based on analysis.&quot;&quot;&quot;<br>        test_cases = []<br>        <br>        for area in analysis.get(&quot;areas_to_test&quot;, []):<br>            prompt = f&quot;&quot;&quot;<br>            Generate comprehensive test cases for: {area}<br>            <br>            Risk Level: {area.get(&#39;risk_level&#39;, &#39;medium&#39;)}<br>            Component Type: {area.get(&#39;type&#39;, &#39;unknown&#39;)}<br>            <br>            Generate test cases covering:<br>            1. Functional testing<br>            2. Edge cases<br>            3. Error scenarios<br>            4. Integration points<br>            <br>            Format each test case as JSON with fields:<br>            - name<br>            - description<br>            - type<br>            - steps (array of objects with action and data)<br>            - expected_result<br>            - priority<br>            - tags<br>            &quot;&quot;&quot;<br>            <br>            response = await self.openai_client.chat.completions.create(<br>                model=&quot;gpt-4&quot;,<br>                messages=[<br>                    {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a test case generation expert.&quot;},<br>                    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}<br>                ],<br>                temperature=0.5<br>            )<br>            <br>            generated_cases = json.loads(response.choices[0].message.content)<br>            <br>            for case_data in generated_cases:<br>                test_case = TestCase(<br>                    id=f&quot;tc_{len(test_cases) + 1}&quot;,<br>                    name=case_data[&quot;name&quot;],<br>                    description=case_data[&quot;description&quot;],<br>                    type=case_data[&quot;type&quot;],<br>                    steps=case_data[&quot;steps&quot;],<br>                    expected_result=case_data[&quot;expected_result&quot;],<br>                    priority=case_data[&quot;priority&quot;],<br>                    tags=case_data[&quot;tags&quot;]<br>                )<br>                test_cases.append(test_case)<br>        <br>        return test_cases<br>    <br>    async def _store_test_cases(self, project_id: str, test_cases: List[TestCase]):<br>        &quot;&quot;&quot;Store generated test cases.&quot;&quot;&quot;<br>        # Implementation would store in database<br>        self.logger.info(f&quot;Stored {len(test_cases)} test cases for project {project_id}&quot;)</pre><h3>Execution Agent</h3><pre># agents/execution_agent.py<br>import asyncio<br>import logging<br>from typing import Dict, List<br>import aiohttp<br>import json<br>from datetime import datetime<br>from selenium import webdriver<br>from selenium.webdriver.chrome.options import Options<br><br>class ExecutionAgent:<br>    def __init__(self, config: Dict):<br>        self.config = config<br>        self.logger = logging.getLogger(__name__)<br>        self.browser_pool = []<br>        <br>    async def initialize(self):<br>        &quot;&quot;&quot;Initialize the execution agent.&quot;&quot;&quot;<br>        await self._setup_browser_pool()<br>        self.logger.info(&quot;Execution Agent initialized&quot;)<br>        <br>    async def _setup_browser_pool(self):<br>        &quot;&quot;&quot;Setup browser instances for UI testing.&quot;&quot;&quot;<br>        chrome_options = Options()<br>        chrome_options.add_argument(&quot;--headless&quot;)<br>        chrome_options.add_argument(&quot;--no-sandbox&quot;)<br>        chrome_options.add_argument(&quot;--disable-dev-shm-usage&quot;)<br>        <br>        # Create browser pool<br>        for i in range(self.config.get(&quot;browser_pool_size&quot;, 3)):<br>            driver = webdriver.Chrome(options=chrome_options)<br>            self.browser_pool.append(driver)<br>    <br>    async def execute(self, payload: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Execute test cases.&quot;&quot;&quot;<br>        test_cases = payload[&quot;test_cases&quot;]<br>        project_id = payload[&quot;project_id&quot;]<br>        <br>        results = []<br>        <br>        # Execute test cases concurrently<br>        semaphore = asyncio.Semaphore(self.config.get(&quot;concurrent_executions&quot;, 5))<br>        tasks = []<br>        <br>        for test_case in test_cases:<br>            task = asyncio.create_task(<br>                self._execute_test_case_with_semaphore(semaphore, test_case)<br>            )<br>            tasks.append(task)<br>        <br>        results = await asyncio.gather(*tasks, return_exceptions=True)<br>        <br>        # Process results<br>        execution_summary = self._process_execution_results(results)<br>        <br>        return {<br>            &quot;results&quot;: results,<br>            &quot;summary&quot;: execution_summary,<br>            &quot;project_id&quot;: project_id,<br>            &quot;executed_at&quot;: datetime.utcnow().isoformat()<br>        }<br>    <br>    async def _execute_test_case_with_semaphore(self, semaphore, test_case):<br>        &quot;&quot;&quot;Execute a single test case with concurrency control.&quot;&quot;&quot;<br>        async with semaphore:<br>            return await self._execute_test_case(test_case)<br>    <br>    async def _execute_test_case(self, test_case: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Execute a single test case.&quot;&quot;&quot;<br>        start_time = datetime.utcnow()<br>        <br>        try:<br>            if test_case[&quot;type&quot;] == &quot;api&quot;:<br>                result = await self._execute_api_test(test_case)<br>            elif test_case[&quot;type&quot;] == &quot;ui&quot;:<br>                result = await self._execute_ui_test(test_case)<br>            elif test_case[&quot;type&quot;] == &quot;performance&quot;:<br>                result = await self._execute_performance_test(test_case)<br>            else:<br>                result = await self._execute_generic_test(test_case)<br>            <br>            end_time = datetime.utcnow()<br>            execution_time = (end_time - start_time).total_seconds()<br>            <br>            return {<br>                &quot;test_case_id&quot;: test_case[&quot;id&quot;],<br>                &quot;status&quot;: &quot;passed&quot; if result[&quot;success&quot;] else &quot;failed&quot;,<br>                &quot;execution_time&quot;: execution_time,<br>                &quot;result&quot;: result,<br>                &quot;executed_at&quot;: start_time.isoformat()<br>            }<br>            <br>        except Exception as e:<br>            end_time = datetime.utcnow()<br>            execution_time = (end_time - start_time).total_seconds()<br>            <br>            return {<br>                &quot;test_case_id&quot;: test_case[&quot;id&quot;],<br>                &quot;status&quot;: &quot;error&quot;,<br>                &quot;execution_time&quot;: execution_time,<br>                &quot;error&quot;: str(e),<br>                &quot;executed_at&quot;: start_time.isoformat()<br>            }<br>    <br>    async def _execute_api_test(self, test_case: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Execute API test case.&quot;&quot;&quot;<br>        results = []<br>        <br>        for step in test_case[&quot;steps&quot;]:<br>            if step[&quot;action&quot;] == &quot;http_request&quot;:<br>                async with aiohttp.ClientSession() as session:<br>                    method = step[&quot;data&quot;][&quot;method&quot;]<br>                    url = step[&quot;data&quot;][&quot;url&quot;]<br>                    headers = step[&quot;data&quot;].get(&quot;headers&quot;, {})<br>                    payload = step[&quot;data&quot;].get(&quot;payload&quot;)<br>                    <br>                    async with session.request(<br>                        method, url, headers=headers, json=payload<br>                    ) as response:<br>                        response_data = await response.text()<br>                        <br>                        results.append({<br>                            &quot;step&quot;: step[&quot;action&quot;],<br>                            &quot;status_code&quot;: response.status,<br>                            &quot;response&quot;: response_data,<br>                            &quot;headers&quot;: dict(response.headers)<br>                        })<br>        <br>        # Validate results against expected outcomes<br>        success = self._validate_api_results(results, test_case[&quot;expected_result&quot;])<br>        <br>        return {<br>            &quot;success&quot;: success,<br>            &quot;steps&quot;: results,<br>            &quot;validation&quot;: &quot;passed&quot; if success else &quot;failed&quot;<br>        }<br>    <br>    async def _execute_ui_test(self, test_case: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Execute UI test case.&quot;&quot;&quot;<br>        driver = self.browser_pool[0]  # Simple pool selection<br>        results = []<br>        <br>        try:<br>            for step in test_case[&quot;steps&quot;]:<br>                step_result = await self._execute_ui_step(driver, step)<br>                results.append(step_result)<br>            <br>            success = all(result[&quot;success&quot;] for result in results)<br>            <br>            return {<br>                &quot;success&quot;: success,<br>                &quot;steps&quot;: results,<br>                &quot;screenshots&quot;: [r.get(&quot;screenshot&quot;) for r in results if r.get(&quot;screenshot&quot;)]<br>            }<br>            <br>        except Exception as e:<br>            return {<br>                &quot;success&quot;: False,<br>                &quot;error&quot;: str(e),<br>                &quot;steps&quot;: results<br>            }<br>    <br>    async def _execute_ui_step(self, driver, step: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Execute a single UI step.&quot;&quot;&quot;<br>        action = step[&quot;action&quot;]<br>        data = step[&quot;data&quot;]<br>        <br>        try:<br>            if action == &quot;navigate&quot;:<br>                driver.get(data[&quot;url&quot;])<br>                return {&quot;success&quot;: True, &quot;action&quot;: action}<br>                <br>            elif action == &quot;click&quot;:<br>                element = driver.find_element(data[&quot;selector_type&quot;], data[&quot;selector&quot;])<br>                element.click()<br>                return {&quot;success&quot;: True, &quot;action&quot;: action}<br>                <br>            elif action == &quot;input&quot;:<br>                element = driver.find_element(data[&quot;selector_type&quot;], data[&quot;selector&quot;])<br>                element.clear()<br>                element.send_keys(data[&quot;value&quot;])<br>                return {&quot;success&quot;: True, &quot;action&quot;: action}<br>                <br>            elif action == &quot;assert&quot;:<br>                element = driver.find_element(data[&quot;selector_type&quot;], data[&quot;selector&quot;])<br>                actual_text = element.text<br>                expected_text = data[&quot;expected&quot;]<br>                success = actual_text == expected_text<br>                <br>                return {<br>                    &quot;success&quot;: success,<br>                    &quot;action&quot;: action,<br>                    &quot;expected&quot;: expected_text,<br>                    &quot;actual&quot;: actual_text<br>                }<br>            <br>            return {&quot;success&quot;: False, &quot;error&quot;: f&quot;Unknown action: {action}&quot;}<br>            <br>        except Exception as e:<br>            return {&quot;success&quot;: False, &quot;error&quot;: str(e), &quot;action&quot;: action}<br>    <br>    async def _execute_performance_test(self, test_case: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Execute performance test case.&quot;&quot;&quot;<br>        # Implementation for performance testing<br>        # This would include load testing, stress testing, etc.<br>        pass<br>    <br>    async def _execute_generic_test(self, test_case: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Execute generic test case.&quot;&quot;&quot;<br>        # Implementation for other test types<br>        pass<br>    <br>    def _validate_api_results(self, results: List[Dict], expected: str) -&gt; bool:<br>        &quot;&quot;&quot;Validate API test results.&quot;&quot;&quot;<br>        # Implementation would parse expected results and validate<br>        return True  # Simplified for example<br>    <br>    def _process_execution_results(self, results: List[Dict]) -&gt; Dict:<br>        &quot;&quot;&quot;Process and summarize execution results.&quot;&quot;&quot;<br>        total_tests = len(results)<br>        passed_tests = sum(1 for r in results if r.get(&quot;status&quot;) == &quot;passed&quot;)<br>        failed_tests = sum(1 for r in results if r.get(&quot;status&quot;) == &quot;failed&quot;)<br>        error_tests = sum(1 for r in results if r.get(&quot;status&quot;) == &quot;error&quot;)<br>        <br>        return {<br>            &quot;total&quot;: total_tests,<br>            &quot;passed&quot;: passed_tests,<br>            &quot;failed&quot;: failed_tests,<br>            &quot;errors&quot;: error_tests,<br>            &quot;pass_rate&quot;: passed_tests / total_tests if total_tests &gt; 0 else 0<br>        }</pre><h3>Analysis Agent</h3><pre># agents/analysis_agent.py<br>import asyncio<br>import logging<br>from typing import Dict, List<br>import numpy as np<br>from sklearn.cluster import KMeans<br>from sklearn.preprocessing import StandardScaler<br>import pandas as pd<br>from datetime import datetime, timedelta<br><br>class AnalysisAgent:<br>    def __init__(self, config: Dict):<br>        self.config = config<br>        self.logger = logging.getLogger(__name__)<br>        <br>    async def initialize(self):<br>        &quot;&quot;&quot;Initialize the analysis agent.&quot;&quot;&quot;<br>        self.logger.info(&quot;Analysis Agent initialized&quot;)<br>        <br>    async def execute(self, payload: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Execute analysis task.&quot;&quot;&quot;<br>        execution_results = payload[&quot;execution_results&quot;]<br>        project_id = payload[&quot;project_id&quot;]<br>        <br>        # Perform various analyses<br>        failure_analysis = await self._analyze_failures(execution_results)<br>        trend_analysis = await self._analyze_trends(project_id, execution_results)<br>        risk_assessment = await self._assess_risk(execution_results)<br>        recommendations = await self._generate_recommendations(<br>            failure_analysis, trend_analysis, risk_assessment<br>        )<br>        <br>        return {<br>            &quot;failure_analysis&quot;: failure_analysis,<br>            &quot;trend_analysis&quot;: trend_analysis,<br>            &quot;risk_assessment&quot;: risk_assessment,<br>            &quot;recommendations&quot;: recommendations,<br>            &quot;analyzed_at&quot;: datetime.utcnow().isoformat()<br>        }<br>    <br>    async def _analyze_failures(self, execution_results: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Analyze test failures to identify patterns.&quot;&quot;&quot;<br>        results = execution_results[&quot;results&quot;]<br>        failed_tests = [r for r in results if r.get(&quot;status&quot;) == &quot;failed&quot;]<br>        <br>        if not failed_tests:<br>            return {&quot;patterns&quot;: [], &quot;categories&quot;: [], &quot;summary&quot;: &quot;No failures detected&quot;}<br>        <br>        # Categorize failures<br>        failure_categories = {}<br>        failure_patterns = []<br>        <br>        for test in failed_tests:<br>            # Extract failure information<br>            error_message = test.get(&quot;error&quot;, &quot;&quot;)<br>            test_type = test.get(&quot;test_case&quot;, {}).get(&quot;type&quot;, &quot;unknown&quot;)<br>            <br>            # Categorize by test type<br>            if test_type not in failure_categories:<br>                failure_categories[test_type] = []<br>            failure_categories[test_type].append(test)<br>            <br>            # Look for common error patterns<br>            if &quot;timeout&quot; in error_message.lower():<br>                failure_patterns.append(&quot;timeout&quot;)<br>            elif &quot;connection&quot; in error_message.lower():<br>                failure_patterns.append(&quot;network&quot;)<br>            elif &quot;assertion&quot; in error_message.lower():<br>                failure_patterns.append(&quot;validation&quot;)<br>            elif &quot;element not found&quot; in error_message.lower():<br>                failure_patterns.append(&quot;ui_element&quot;)<br>        <br>        # Count pattern frequencies<br>        pattern_counts = {}<br>        for pattern in failure_patterns:<br>            pattern_counts[pattern] = pattern_counts.get(pattern, 0) + 1<br>        <br>        return {<br>            &quot;total_failures&quot;: len(failed_tests),<br>            &quot;categories&quot;: failure_categories,<br>            &quot;common_patterns&quot;: pattern_counts,<br>            &quot;failure_rate&quot;: len(failed_tests) / len(results) if results else 0<br>        }<br>    <br>    async def _analyze_trends(self, project_id: str, current_results: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Analyze trends compared to historical data.&quot;&quot;&quot;<br>        # This would typically fetch historical data from database<br>        historical_data = await self._fetch_historical_data(project_id)<br>        <br>        current_pass_rate = current_results[&quot;summary&quot;][&quot;pass_rate&quot;]<br>        <br>        if not historical_data:<br>            return {<br>                &quot;trend&quot;: &quot;no_history&quot;,<br>                &quot;current_pass_rate&quot;: current_pass_rate,<br>                &quot;message&quot;: &quot;No historical data available for trend analysis&quot;<br>            }<br>        <br>        # Calculate trend<br>        historical_pass_rates = [h[&quot;pass_rate&quot;] for h in historical_data[-10:]]  # Last 10 runs<br>        avg_historical_rate = np.mean(historical_pass_rates)<br>        <br>        trend_direction = &quot;improving&quot; if current_pass_rate &gt; avg_historical_rate else &quot;declining&quot;<br>        trend_magnitude = abs(current_pass_rate - avg_historical_rate)<br>        <br>        return {<br>            &quot;trend&quot;: trend_direction,<br>            &quot;current_pass_rate&quot;: current_pass_rate,<br>            &quot;historical_average&quot;: avg_historical_rate,<br>            &quot;trend_magnitude&quot;: trend_magnitude,<br>            &quot;historical_data&quot;: historical_pass_rates<br>        }<br>    <br>    async def _assess_risk(self, execution_results: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Assess the risk level based on test results.&quot;&quot;&quot;<br>        results = execution_results[&quot;results&quot;]<br>        summary = execution_results[&quot;summary&quot;]<br>        <br>        # Calculate risk factors<br>        pass_rate = summary[&quot;pass_rate&quot;]<br>        failure_rate = summary[&quot;failed&quot;] / summary[&quot;total&quot;] if summary[&quot;total&quot;] &gt; 0 else 0<br>        error_rate = summary[&quot;errors&quot;] / summary[&quot;total&quot;] if summary[&quot;total&quot;] &gt; 0 else 0<br>        <br>        # Risk scoring (0-100 scale)<br>        risk_score = 0<br>        <br>        # Pass rate factor (higher pass rate = lower risk)<br>        if pass_rate &lt; 0.5:<br>            risk_score += 40<br>        elif pass_rate &lt; 0.8:<br>            risk_score += 20<br>        elif pass_rate &lt; 0.95:<br>            risk_score += 10<br>        <br>        # Error rate factor<br>        if error_rate &gt; 0.1:<br>            risk_score += 30<br>        elif error_rate &gt; 0.05:<br>            risk_score += 15<br>        <br>        # Critical test failures<br>        critical_failures = [<br>            r for r in results <br>            if r.get(&quot;status&quot;) == &quot;failed&quot; and <br>            r.get(&quot;test_case&quot;, {}).get(&quot;priority&quot;) == &quot;high&quot;<br>        ]<br>        <br>        if critical_failures:<br>            risk_score += 25<br>        <br>        # Determine risk level<br>        if risk_score &gt;= 70:<br>            risk_level = &quot;high&quot;<br>        elif risk_score &gt;= 40:<br>            risk_level = &quot;medium&quot;<br>        else:<br>            risk_level = &quot;low&quot;<br>        <br>        return {<br>            &quot;risk_score&quot;: risk_score,<br>            &quot;risk_level&quot;: risk_level,<br>            &quot;factors&quot;: {<br>                &quot;pass_rate&quot;: pass_rate,<br>                &quot;failure_rate&quot;: failure_rate,<br>                &quot;error_rate&quot;: error_rate,<br>                &quot;critical_failures&quot;: len(critical_failures)<br>            },<br>            &quot;recommendations&quot;: self._get_risk_recommendations(risk_level, risk_score)<br>        }<br>    <br>    async def _generate_recommendations(self, failure_analysis: Dict, <br>                                      trend_analysis: Dict, <br>                                      risk_assessment: Dict) -&gt; List[Dict]:<br>        &quot;&quot;&quot;Generate actionable recommendations based on analysis.&quot;&quot;&quot;<br>        recommendations = []<br>        <br>        # Failure-based recommendations<br>        if failure_analysis[&quot;failure_rate&quot;] &gt; 0.2:<br>            recommendations.append({<br>                &quot;type&quot;: &quot;failure_mitigation&quot;,<br>                &quot;priority&quot;: &quot;high&quot;,<br>                &quot;title&quot;: &quot;High Failure Rate Detected&quot;,<br>                &quot;description&quot;: f&quot;Failure rate of {failure_analysis[&#39;failure_rate&#39;]:.2%} exceeds threshold&quot;,<br>                &quot;action&quot;: &quot;Review and fix failing test cases immediately&quot;<br>            })<br>        <br>        # Pattern-based recommendations<br>        common_patterns = failure_analysis.get(&quot;common_patterns&quot;, {})<br>        if &quot;timeout&quot; in common_patterns and common_patterns[&quot;timeout&quot;] &gt; 2:<br>            recommendations.append({<br>                &quot;type&quot;: &quot;performance&quot;,<br>                &quot;priority&quot;: &quot;medium&quot;,<br>                &quot;title&quot;: &quot;Multiple Timeout Failures&quot;,<br>                &quot;description&quot;: f&quot;{common_patterns[&#39;timeout&#39;]} tests failed due to timeouts&quot;,<br>                &quot;action&quot;: &quot;Investigate performance issues and increase timeout thresholds&quot;<br>            })<br>        <br>        # Trend-based recommendations<br>        if trend_analysis.get(&quot;trend&quot;) == &quot;declining&quot;:<br>            recommendations.append({<br>                &quot;type&quot;: &quot;quality_trend&quot;,<br>                &quot;priority&quot;: &quot;medium&quot;,<br>                &quot;title&quot;: &quot;Quality Trend Declining&quot;,<br>                &quot;description&quot;: &quot;Test pass rate is decreasing compared to historical average&quot;,<br>                &quot;action&quot;: &quot;Review recent changes and implement quality gates&quot;<br>            })<br>        <br>        # Risk-based recommendations<br>        if risk_assessment[&quot;risk_level&quot;] == &quot;high&quot;:<br>            recommendations.append({<br>                &quot;type&quot;: &quot;risk_mitigation&quot;,<br>                &quot;priority&quot;: &quot;critical&quot;,<br>                &quot;title&quot;: &quot;High Risk Release&quot;,<br>                &quot;description&quot;: f&quot;Risk score: {risk_assessment[&#39;risk_score&#39;]}/100&quot;,<br>                &quot;action&quot;: &quot;Consider delaying release until critical issues are resolved&quot;<br>            })<br>        <br>        return recommendations<br>    <br>    def _get_risk_recommendations(self, risk_level: str, risk_score: int) -&gt; List[str]:<br>        &quot;&quot;&quot;Get recommendations based on risk assessment.&quot;&quot;&quot;<br>        if risk_level == &quot;high&quot;:<br>            return [<br>                &quot;Block deployment until critical issues are resolved&quot;,<br>                &quot;Conduct thorough manual testing&quot;,<br>                &quot;Review all failed test cases&quot;,<br>                &quot;Consider rollback plan&quot;<br>            ]<br>        elif risk_level == &quot;medium&quot;:<br>            return [<br>                &quot;Review failed tests before deployment&quot;,<br>                &quot;Monitor deployment closely&quot;,<br>                &quot;Have rollback plan ready&quot;<br>            ]<br>        else:<br>            return [<br>                &quot;Deployment approved from QA perspective&quot;,<br>                &quot;Continue monitoring test trends&quot;<br>            ]<br>    <br>    async def _fetch_historical_data(self, project_id: str) -&gt; List[Dict]:<br>        &quot;&quot;&quot;Fetch historical test data for trend analysis.&quot;&quot;&quot;<br>        # This would typically query a database<br>        # Returning mock data for example<br>        return [<br>            {&quot;date&quot;: &quot;2024-01-01&quot;, &quot;pass_rate&quot;: 0.85},<br>            {&quot;date&quot;: &quot;2024-01-02&quot;, &quot;pass_rate&quot;: 0.87},<br>            {&quot;date&quot;: &quot;2024-01-03&quot;, &quot;pass_rate&quot;: 0.82},<br>            {&quot;date&quot;: &quot;2024-01-04&quot;, &quot;pass_rate&quot;: 0.89},<br>            {&quot;date&quot;: &quot;2024-01-05&quot;, &quot;pass_rate&quot;: 0.91}<br>        ]</pre><h3>Reporting Agent</h3><pre># agents/reporting_agent.py<br>import asyncio<br>import logging<br>from typing import Dict, List<br>from datetime import datetime<br>import jinja2<br>import pdfkit<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>import io<br>import base64<br><br>class ReportingAgent:<br>    def __init__(self, config: Dict):<br>        self.config = config<br>        self.logger = logging.getLogger(__name__)<br>        self.template_env = jinja2.Environment(<br>            loader=jinja2.FileSystemLoader(&#39;templates/&#39;)<br>        )<br>        <br>    async def initialize(self):<br>        &quot;&quot;&quot;Initialize the reporting agent.&quot;&quot;&quot;<br>        self.logger.info(&quot;Reporting Agent initialized&quot;)<br>    <br>    async def execute(self, payload: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Execute reporting task.&quot;&quot;&quot;<br>        analysis_results = payload[&quot;analysis_results&quot;]<br>        project_id = payload[&quot;project_id&quot;]<br>        flow_id = payload[&quot;flow_id&quot;]<br>        <br>        # Generate different types of reports<br>        summary_report = await self._generate_summary_report(analysis_results)<br>        detailed_report = await self._generate_detailed_report(analysis_results)<br>        visual_report = await self._generate_visual_report(analysis_results)<br>        <br>        # Send notifications<br>        await self._send_notifications(summary_report, project_id)<br>        <br>        return {<br>            &quot;summary_report&quot;: summary_report,<br>            &quot;detailed_report&quot;: detailed_report,<br>            &quot;visual_report&quot;: visual_report,<br>            &quot;generated_at&quot;: datetime.utcnow().isoformat(),<br>            &quot;flow_id&quot;: flow_id<br>        }<br>    <br>    async def _generate_summary_report(self, analysis: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Generate a concise summary report.&quot;&quot;&quot;<br>        risk_assessment = analysis[&quot;risk_assessment&quot;]<br>        failure_analysis = analysis[&quot;failure_analysis&quot;]<br>        recommendations = analysis[&quot;recommendations&quot;]<br>        <br>        summary = {<br>            &quot;overall_status&quot;: self._determine_overall_status(risk_assessment),<br>            &quot;pass_rate&quot;: failure_analysis.get(&quot;failure_rate&quot;, 1.0),<br>            &quot;total_failures&quot;: failure_analysis.get(&quot;total_failures&quot;, 0),<br>            &quot;risk_level&quot;: risk_assessment[&quot;risk_level&quot;],<br>            &quot;risk_score&quot;: risk_assessment[&quot;risk_score&quot;],<br>            &quot;critical_recommendations&quot;: [<br>                r for r in recommendations <br>                if r.get(&quot;priority&quot;) in [&quot;critical&quot;, &quot;high&quot;]<br>            ],<br>            &quot;deployment_recommendation&quot;: self._get_deployment_recommendation(risk_assessment)<br>        }<br>        <br>        return summary<br>    <br>    async def _generate_detailed_report(self, analysis: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Generate a comprehensive detailed report.&quot;&quot;&quot;<br>        template = self.template_env.get_template(&#39;detailed_report.html&#39;)<br>        <br>        report_data = {<br>            &quot;generated_at&quot;: datetime.utcnow().strftime(&quot;%Y-%m-%d %H:%M:%S UTC&quot;),<br>            &quot;analysis&quot;: analysis,<br>            &quot;charts&quot;: await self._generate_charts(analysis)<br>        }<br>        <br>        html_content = template.render(report_data)<br>        <br>        # Convert to PDF if needed<br>        pdf_content = None<br>        if self.config.get(&quot;generate_pdf&quot;, False):<br>            pdf_content = pdfkit.from_string(html_content)<br>        <br>        return {<br>            &quot;html&quot;: html_content,<br>            &quot;pdf&quot;: base64.b64encode(pdf_content).decode() if pdf_content else None,<br>            &quot;data&quot;: report_data<br>        }<br>    <br>    async def _generate_visual_report(self, analysis: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Generate visual reports with charts and graphs.&quot;&quot;&quot;<br>        charts = {}<br>        <br>        # Failure distribution chart<br>        failure_analysis = analysis[&quot;failure_analysis&quot;]<br>        if failure_analysis.get(&quot;categories&quot;):<br>            charts[&quot;failure_by_category&quot;] = await self._create_failure_category_chart(<br>                failure_analysis[&quot;categories&quot;]<br>            )<br>        <br>        # Trend chart<br>        trend_analysis = analysis[&quot;trend_analysis&quot;]<br>        if trend_analysis.get(&quot;historical_data&quot;):<br>            charts[&quot;trend_chart&quot;] = await self._create_trend_chart(<br>                trend_analysis[&quot;historical_data&quot;],<br>                trend_analysis[&quot;current_pass_rate&quot;]<br>            )<br>        <br>        # Risk assessment radar chart<br>        charts[&quot;risk_radar&quot;] = await self._create_risk_radar_chart(<br>            analysis[&quot;risk_assessment&quot;][&quot;factors&quot;]<br>        )<br>        <br>        return {&quot;charts&quot;: charts}<br>    <br>    async def _create_failure_category_chart(self, categories: Dict) -&gt; str:<br>        &quot;&quot;&quot;Create failure distribution pie chart.&quot;&quot;&quot;<br>        plt.figure(figsize=(10, 8))<br>        <br>        labels = list(categories.keys())<br>        sizes = [len(tests) for tests in categories.values()]<br>        colors = plt.cm.Set3(range(len(labels)))<br>        <br>        plt.pie(sizes, labels=labels, colors=colors, autopct=&#39;%1.1f%%&#39;, startangle=90)<br>        plt.title(&#39;Test Failures by Category&#39;)<br>        plt.axis(&#39;equal&#39;)<br>        <br>        # Convert to base64 string<br>        buffer = io.BytesIO()<br>        plt.savefig(buffer, format=&#39;png&#39;, bbox_inches=&#39;tight&#39;, dpi=150)<br>        buffer.seek(0)<br>        chart_data = base64.b64encode(buffer.read()).decode()<br>        plt.close()<br>        <br>        return chart_data<br>    <br>    async def _create_trend_chart(self, historical_data: List[float], current_rate: float) -&gt; str:<br>        &quot;&quot;&quot;Create pass rate trend line chart.&quot;&quot;&quot;<br>        plt.figure(figsize=(12, 6))<br>        <br>        x_values = list(range(len(historical_data)))<br>        x_values.append(len(historical_data))<br>        y_values = historical_data + [current_rate]<br>        <br>        plt.plot(x_values[:-1], historical_data, &#39;b-o&#39;, label=&#39;Historical&#39;, linewidth=2)<br>        plt.plot([x_values[-2], x_values[-1]], [historical_data[-1], current_rate], <br>                &#39;r-o&#39;, label=&#39;Current&#39;, linewidth=2)<br>        <br>        plt.xlabel(&#39;Test Runs&#39;)<br>        plt.ylabel(&#39;Pass Rate&#39;)<br>        plt.title(&#39;Test Pass Rate Trend&#39;)<br>        plt.legend()<br>        plt.grid(True, alpha=0.3)<br>        plt.ylim(0, 1)<br>        <br>        # Convert to base64 string<br>        buffer = io.BytesIO()<br>        plt.savefig(buffer, format=&#39;png&#39;, bbox_inches=&#39;tight&#39;, dpi=150)<br>        buffer.seek(0)<br>        chart_data = base64.b64encode(buffer.read()).decode()<br>        plt.close()<br>        <br>        return chart_data<br>    <br>    async def _create_risk_radar_chart(self, factors: Dict) -&gt; str:<br>        &quot;&quot;&quot;Create risk assessment radar chart.&quot;&quot;&quot;<br>        categories = list(factors.keys())<br>        values = list(factors.values())<br>        <br>        # Number of variables<br>        N = len(categories)<br>        <br>        # Compute angle for each axis<br>        angles = [n / float(N) * 2 * 3.14159 for n in range(N)]<br>        angles += angles[:1]  # Complete the circle<br>        <br>        # Add first value to end to complete circle<br>        values += values[:1]<br>        <br>        fig, ax = plt.subplots(figsize=(8, 8), subplot_kw=dict(projection=&#39;polar&#39;))<br>        <br>        # Plot<br>        ax.plot(angles, values, &#39;o-&#39;, linewidth=2, color=&#39;red&#39;, alpha=0.7)<br>        ax.fill(angles, values, alpha=0.25, color=&#39;red&#39;)<br>        <br>        # Add labels<br>        ax.set_xticks(angles[:-1])<br>        ax.set_xticklabels(categories)<br>        ax.set_ylim(0, 1)<br>        ax.set_title(&#39;Risk Factors Assessment&#39;, y=1.08)<br>        <br>        # Convert to base64 string<br>        buffer = io.BytesIO()<br>        plt.savefig(buffer, format=&#39;png&#39;, bbox_inches=&#39;tight&#39;, dpi=150)<br>        buffer.seek(0)<br>        chart_data = base64.b64encode(buffer.read()).decode()<br>        plt.close()<br>        <br>        return chart_data<br>    <br>    async def _send_notifications(self, summary: Dict, project_id: str):<br>        &quot;&quot;&quot;Send notifications based on results.&quot;&quot;&quot;<br>        risk_level = summary[&quot;risk_level&quot;]<br>        <br>        if risk_level == &quot;high&quot;:<br>            await self._send_critical_alert(summary, project_id)<br>        elif risk_level == &quot;medium&quot;:<br>            await self._send_warning_notification(summary, project_id)<br>        else:<br>            await self._send_success_notification(summary, project_id)<br>    <br>    async def _send_critical_alert(self, summary: Dict, project_id: str):<br>        &quot;&quot;&quot;Send critical alert for high-risk situations.&quot;&quot;&quot;<br>        message = f&quot;&quot;&quot;<br>        🚨 CRITICAL QA ALERT - Project {project_id}<br>        <br>        Risk Level: {summary[&#39;risk_level&#39;].upper()}<br>        Risk Score: {summary[&#39;risk_score&#39;]}/100<br>        Pass Rate: {summary[&#39;pass_rate&#39;]:.1%}<br>        Total Failures: {summary[&#39;total_failures&#39;]}<br>        <br>        🚫 DEPLOYMENT NOT RECOMMENDED<br>        <br>        Critical Actions Required:<br>        {chr(10).join(&#39;• &#39; + r[&#39;action&#39;] for r in summary[&#39;critical_recommendations&#39;])}<br>        &quot;&quot;&quot;<br>        <br>        # Send to Slack, email, etc.<br>        await self._send_slack_message(message, channel=&quot;#qa-alerts&quot;)<br>    <br>    async def _send_warning_notification(self, summary: Dict, project_id: str):<br>        &quot;&quot;&quot;Send warning notification for medium-risk situations.&quot;&quot;&quot;<br>        message = f&quot;&quot;&quot;<br>        ⚠️ QA Warning - Project {project_id}<br>        <br>        Risk Level: {summary[&#39;risk_level&#39;].title()}<br>        Risk Score: {summary[&#39;risk_score&#39;]}/100<br>        Pass Rate: {summary[&#39;pass_rate&#39;]:.1%}<br>        <br>        ⚡ PROCEED WITH CAUTION<br>        <br>        Recommended Actions:<br>        {chr(10).join(&#39;• &#39; + r[&#39;action&#39;] for r in summary[&#39;critical_recommendations&#39;])}<br>        &quot;&quot;&quot;<br>        <br>        await self._send_slack_message(message, channel=&quot;#qa-notifications&quot;)<br>    <br>    async def _send_success_notification(self, summary: Dict, project_id: str):<br>        &quot;&quot;&quot;Send success notification for low-risk situations.&quot;&quot;&quot;<br>        message = f&quot;&quot;&quot;<br>        ✅ QA Success - Project {project_id}<br>        <br>        Risk Level: {summary[&#39;risk_level&#39;].title()}<br>        Pass Rate: {summary[&#39;pass_rate&#39;]:.1%}<br>        <br>        🚀 DEPLOYMENT APPROVED<br>        <br>        All quality gates passed successfully!<br>        &quot;&quot;&quot;<br>        <br>        await self._send_slack_message(message, channel=&quot;#qa-notifications&quot;)<br>    <br>    async def _send_slack_message(self, message: str, channel: str):<br>        &quot;&quot;&quot;Send message to Slack.&quot;&quot;&quot;<br>        # Implementation would use Slack API<br>        self.logger.info(f&quot;Slack notification sent to {channel}: {message[:100]}...&quot;)<br>    <br>    def _determine_overall_status(self, risk_assessment: Dict) -&gt; str:<br>        &quot;&quot;&quot;Determine overall status based on risk assessment.&quot;&quot;&quot;<br>        risk_level = risk_assessment[&quot;risk_level&quot;]<br>        <br>        if risk_level == &quot;high&quot;:<br>            return &quot;FAILED&quot;<br>        elif risk_level == &quot;medium&quot;:<br>            return &quot;WARNING&quot;<br>        else:<br>            return &quot;PASSED&quot;<br>    <br>    def _get_deployment_recommendation(self, risk_assessment: Dict) -&gt; str:<br>        &quot;&quot;&quot;Get deployment recommendation based on risk.&quot;&quot;&quot;<br>        risk_level = risk_assessment[&quot;risk_level&quot;]<br>        <br>        if risk_level == &quot;high&quot;:<br>            return &quot;BLOCK_DEPLOYMENT&quot;<br>        elif risk_level == &quot;medium&quot;:<br>            return &quot;PROCEED_WITH_CAUTION&quot;<br>        else:<br>            return &quot;APPROVED&quot;<br>    <br>    async def _generate_charts(self, analysis: Dict) -&gt; Dict:<br>        &quot;&quot;&quot;Generate all charts for the detailed report.&quot;&quot;&quot;<br>        return await self._generate_visual_report(analysis)</pre><h3>API Gateway</h3><pre># api/gateway.py<br>from fastapi import FastAPI, HTTPException, Depends, BackgroundTasks<br>from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials<br>from fastapi.middleware.cors import CORSMiddleware<br>from pydantic import BaseModel<br>from typing import Dict, List, Optional<br>import asyncio<br>import logging<br>from datetime import datetime<br><br>from agent_orchestrator import AgentOrchestrator, AgentType<br><br>app = FastAPI(title=&quot;QA Agentic Flow API&quot;, version=&quot;1.0.0&quot;)<br>security = HTTPBearer()<br><br># CORS middleware<br>app.add_middleware(<br>    CORSMiddleware,<br>    allow_origins=[&quot;*&quot;],<br>    allow_credentials=True,<br>    allow_methods=[&quot;*&quot;],<br>    allow_headers=[&quot;*&quot;],<br>)<br><br># Global orchestrator instance<br>orchestrator: Optional[AgentOrchestrator] = None<br><br>class TriggerRequest(BaseModel):<br>    project_id: str<br>    trigger_type: str<br>    data: Dict<br>    <br>class TaskStatusResponse(BaseModel):<br>    task_id: str<br>    status: str<br>    result: Optional[Dict] = None<br>    error: Optional[str] = None<br><br>@app.on_startup<br>async def startup_event():<br>    &quot;&quot;&quot;Initialize the QA Agentic Flow system.&quot;&quot;&quot;<br>    global orchestrator<br>    <br>    config = {<br>        &quot;openai_api_key&quot;: &quot;your-openai-api-key&quot;,<br>        &quot;browser_pool_size&quot;: 3,<br>        &quot;concurrent_executions&quot;: 5,<br>        &quot;generate_pdf&quot;: True<br>    }<br>    <br>    orchestrator = AgentOrchestrator(config)<br>    await orchestrator.initialize()<br>    <br>    logging.info(&quot;QA Agentic Flow API started successfully&quot;)<br><br>@app.get(&quot;/health&quot;)<br>async def health_check():<br>    &quot;&quot;&quot;Health check endpoint.&quot;&quot;&quot;<br>    return {&quot;status&quot;: &quot;healthy&quot;, &quot;timestamp&quot;: datetime.utcnow().isoformat()}<br><br>@app.post(&quot;/qa/trigger&quot;)<br>async def trigger_qa_flow(<br>    request: TriggerRequest,<br>    background_tasks: BackgroundTasks,<br>    credentials: HTTPAuthorizationCredentials = Depends(security)<br>):<br>    &quot;&quot;&quot;Trigger a complete QA flow.&quot;&quot;&quot;<br>    try:<br>        # Validate authentication<br>        await validate_token(credentials.credentials)<br>        <br>        # Start QA flow in background<br>        background_tasks.add_task(<br>            run_qa_flow_background,<br>            request.project_id,<br>            request.data<br>        )<br>        <br>        return {<br>            &quot;message&quot;: &quot;QA flow triggered successfully&quot;,<br>            &quot;project_id&quot;: request.project_id,<br>            &quot;triggered_at&quot;: datetime.utcnow().isoformat()<br>        }<br>        <br>    except Exception as e:<br>        raise HTTPException(status_code=500, detail=str(e))<br><br>@app.post(&quot;/qa/test-generation&quot;)<br>async def generate_tests(<br>    request: TriggerRequest,<br>    credentials: HTTPAuthorizationCredentials = Depends(security)<br>):<br>    &quot;&quot;&quot;Generate test cases only.&quot;&quot;&quot;<br>    try:<br>        await validate_token(credentials.credentials)<br>        <br>        task_id = await orchestrator.submit_task(<br>            AgentType.TEST_GENERATION,<br>            {<br>                &quot;project_id&quot;: request.project_id,<br>                &quot;trigger_data&quot;: request.data<br>            }<br>        )<br>        <br>        return {&quot;task_id&quot;: task_id, &quot;status&quot;: &quot;submitted&quot;}<br>        <br>    except Exception as e:<br>        raise HTTPException(status_code=500, detail=str(e))<br><br>@app.post(&quot;/qa/execute-tests&quot;)<br>async def execute_tests(<br>    project_id: str,<br>    test_cases: List[Dict],<br>    credentials: HTTPAuthorizationCredentials = Depends(security)<br>):<br>    &quot;&quot;&quot;Execute provided test cases.&quot;&quot;&quot;<br>    try:<br>        await validate_token(credentials.credentials)<br>        <br>        task_id = await orchestrator.submit_task(<br>            AgentType.EXECUTION,<br>            {<br>                &quot;project_id&quot;: project_id,<br>                &quot;test_cases&quot;: test_cases<br>            }<br>        )<br>        <br>        return {&quot;task_id&quot;: task_id, &quot;status&quot;: &quot;submitted&quot;}<br>        <br>    except Exception as e:<br>        raise HTTPException(status_code=500, detail=str(e))<br><br>@app.get(&quot;/qa/task/{task_id}&quot;)<br>async def get_task_status(<br>    task_id: str,<br>    credentials: HTTPAuthorizationCredentials = Depends(security)<br>) -&gt; TaskStatusResponse:<br>    &quot;&quot;&quot;Get the status of a specific task.&quot;&quot;&quot;<br>    try:<br>        await validate_token(credentials.credentials)<br>        <br>        task = await orchestrator.get_task_status(task_id)<br>        if not task:<br>            raise HTTPException(status_code=404, detail=&quot;Task not found&quot;)<br>        <br>        return TaskStatusResponse(<br>            task_id=task_id,<br>            status=task.status.value,<br>            result=task.result,<br>            error=task.error<br>        )<br>        <br>    except HTTPException:<br>        raise<br>    except Exception as e:<br>        raise HTTPException(status_code=500, detail=str(e))<br><br>@app.get(&quot;/qa/projects/{project_id}/history&quot;)<br>async def get_project_history(<br>    project_id: str,<br>    limit: int = 10,<br>    credentials: HTTPAuthorizationCredentials = Depends(security)<br>):<br>    &quot;&quot;&quot;Get QA history for a project.&quot;&quot;&quot;<br>    try:<br>        await validate_token(credentials.credentials)<br>        <br>        # This would typically query a database<br>        # Returning mock data for example<br>        history = [<br>            {<br>                &quot;flow_id&quot;: f&quot;flow_{i}&quot;,<br>                &quot;triggered_at&quot;: datetime.utcnow().isoformat(),<br>                &quot;status&quot;: &quot;completed&quot;,<br>                &quot;pass_rate&quot;: 0.85 + (i * 0.02),<br>                &quot;total_tests&quot;: 50 + i<br>            }<br>            for i in range(limit)<br>        ]<br>        <br>        return {&quot;project_id&quot;: project_id, &quot;history&quot;: history}<br>        <br>    except Exception as e:<br>        raise HTTPException(status_code=500, detail=str(e))<br><br>@app.get(&quot;/qa/metrics&quot;)<br>async def get_system_metrics(<br>    credentials: HTTPAuthorizationCredentials = Depends(security)<br>):<br>    &quot;&quot;&quot;Get system-wide QA metrics.&quot;&quot;&quot;<br>    try:<br>        await validate_token(credentials.credentials)<br>        <br>        # This would typically aggregate from multiple sources<br>        metrics = {<br>            &quot;total_projects&quot;: 25,<br>            &quot;active_flows&quot;: 3,<br>            &quot;completed_today&quot;: 12,<br>            &quot;average_pass_rate&quot;: 0.87,<br>            &quot;system_uptime&quot;: &quot;99.9%&quot;<br>        }<br>        <br>        return metrics<br>        <br>    except Exception as e:<br>        raise HTTPException(status_code=500, detail=str(e))<br><br>async def validate_token(token: str):<br>    &quot;&quot;&quot;Validate authentication token.&quot;&quot;&quot;<br>    # Implementation would validate JWT token or API key<br>    if not token or token == &quot;invalid&quot;:<br>        raise HTTPException(status_code=401, detail=&quot;Invalid authentication token&quot;)<br><br>async def run_qa_flow_background(project_id: str, trigger_data: Dict):<br>    &quot;&quot;&quot;Run QA flow in background.&quot;&quot;&quot;<br>    try:<br>        result = await orchestrator.start_qa_flow(project_id, trigger_data)<br>        logging.info(f&quot;QA flow completed for project {project_id}: {result[&#39;status&#39;]}&quot;)<br>    except Exception as e:<br>        logging.error(f&quot;QA flow failed for project {project_id}: {e}&quot;)<br><br>if __name__ == &quot;__main__&quot;:<br>    import uvicorn<br>    uvicorn.run(app, host=&quot;0.0.0.0&quot;, port=8000)</pre><h3>In this article, I will cover the following key sections:</h3><ol><li><strong>Overview</strong></li><li><strong>System Architecture</strong></li><li><strong>Core Components</strong></li><li><strong>Implementation Code</strong></li></ol><p>For the remaining sections, please refer to the other articles in this series:</p><p><strong>5. Deployment Architecture</strong></p><p><strong>6. Configuration &amp; Setup</strong></p><p><strong>7. API Documentation</strong></p><p><strong>8. Monitoring &amp; Observability</strong></p><p><strong>9. Security &amp; Compliance</strong></p><p><strong>10. Troubleshooting</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3e76a581ec0a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>