<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sharath Reddy on Medium]]></title>
        <description><![CDATA[Stories by Sharath Reddy on Medium]]></description>
        <link>https://medium.com/@mr.sharath.7?source=rss-3795b55ce854------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*6LSJa6CLRe7dvlH0pPSL6Q.jpeg</url>
            <title>Stories by Sharath Reddy on Medium</title>
            <link>https://medium.com/@mr.sharath.7?source=rss-3795b55ce854------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 18:07:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mr.sharath.7/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Beyond the Credit Wall: How I Built a Free, Terminal-Native AI Coding Agent Using OpenCode and…]]></title>
            <link>https://medium.com/@mr.sharath.7/beyond-the-credit-wall-how-i-built-a-free-terminal-native-ai-coding-agent-using-opencode-and-146555b4de13?source=rss-3795b55ce854------2</link>
            <guid isPermaLink="false">https://medium.com/p/146555b4de13</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[open-code]]></category>
            <dc:creator><![CDATA[Sharath Reddy]]></dc:creator>
            <pubDate>Tue, 28 Jul 2026 16:02:18 GMT</pubDate>
            <atom:updated>2026-07-28T16:02:18.878Z</atom:updated>
            <content:encoded><![CDATA[<h3>Beyond the Credit Wall: How I Built a Free, Terminal-Native AI Coding Agent Using OpenCode and OmniRoute</h3><p>If you’ve spent any time coding with terminal-native AI assistants, you’ve probably hit the “credit wall.” Proprietary tools like Claude Code are incredibly capable, but they come tied to subscription limits, token consumption counters, and rate-limiting blocks that always seem to trigger right when you are in the middle of a flow state.</p><p>For developers who prefer complete customization and control, local-first architectures offer an exciting escape hatch.</p><p>By combining two powerful open-source tools — OpenCode and OmniRoute — I built a 100% free, terminal-native AI coding agent pipeline on my Mac. It runs seamlessly, bypasses credit blocks, and gives me access to top-tier reasoning models. Here is how it works and how you can set it up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o3JexzBPG5NJXnGi5DLE1A.jpeg" /></figure><h3>The Stack: OpenCode + OmniRoute</h3><p>To build a custom assistant, you need two things: an agent to do the work, and a gateway to route the models.</p><ul><li><strong>The Agent (OpenCode):</strong> An open-source, terminal-native AI agentic coding framework. It runs directly in your CLI (similar to Claude Code) with an interactive Terminal User Interface (TUI). It can read your repository, spin up subagents for complex tasks, edit files, and execute terminal commands.</li><li><strong>The Gateway (OmniRoute):</strong> An open-source, local-first AI gateway that runs in the background. It acts as an OpenAI-compatible proxy, allowing you to pool multiple provider credentials, manage rate-limiting fallbacks, and route requests to over 200 models.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GQTzSqVqDk-8yaZvjpVBag.png" /></figure><h3>Step-by-Step Configuration</h3><p>Setting up the integration is straightforward.</p><p>First, start the local gateway server in background daemon mode so it runs persistently:</p><p>omniroute serve --daemon</p><p>Next, run the configuration setup to automatically wire the OmniRoute plugin into your OpenCode installation:</p><p>omniroute setup opencode --non-interactive</p><p>To let the two tools communicate securely, we configure a local API key. In your local OmniRoute environment file, define a custom key (like sk-omniroute-admin-key-2026). Then, add this key to your OpenCode credentials file (auth.json) under the &quot;omniroute&quot; provider. Restart the OmniRoute daemon to apply the change, and the handshake is complete.</p><h3>Selecting the Right Models</h3><p>One of the best features of this setup is model flexibility. You can route to free models, or add your own developer API keys.</p><h3>1. Defaulting to DeepSeek V4 Flash</h3><p>To ensure everything launches immediately without selecting a model every time, you can configure your OpenCode settings file (opencode.jsonc) to set <a href="https://medium.com/u/85bb187d4afc">DeepSeek</a><strong> V4 Flash Free</strong> as the default model for both the primary coding agent and the background explore subagents.</p><p>This model runs natively through the public OpenCode Zen tier, is blisteringly fast, and is highly optimized for software engineering tasks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UruAOQ3gXrFFQM2IeUWdiw.png" /></figure><h3>2. Bypassing Upstream Blocks</h3><p>Public free endpoints (like OpenCode Zen) use Cloudflare protection, which blocks proxy requests with authentication errors. To bypass this, add the following flag to your local OmniRoute environment (.env) file: OPENCODE_SYNTHESIZE_CLI_HEADERS=true</p><p>This tells OmniRoute to automatically synthesize official OpenCode CLI headers, bypassing <a href="https://medium.com/u/a00e599743a7">Cloudflare</a>’s firewall seamlessly.</p><h3>3. Integrating Nvidia NIM</h3><p>If you want to use large-scale models like <a href="https://medium.com/u/b2cb71b98e5b">LlamaIndex</a><strong> 3.3 70B Instruct</strong> for heavy reasoning, you can log in to the <a href="https://medium.com/u/ab69c39a85e1">NVIDIA AI</a> Build console to get a developer API key. It comes with 1,000 free credits — plenty to last for months of daily coding.</p><p>Add your Nvidia key directly to your OpenCode credentials file, and you can spin up the chat using Llama 3.3 whenever you need extra reasoning power: opencode -m nvidia/meta/llama-3.3-70b-instruct</p><h3>Real-world Test Drive: Codebase Audit</h3><p>To test this configuration, I navigated to a local project repository and ran a codebase-wide audit by simply typing: opencode</p><p>This spun up the TUI instantly using the free DeepSeek V4 Flash model. I prompted the assistant to find bugs in the entire codebase.</p><p>OpenCode automatically spawned a background explore subagent. Running on DeepSeek:</p><ul><li>The subagent made <strong>44 tool calls</strong> (scanning files, running greps, auditing configurations).</li><li>It completed the entire scan in <strong>3 minutes and 3 seconds</strong>.</li><li>It successfully identified <strong>38 bugs and security vulnerabilities</strong>, categorizing them from critical database leaks to frontend styling bugs.</li></ul><p>The speed and precision of the free DeepSeek model made the workflow incredibly smooth, with zero delay and no token counts ticking away.</p><h3>Key Takeaways</h3><p>Building this setup proved that you don’t need to pay monthly subscription fees or buy expensive API credits to have a high-performance AI coding assistant:</p><ol><li><strong>Complete Customization:</strong> You can configure the background subagents and the primary coding agent to use different models, routing lighter tasks to faster models to save time.</li><li><strong>Infinite Value:</strong> By leveraging public endpoints and free developer keys, the running costs are effectively zero.</li><li><strong>Privacy and Control:</strong> Your gateway runs locally on your machine, giving you full audit logs and metrics on what data is leaving your terminal.</li></ol><p>If you like terminal-native workflows and want to customize your AI coding pipeline, combining OpenCode and OmniRoute is a stack worth exploring.</p><p>#OpenCode #OmniRoute #ClaudeCode #DeepSeek #NvidiaNIM #Llama3 #AICoding #AICodingAgent #OpenSource #DevTools #DeveloperTools #LocalAI #LLM #SoftwareEngineering #CLI</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=146555b4de13" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I Instrumented KAIROS’s Causal AI LLM on Modal with SigNoz LLM Observability in 30 Minutes]]></title>
            <link>https://medium.com/@mr.sharath.7/how-i-instrumented-kaiross-causal-ai-llm-on-modal-with-signoz-llm-observability-in-30-minutes-15bff5a645b1?source=rss-3795b55ce854------2</link>
            <guid isPermaLink="false">https://medium.com/p/15bff5a645b1</guid>
            <category><![CDATA[signoz]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[opentelemetry]]></category>
            <dc:creator><![CDATA[Sharath Reddy]]></dc:creator>
            <pubDate>Thu, 16 Jul 2026 12:03:40 GMT</pubDate>
            <atom:updated>2026-07-16T12:03:40.912Z</atom:updated>
            <content:encoded><![CDATA[<p>Can you monitor a serverless AI agent without getting buried in SaaS costs?</p><p>When building <strong>KAIROS</strong> — our AI-powered Causal Analysis platform designed to discover second-order technological consequences in news data — we faced a significant challenge. KAIROS operates a two-stage live pipeline: an orchestrator pulls daily tech news, triggers an analysis agent, runs inference on a fine-tuned model, and writes structured outputs to Google Cloud Firestore.</p><p>For the core reasoning engine, we use our fine-tuned LLaMA-3 model (<a href="https://huggingface.co/SharathReddy/kairos-llama3-finetune"><strong>SharathReddy/kairos-llama3-finetune</strong></a>). We host this model on <strong>Modal</strong> (the kairos-hf-endpoint app) running on serverless <strong>NVIDIA L4 GPUs</strong>. Serverless workloads are transient, highly dynamic, and subject to cold starts. Traditional APM tools either charge massive premiums for AI tracing or fail to capture the short lifecycles of serverless containers.</p><p>To solve this, we decided to self-host <a href="https://medium.com/u/58a6e6997fea">SigNoz</a> on Google Cloud Platform (GCP). Within 30 minutes, we provisioned the VM, configured a secure Virtual Private Cloud (VPC), and instrumented our LLaMA-3 inference container using standard OpenTelemetry gen_ai semantic conventions.</p><p>However, getting the telemetry pipeline to run smoothly was far from seamless. We hit multiple network routing, path resolution, and serverless “warm start” bugs. Here is a genuine, step-by-step review of our self-hosting deployment, the debugging gotchas we solved, and our honest take on <a href="https://medium.com/u/58a6e6997fea">SigNoz</a> as an LLM observability platform.</p><p>“If you’re setting up your own self-hosted collector, you can check out the source code and installation parameters on the official <a href="https://github.com/SigNoz"><strong>SigNoz GitHub Repository</strong></a>, or reference their comprehensive <a href="https://signoz.io/docs/"><strong>SigNoz Documentation</strong></a> for alternative cloud provider configurations.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M3tLXJHnUl1dGMlgIsxO4g.jpeg" /></figure><h3>🎥 KAIROS Causal Observability in Action</h3><p>To see this entire serverless tracing pipeline in action, check out the video overview we put together. It visualizes how data hops across Google Cloud Functions into our serverless Modal GPU container, how we solved the warm-start gotchas, and how SigNoz registers these traces and correlated logs in real time:</p><p><a href="https://www.youtube.com/watch?v=OWr68JoTf8k"><strong>Watch the KAIROS Causal Observability with Signoz — Teaser on YouTube</strong></a><strong>:</strong></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FOWr68JoTf8k%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DOWr68JoTf8k&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FOWr68JoTf8k%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ffe98f09b4027bfa393462c9d07b40dc/href">https://medium.com/media/ffe98f09b4027bfa393462c9d07b40dc/href</a></iframe><h3>The KAIROS Architecture Stack</h3><p>Before diving into the observability layer, here is how the KAIROS pipeline is structured:</p><ol><li><strong>Frontend</strong>: A React/Vite dashboard styled with Tailwind CSS, hosted on Firebase (<a href="https://kairos-skr.web.app/">https://kairos-skr.web.app/</a>).</li><li><strong>Orchestration Layer</strong>: Deployments on GCP Cloud Run (built on Google Cloud Functions): <strong>news-scraper-v2 agent , </strong><strong>analysis-agent-v2 agent</strong></li><li><strong>Model Repository</strong>: Our fine-tuned model, <a href="https://huggingface.co/SharathReddy/kairos-llama3-finetune">SharathReddy/kairos-llama3-finetune</a> (trained using QLoRA from a Gemini-generated dataset), stored on Hugging Face.</li><li><strong>Inference Compute</strong>: Serverless containers running on Modal (the kairos-hf-endpoint app) running inference on NVIDIA L4 GPUs.</li><li><strong>Database</strong>: Google Cloud Firestore, storing raw articles and analyzed causal chains (including confidence scores).</li><li><strong>Observability Host</strong>: <a href="https://medium.com/u/58a6e6997fea">SigNoz</a> core server deployed via Docker Compose on a GCP Compute Engine VM.</li></ol><pre>[React Frontend (Firebase)] ---&gt; [Firestore Database]<br>                                       ^<br>                                       | (Real-time updates)<br>[Cloud Scheduler] ---&gt; [news-scraper-v2 (GCP)] ---&gt; [analysis-agent-v2 (GCP)]<br>                                                              |<br>                                                              v (Trigger Inference)<br>                                                   [kairos-hf-endpoint (Modal L4 GPU)]<br>                                                              |<br>                                                              +---&gt; (Traces &amp; Logs) ---&gt; [SigNoz VM (Port 4317)]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*McUpx0lnAfcdcn2wc2AUiw.png" /></figure><h3>Part 1: Step-by-Step Self-Hosting Deployment</h3><p>We wanted to self-host <a href="https://medium.com/u/58a6e6997fea">SigNoz</a> to qualify for the WeMakeDevs hackathon warm-up criteria and to evaluate its footprint. Since our main KAIROS backend is on GCP, we leveraged a new GCP account with ₹23,400 (approx. $300) in trial credits.</p><h3>1. Provisioning the Virtual Machine</h3><p>ClickHouse (the columnar database <a href="https://medium.com/u/58a6e6997fea">SigNoz</a> uses for high-throughput logs and traces) is resource-hungry. While GCP offers a free e2-micro instance (2 vCPUs, 1 GB RAM) in its permanent free tier, trying to run ClickHouse on it will crash the database almost immediately.</p><p>To give ClickHouse enough headroom, we provisioned an <strong>e2-standard-2</strong> instance (2 vCPUs, 8 GB RAM) on GCP Compute Engine running Ubuntu 24.04 LTS. We attached a 50 GB Balanced Persistent Disk to handle the OS and incoming trace files.</p><h3>2. Gotcha #1: The VPC Network Isolation Trap</h3><p>During firewall setup, we created custom rules to open:</p><ul><li>Port <strong>8080 (TCP)</strong>: The frontend Web UI.</li><li>Port <strong>4317 (TCP)</strong>: gRPC telemetry ingestion.</li><li>Port <strong>4318 (TCP)</strong>: HTTP telemetry ingestion.</li></ul><p>However, when we tested loading the web dashboard, it timed out.</p><p><strong>The Bug</strong>: During firewall creation, we had accidentally created a brand new, isolated VPC network called allow-signoz-ingestion. However, GCP had automatically launched the signoz-core-server VM inside the account&#39;s default VPC network. Because the firewall rules and the VM existed on two entirely separate networking islands, the ports remained blocked.</p><p><strong>The Fix</strong>: We deleted the isolated firewall rules and recreated them directly inside the default VPC network as signoz-ports-default targeting &quot;All instances in the network&quot; with 0.0.0.0/0 source IP ranges.</p><h3>3. Deploying via Foundry</h3><p>Instead of using deprecated bash installer scripts, we deployed using <a href="https://medium.com/u/58a6e6997fea">SigNoz</a>’s modern setup utility, <strong>Foundry</strong>.</p><pre># Update packages<br>sudo apt-get update &amp;&amp; sudo apt-get upgrade -y<br><br># Install Docker<br>curl -fsSL https://get.docker.com | sh<br>sudo usermod -aG docker $USER</pre><p><em>Note: After adding your user to the </em><em>docker group, you must run </em><em>newgrp docker in your active shell to apply the group changes without logging out.</em></p><p>Next, download the Foundry control binary:</p><pre>curl -fsSL https://signoz.io/foundry.sh | bash</pre><h3>4. Gotcha #2: The Missing Local PATH Bug</h3><p>After running the Foundry installer, running the deploy script threw a -bash: ./foundryctl: No such file or directory error.</p><p><strong>The Bug</strong>: The install script successfully placed the foundryctl binary inside the user&#39;s local bin folder at /home/greywolf_dev1/.local/bin/. Because this folder was not included in our system&#39;s PATH variable, shell executors could not locate it.</p><p><strong>The Fix</strong>: We updated our environment variables by running:</p><pre>echo &#39;export PATH=&quot;$HOME/.local/bin:$PATH&quot;&#39; &gt;&gt; ~/.bashrc<br>source ~/.bashrc</pre><p>We created a deployment config file named casting.yaml:</p><pre>apiVersion: v1alpha1<br>kind: Installation<br>metadata:<br>  name: signoz<br>spec:<br>  deployment:<br>    flavor: compose<br>    mode: docker</pre><p>And finally triggered the installation:</p><pre>foundryctl cast -f casting.yaml</pre><p>The engine checked our system specs, downloaded ClickHouse, PostgreSQL (metadata), and the OTel collectors, and spun up the containers on port 8080 in under 3 minutes.</p><h3>Part 2: OpenTelemetry Code Integration &amp; Serverless Gotchas</h3><p>Once the dashboard at <a href="http://35.231.67.167:8080/">http://35.231.67.167:8080</a> loaded successfully and we completed our onboarding account setup, we began integrating telemetry into KAIROS&#39;s serverless Modal backend. To avoid disrupting our main kairos-hf-endpoint production pipeline, we created a local dev copy (kairos_dev.py) and deployed it as a temporary Modal application: kairos-signoz-dev.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NcYjA4cbtY-uVnOda19kmw.jpeg" /></figure><h3>Gotcha #3: The Serverless Warm-Start Telemetry Silence</h3><p>To trace requests in kairos_dev.py, we initially registered our TracerProvider inside the FastAPI endpoint function block and called provider.shutdown() before returning the API response.</p><p>The first request worked perfectly. However, subsequent requests were received by the endpoint but <strong>never</strong> appeared in <a href="https://medium.com/u/58a6e6997fea">SigNoz</a>.</p><p><strong>The Bug</strong>: Modal maintains “warm starts” where container processes are kept alive to process subsequent requests instantly.</p><ul><li>During the first request, the TracerProvider was initialized, registered globally, and then shut down.</li><li>During the second request, our code tried to create a new TracerProvider and register it. OpenTelemetry blocked this, logging: Overriding of current TracerProvider is not allowed.</li><li>Because OpenTelemetry fell back to the globally registered provider — which had already been closed via .shutdown()—all subsequent spans were quietly discarded.</li></ul><p><strong>The Fix</strong>: We refactored our code to use a lazy-initialization global caching pattern. We cache the providers, tracers, and loggers in module-level global variables. Instead of calling .shutdown(), we invoke .force_flush(), which pushes all buffered logs and spans to our <a href="https://medium.com/u/58a6e6997fea">SigNoz</a> endpoint while keeping the communication channels open for the next request.</p><p>Here is the robust, serverless-compatible implementation:</p><pre># Global variables to cache telemetry resources across warm starts<br>_provider = None<br>_logger_provider = None<br>_tracer = None<br>_logger = None<br><br>def get_otel_resources():<br>    global _provider, _logger_provider, _tracer, _logger<br>    if _provider is not None:<br>        return _tracer, _logger, _provider, _logger_provider<br>    <br>    import logging<br>    from opentelemetry import trace<br>    from opentelemetry.sdk.trace import TracerProvider<br>    from opentelemetry.sdk.trace.export import BatchSpanProcessor<br>    from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter<br>    from opentelemetry.sdk.resources import Resource<br><br>    from opentelemetry._logs import set_logger_provider<br>    from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler<br>    from opentelemetry.sdk._logs.export import BatchLogRecordProcessor<br>    from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter<br><br>    # Configure a shared resource with the KAIROS service name<br>    resource = Resource.create(attributes={&quot;service.name&quot;: &quot;kairos-modal-dev&quot;})<br>    <br>    # 1. Traces Setup<br>    _provider = TracerProvider(resource=resource)<br>    processor = BatchSpanProcessor(<br>        OTLPSpanExporter(endpoint=os.environ[&quot;OTEL_EXPORTER_OTLP_ENDPOINT&quot;], insecure=True)<br>    )<br>    _provider.add_span_processor(processor)<br>    trace.set_tracer_provider(_provider)<br>    _tracer = trace.get_tracer(&quot;kairos.dev.tracer&quot;)<br>    <br>    # 2. Logs Setup<br>    _logger_provider = LoggerProvider(resource=resource)<br>    log_processor = BatchLogRecordProcessor(<br>        OTLPLogExporter(endpoint=os.environ[&quot;OTEL_EXPORTER_OTLP_ENDPOINT&quot;], insecure=True)<br>    )<br>    _logger_provider.add_log_record_processor(log_processor)<br>    set_logger_provider(_logger_provider)<br>    <br>    # Integrate OpenTelemetry with Python&#39;s standard logging library<br>    handler = LoggingHandler(level=logging.INFO, logger_provider=_logger_provider)<br>    _logger = logging.getLogger(&quot;kairos.dev.logger&quot;)<br>    if not _logger.handlers:<br>        _logger.addHandler(handler)<br>    _logger.setLevel(logging.INFO)<br>    <br>    return _tracer, _logger, _provider, _logger_provider</pre><h3>Part 3: Adopting SigNoz LLM Observability</h3><p>To monitor our fine-tuned LLaMA-3 model, we bypassed generic custom string attributes and adopted the <strong>OpenTelemetry standard </strong><strong>gen_ai semantic conventions</strong>. This allows SigNoz&#39;s built-in LLM Observability components to natively parse and display our AI workloads.</p><p>Here is how we set up the inference handler block in our Modal app:</p><pre>@app.function(secrets=[signoz_env])<br>@modal.fastapi_endpoint(method=&quot;POST&quot;)<br>async def test_inference(request: Request):<br>    from opentelemetry import trace<br>    <br>    tracer, logger, provider, logger_provider = get_otel_resources()<br>    <br>    with tracer.start_as_current_span(&quot;sandbox_llm_execution&quot;) as span:<br>        try:<br>            body = await request.json()<br>            input_text = body.get(&quot;text&quot;, &quot;Default test string&quot;)<br>            <br>            # Record prompt-phase semantic conventions<br>            prompt_tokens = len(input_text) // 4 + 1  # Character estimation<br>            span.set_attribute(&quot;gen_ai.system&quot;, &quot;huggingface&quot;)<br>            span.set_attribute(&quot;gen_ai.request.model&quot;, &quot;kairos-llama3-finetune&quot;)<br>            span.set_attribute(&quot;gen_ai.response.model&quot;, &quot;kairos-llama3-finetune&quot;)<br>            span.set_attribute(&quot;gen_ai.request.temperature&quot;, 0.7)<br>            span.set_attribute(&quot;gen_ai.usage.prompt_tokens&quot;, prompt_tokens)<br>            <br>            logger.info(f&quot;Processing input text: {input_text}&quot;)<br>            <br>            # Simulate a handled causal analysis exception (e.g., Firestore network failure)<br>            if &quot;fail&quot; in input_text.lower() or &quot;error&quot; in input_text.lower():<br>                raise ValueError(f&quot;Intentional sandbox error triggered by input: {input_text}&quot;)<br>            <br>            # Mock successful causal discovery response<br>            response_text = f&quot;Processed text successfully: {input_text[:20]}...&quot;<br>            completion_tokens = len(response_text) // 4 + 1<br>            <br>            # Record completion-phase conventions<br>            span.set_attribute(&quot;gen_ai.usage.completion_tokens&quot;, completion_tokens)<br>            span.set_attribute(&quot;gen_ai.usage.total_tokens&quot;, prompt_tokens + completion_tokens)<br>            <br>            logger.info(&quot;Inference completed successfully!&quot;)<br>            <br>            return {<br>                &quot;status&quot;: &quot;dev_success&quot;,<br>                &quot;message&quot;: response_text<br>            }<br>        except Exception as e:<br>            logger.error(f&quot;Error occurred during LLM processing: {str(e)}&quot;, exc_info=True)<br>            span.record_exception(e)<br>            span.set_status(trace.StatusCode.ERROR, str(e))<br>            <br>            # Record partial usage stats on error<br>            span.set_attribute(&quot;gen_ai.usage.completion_tokens&quot;, 0)<br>            span.set_attribute(&quot;gen_ai.usage.total_tokens&quot;, prompt_tokens)<br>            <br>            return {<br>                &quot;status&quot;: &quot;dev_error&quot;,<br>                &quot;message&quot;: str(e)<br>            }<br>        finally:<br>            # Force flush telemetry buffers without tearing down connection channels<br>            provider.force_flush()<br>            logger_provider.force_flush()</pre><h3>Part 4: Honest Review of SigNoz (Pros &amp; Cons)</h3><p>After testing our setup with multiple warm-start successes, simulated database timeouts, and invalid input strings, we got a clear picture of self-hosting SigNoz. Here is our honest developer review:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fr4NSydCsgE8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dr4NSydCsgE8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fr4NSydCsgE8%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/227aaa04ca3716e7c857ee194794b09e/href">https://medium.com/media/227aaa04ca3716e7c857ee194794b09e/href</a></iframe><h3>What We Liked (The Pros)</h3><ol><li><strong>OpenTelemetry-Native Foundation</strong>: <a href="https://medium.com/u/58a6e6997fea">SigNoz</a> is built completely on OpenTelemetry standards. Setting up semantic conventions like gen_ai meant we didn&#39;t have to install any proprietary <a href="https://medium.com/u/58a6e6997fea">SigNoz</a> SDKs. Our code remains 100% vendor-agnostic.</li><li><strong>Seamless Log-Trace Correlation</strong>: If an LLM call fails, you don’t have to manually match timestamps across different files. Clicking on the trace span immediately opens the corresponding logger.error traceback output in the right-hand panel. This alone saves massive debugging time.</li><li><strong>Microsecond Precision</strong>: The traces explorer and flamegraphs load instantly. Finding latency spikes and tracking which model execution step dragged the request is visual and intuitive.</li></ol><h3>Areas for Improvement (The Cons)</h3><ol><li><strong>Heavier Resource Footprint</strong>: ClickHouse is incredibly fast, but it is not built for tiny virtual machines. Running <a href="https://medium.com/u/58a6e6997fea">SigNoz</a> requires at least an e2-standard-2 (8 GB RAM) VM, costing roughly $25–$30/month on GCP if you run out of trial credits. We would love to see a lighter-weight dev mode (perhaps using SQLite or DuckDB) for rapid local testing.</li><li><strong>Steep Dashboard Learning Curve</strong>: While the out-of-the-box logs and traces explorers are excellent, creating a custom dashboard to track specialized LLM metrics (like token counts or model accuracy) requires writing raw ClickHouse SQL queries. An easier visual widget builder would make dashboard customization faster for beginners.</li></ol><h3>Conclusion &amp; Hackathon Week Roadmap</h3><p>Self-hosting SigNoz for our serverless LLM pipeline proved highly successful. It gave us microsecond-level tracing, simplified our debugging cycles, and set a robust foundation for our upcoming build week.</p><p>As we move into the building phase of the hackathon starting <strong>July 20th</strong>, we plan to integrate <a href="https://medium.com/u/58a6e6997fea">SigNoz</a> deeper into the root of our KAIROS architecture:</p><ul><li>Auto-instrumenting our GCP Cloud Run orchestration services (news-scraper-v2 and analysis-agent-v2) to track incoming requests and propagate trace context headers.</li><li>Tracing the main React frontend app (https://kairos-skr.web.app/) using the @opentelemetry/sdk-trace-web package for full end-to-end user-to-LLM tracing.</li><li>Monitoring LLaMA-3 model download latencies from Hugging Face.</li><li>Designing custom cost dashboards using <a href="https://medium.com/u/58a6e6997fea">SigNoz</a>’s native ClickHouse query engine.</li></ul><p><em>Are you building serverless AI agents? Give self-hosted </em><a href="https://medium.com/u/58a6e6997fea"><em>SigNoz</em></a><em> and OpenTelemetry a try to bring complete visibility to your AI infrastructure.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15bff5a645b1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Announcing Vidhaan AI v0.2]]></title>
            <link>https://medium.com/@mr.sharath.7/announcing-vidhaan-ai-v0-2-961bb700de3d?source=rss-3795b55ce854------2</link>
            <guid isPermaLink="false">https://medium.com/p/961bb700de3d</guid>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[legal-ai]]></category>
            <category><![CDATA[indian-law]]></category>
            <category><![CDATA[generative-ai]]></category>
            <dc:creator><![CDATA[Sharath Reddy]]></dc:creator>
            <pubDate>Sat, 27 Jun 2026 17:53:33 GMT</pubDate>
            <atom:updated>2026-06-27T17:54:36.357Z</atom:updated>
            <content:encoded><![CDATA[<h3><em>Announcing Vidhaan AI v0.2: Bridging Statutory Retrieval with a Pinned-Citations Legal Workbench</em></h3><h3>Introduction</h3><p>Legal research is traditionally fragmented. A researcher queries a database, opens multiple PDF browser tabs, copies relevant acts, and pastes them into a word processor to begin drafting arguments.</p><p><strong>Vidhaan AI</strong> was built to unify this loop. In v0.1, we introduced our sovereign, hybrid dense-sparse RAG search engine mapping Indian legislative acts and constitution articles. Today, we are releasing <strong>Version 0.2</strong>, introducing <strong>“My Notebook”</strong> — an interactive workspace that lets researchers transition from discovery to synthesis inside a single workspace.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x2xRWhdZjHMuIfbkwUVY0g.png" /></figure><h3>The Heart of v0.2: “My Notebook” Workspace</h3><p>Finding statutory references is only useful if you can organize them. In v0.2, users have access to a dedicated note-taking whiteboard split into a clean columns layout:</p><ol><li><strong>Pinned Source Drawer</strong>: Next to every statutory citation in your chat conversation, you will now see a <strong>Pin</strong> toggle. Clicking it locks the citation, section number, and Act title into your sidebar notebook dashboard.</li><li><strong>Inline Research Annotations</strong>: For each pinned reference, researchers can add notes, court arguments, or annotations directly onto the card.</li><li><strong>Drafting Whiteboard</strong>: A responsive rich text editor on the main screen allows you to compile legal outlines. You can insert references with one click, copy the draft to your clipboard, export as a text brief, or print outlines instantly.</li></ol><h3>sovereign Fullscreen PWA Mobilization</h3><p>Statutory research doesn’t just happen at a desk. Vidhaan AI is now fully packaged as an installable <strong>Progressive Web App (PWA)</strong>:</p><ul><li><strong>Android/Chrome</strong>: Launches native browser installation.</li><li><strong>iOS/Safari</strong>: Automatically detects Apple devices and guides users to add it to their home screen with a one-click instruction banner.</li><li><strong>Immersive Display</strong>: Running in standalone mode hides the browser URL and navigation chrome, rendering the workbench as a native full-screen mobile application.</li></ul><h3>Technical Pipeline: Under the Hood</h3><p>To support this research flow, the platform operates on an institutional-grade hybrid pipeline:</p><ul><li><strong>Ingestion Node</strong>: Indian legislative acts are parsed, cleaned of OCR noise, split into semantic chunks, and embedded into 768-dimensional vector space using Google Gemini embeddings.</li><li><strong>PostgreSQL &amp; pgvector</strong>: Store chunk text, vectors, and user citation nodes securely.</li><li><strong>Hybrid Dense-Sparse RAG</strong>: Merges cosine vector similarity (dense) and BM25 keywords search (sparse) using Reciprocal Rank Fusion (RRF) algorithms.</li><li><strong>Grounded Synthesis</strong>: Grounded directly in verified statutory database records using Llama-3.3-70b-versatile, producing footnoted insights with zero AI preamble fluff.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_Q2iZAhRhs6Dl5-hxjA0iQ.png" /></figure><h3>We Want Your Feedback</h3><p>We are actively seeking feedback from lawyers, students, and legal-tech enthusiasts. What features would you like to see in version 0.3? Automated case summaries? Multi-document citation matching?</p><p><a href="https://www.vidhaanai.online/">Vidhaan AI | Production-Grade Indian Legal Assistant Platform</a></p><p>Try the platform today and let us know your thoughts in the comments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=961bb700de3d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Built a Real-Time Causal Analysis AI with Llama 3 and Free Cloud Services. Here’s How.]]></title>
            <link>https://medium.com/@mr.sharath.7/i-built-a-real-time-causal-analysis-ai-with-llama-3-and-free-cloud-services-heres-how-27ebc918e20a?source=rss-3795b55ce854------2</link>
            <guid isPermaLink="false">https://medium.com/p/27ebc918e20a</guid>
            <category><![CDATA[firebase]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[news-analysis]]></category>
            <category><![CDATA[llama-3]]></category>
            <dc:creator><![CDATA[Sharath Reddy]]></dc:creator>
            <pubDate>Sat, 05 Jul 2025 23:47:02 GMT</pubDate>
            <atom:updated>2025-07-05T23:47:02.879Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Moving beyond basic summarization to predict second-order effects. A detailed breakdown of the architecture, the fine-tuning process, and the multi-cloud deployment of the KAIROS project.</em></p><p><a href="https://kairos-skr.web.app/">KAIROS</a></p><p>In the world of AI, it’s easy to create a simple text summarizer. But what if we could build something more? What if an AI could not only understand an event but also reason about its non-obvious, downstream consequences?</p><p>That was the goal behind my latest project, <strong>KAIROS</strong>: an autonomous AI pipeline that fetches live financial news and performs a deep causal analysis to predict its second-order effects. And I built it entirely from scratch using open-source models and free-tier cloud services.</p><p>This article is a deep dive into the architecture, the technology choices, and the challenges of building a real-world AI application.</p><p><strong>You can see the final, live application here:</strong> <a href="https://kairos-skr.web.app/">https://kairos-skr.web.app/</a></p><h3>The Core Idea: From Information to Insight</h3><p>The internet is flooded with news. The challenge isn’t access to information, but the extraction of true insight. KAIROS was designed to tackle this by answering questions like: “If the central bank raises interest rates, what are the non-obvious ripple effects on specific market sectors, and what is the AI’s confidence in that prediction?”</p><p>To achieve this, the project was broken into two main phases: training a specialized AI “brain” and building an automated pipeline to feed it live data.</p><h3>Phase 1: Creating the Custom AI Brain</h3><p>The heart of KAIROS is a fine-tuned version of Meta’s Llama 3 8B model. A general-purpose model can summarize, but it can’t reliably perform nuanced causal analysis without specialized training.</p><ol><li><strong>Dataset Generation (Model Distillation):</strong> Manually creating a dataset for this task would be slow and require deep domain expertise. Instead, I used a technique called model distillation. A powerful “teacher” model (Google’s Gemini 1.5 Pro) was prompted with raw news articles and a complex JSON schema. It generated hundreds of high-quality training examples, each mapping a news event to its summary, type, direct impacts, and predicted second-order hypotheses with reasoning and confidence scores.</li><li><strong>Fine-Tuning (QLoRA):</strong> Using a free T4 GPU on Google Colab, I performed Parameter-Efficient Fine-Tuning (PEFT) using the QLoRA method. This allowed me to train the massive 8-billion-parameter model on a single, free-tier GPU. The result was a lightweight LoRA “adapter” that imbues the base Llama 3 model with its new analytical skill.</li><li><strong>You can find the final fine-tuned model adapter on the Hugging Face Hub:</strong> @<a href="https://huggingface.co/SharathReddy/kairos-llama3-finetune">huggingface.co/SharathReddy/kairos-llama3-finetune</a></li></ol><h3>Phase 2: The Automated, Multi-Cloud Pipeline</h3><p>With the model trained, the next step was to build a system to use it in real time. I chose a serverless, event-driven architecture to ensure scalability and cost-effectiveness (zero cost on the free tiers).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CHIZ-fA_HwCDWhlVrr_eIg.png" /></figure><ol><li><strong>The Trigger (@Google Cloud Scheduler):</strong> A cron job runs every hour, initiating the entire process by calling an HTTP endpoint.</li><li><strong>The Scraper (@Google Cloud Functions):</strong> A 1st Gen Cloud Function receives the trigger, fetches the top 5 trending financial news articles using the NewsAPI, saves the raw data to Firestore, and then triggers the next step.</li><li><strong>The Orchestrator (@Google Cloud Run):</strong> A 2nd Gen Cloud Run service acts as the central agent. It receives the list of new articles from the scraper and, for each one, makes an API call to our deployed model.</li><li><strong>The Inference Endpoint (@Hugging Face Spaces):</strong> The Llama 3 model and the trained adapter are deployed on a Hugging Face Space running on a GPU. This provides a fast, reliable API endpoint. The Gradio interface also allows for direct testing.<br><strong>Interact with the live API here:</strong> <a href="https://huggingface.co/spaces/SharathReddy/kairos">@SharathReddy/kairos Space</a></li><li><strong>The Database (@Firebase Firestore):</strong> All raw articles and the final structured analyses from the AI are stored in Firestore, a NoSQL real-time database.</li><li><strong>The Dashboard (@Firebase Hosting):</strong> A responsive frontend built with vanilla JavaScript and Tailwind CSS connects directly to Firestore. It uses a real-time listener to display new analyses the moment they are saved, providing a live, constantly updating view of the AI’s work.</li></ol><h3>Conclusion</h3><p>Building KAIROS was a journey through the entire modern AI stack — from data engineering and advanced fine-tuning techniques to multi-cloud serverless architecture and frontend development. It demonstrates that with the right tools and approach, it’s possible to build truly intelligent, impactful systems that go far beyond simple demos.</p><p><em>This project was created by Sharath Kumar Reddy. You can find more of my work at my personal site: </em><a href="http://skr.lovable.app/"><em>skr.lovable.app</em></a></p><p>#ArtificialIntelligence #LLM #MachineLearning #MLOps #GoogleCloud #HuggingFace #Llama3 #FineTuning #Serverless #DataScience #Portfolio</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=27ebc918e20a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ Deep Quantile Learning for VaR & ES: Modeling Market Risk with Mogrifier RNNs & GANs ]]></title>
            <link>https://medium.com/@mr.sharath.7/deep-quantile-learning-for-var-es-modeling-market-risk-with-mogrifier-rnns-gans-8f5102f373d1?source=rss-3795b55ce854------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f5102f373d1</guid>
            <category><![CDATA[generative-adversarial]]></category>
            <category><![CDATA[expected-shortfall]]></category>
            <category><![CDATA[quantile-regression]]></category>
            <category><![CDATA[value-at-risk]]></category>
            <dc:creator><![CDATA[Sharath Reddy]]></dc:creator>
            <pubDate>Sun, 04 May 2025 15:42:17 GMT</pubDate>
            <atom:updated>2025-05-28T02:23:25.805Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>In the complex world of finance, understanding <strong>downside risk</strong> is essential — not only for investors and hedge funds, but also for regulators and policymakers. Two key tools used for this purpose are:</p><p>🔻 <strong>Value at Risk (VaR)</strong> — How bad can things get with X% probability?<br>📉 <strong>Expected Shortfall (ES)</strong> — If we hit the tail, how much might we actually lose?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*hKHjmRhomxgk6fdf7TJrmg.jpeg" /></figure><p>But here’s the catch — traditional models like Historical Simulation, Variance-Covariance, or GARCH often struggle to:</p><ul><li>🔹 Accurately model <strong>non-linear dependencies</strong></li><li>🔹 Handle <strong>volatility clustering</strong> or <strong>long memory</strong></li><li>🔹 Reflect <strong>heterogeneous behaviors</strong> across financial actors</li></ul><p>So I set out to build a modern solution leveraging deep learning 💡</p><h3>🧠 Model Highlights</h3><p>✅ <strong>Mogrifier LSTM + Quantile Regression</strong><br>Captures temporal patterns with memory and cluster sensitivity, enabling precise estimation of <strong>VaR</strong> at arbitrary quantiles.</p><p>✅ <strong>GAN-Based Tail Scenario Generation</strong><br>A conditional GAN is trained to generate synthetic tail risk scenarios — allowing us to estimate <strong>ES</strong> more robustly, with better generalization than sampling historical losses.</p><p>🧪 Based on:</p><ul><li><strong>Heterogeneous Market Hypothesis</strong></li><li>Real-time inference-ready architecture</li><li>End-to-end automated forecasting pipeline</li></ul><p>🔬 <strong>Best Results (Tested on BlackRock Returns)</strong></p><ul><li><strong>VaR Hit Ratio</strong>: 0.0513 (close to 0.05 target ✅)</li><li><strong>Estimated ES (0.05 quantile)</strong>: −0.336</li><li><strong>Kupiec Test p-value</strong>: 0.81 — excellent calibration!</li><li>Outperforms naive historical ES by &gt;30%</li></ul><p>💻 Full codebase includes:</p><ul><li>Modular PyTorch models</li><li>GAN + RNN training loops</li><li>Evaluation metrics (Hit Ratio, Kupiec Test)</li><li>Jupyter Notebook for visualizations 📈</li></ul><h3>📂 Coming Next</h3><ul><li>📊 Portfolio-level VaR/ES</li><li>🧩 Attention mechanisms for context-aware risk</li><li>🌐 Integration with live data streams</li></ul><p>Check out the full code &amp; notebook, and let’s reimagine financial risk modeling! 💸<br>📬 <a href="https://github.com/mr-sharath/Deep_Quantile_Regression">https://github.com/mr-sharath/Deep_Quantile_Regression</a></p><p>#DeepLearning #FinanceAI #QuantResearch #GANs #RNN #VaR #ExpectedShortfall #MachineLearning #QuantFinance #PyTorch</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f5102f373d1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an AI-Powered Multimodal Voice Assistant: A Journey into Speech, Vision, and Language…]]></title>
            <link>https://medium.com/@mr.sharath.7/building-an-ai-powered-multimodal-voice-assistant-a-journey-into-speech-vision-and-language-a8ffc5db7cbf?source=rss-3795b55ce854------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8ffc5db7cbf</guid>
            <category><![CDATA[image-analysis]]></category>
            <category><![CDATA[gradio]]></category>
            <category><![CDATA[speech-to-text-conversion]]></category>
            <category><![CDATA[llava]]></category>
            <category><![CDATA[multimodal-ai]]></category>
            <dc:creator><![CDATA[Sharath Reddy]]></dc:creator>
            <pubDate>Thu, 20 Mar 2025 05:03:06 GMT</pubDate>
            <atom:updated>2025-03-20T05:03:06.654Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building an AI-Powered Multimodal Voice Assistant: A Journey into Speech, Vision, and Language Integration</h3><p>In today’s world, AI is no longer limited to single modalities like text or images. The future lies in <strong>multimodal systems</strong> that can seamlessly integrate speech, vision, and language to create more intuitive and human-like interactions.</p><p>I recently had the opportunity to work on such a project at the <strong>University of Houston’s AI &amp; Computer Vision Lab</strong>, where I developed an <strong>AI-Powered Multimodal Voice Assistant</strong>. This system combines <strong>speech recognition</strong>, <strong>image analysis</strong>, and <strong>text-to-speech synthesis</strong> to enable natural language Q&amp;A about visual content.</p><h4>What Does It Do?</h4><p>The voice assistant allows users to:</p><ol><li><strong>Upload an image</strong> and ask questions about it via voice input.</li><li>Receive <strong>text and audio responses</strong> describing the image or answering their questions.</li></ol><p>For example, you could upload a picture of a busy street and ask, <em>“What’s happening in this image?”</em>The system would analyze the image, generate a caption, and respond with something like, <em>“The image shows a busy city street with cars, pedestrians, and tall buildings.”</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*75bos_l-ATjm1w2slN70dw.png" /><figcaption>Screenshot of the UI</figcaption></figure><h4>How Does It Work?</h4><p>The system is powered by state-of-the-art AI models:</p><ul><li><strong>Speech Recognition</strong>: OpenAI’s Whisper model transcribes spoken audio into text.</li><li><strong>Image Analysis</strong>: Microsoft’s GIT-base model generates captions and answers questions about the uploaded image.</li><li><strong>Text-to-Speech</strong>: Google’s gTTS converts text responses back into speech.</li></ul><p>The user interface is built using <strong>Gradio</strong>, a Python library that makes it easy to create interactive web interfaces for machine learning models. The system is designed to be fast, with responses generated in under <strong>2 seconds</strong>.</p><pre>import torch<br>from transformers import pipeline<br>import whisper<br>import gradio as gr<br>from gtts import gTTS<br>from PIL import Image<br>import nltk<br>import re<br>import tempfile<br>import os<br>import multiprocessing<br><br># Enable multiprocessing for MacOS<br>multiprocessing.freeze_support()<br><br># Download NLTK data<br>nltk.download(&#39;punkt&#39;, quiet=True)<br><br># Configure device<br>DEVICE = &quot;cuda&quot; if torch.cuda.is_available() else &quot;cpu&quot;<br>print(f&quot;Using device: {DEVICE}&quot;)<br><br># Initialize a smaller vision model<br>model_id = &quot;microsoft/git-base&quot;  # Using a more stable model<br>print(&quot;Loading image captioning model...&quot;)<br>pipe = None  # We&#39;ll initialize this later to avoid multiprocessing issues<br><br># Initialize Whisper model<br>print(&quot;Loading Whisper model...&quot;)<br>audio_model = None  # We&#39;ll initialize this later<br><br>def initialize_models():<br>    &quot;&quot;&quot;Initialize models safely&quot;&quot;&quot;<br>    global pipe, audio_model<br>    if pipe is None:<br>        pipe = pipeline(&quot;image-to-text&quot;, model=model_id)<br>    if audio_model is None:<br>        audio_model = whisper.load_model(&quot;medium&quot;, device=DEVICE)<br>    return pipe, audio_model<br><br>def img2txt(input_text, input_image):<br>    &quot;&quot;&quot;Process image with the vision model&quot;&quot;&quot;<br>    global pipe<br>    if pipe is None:<br>        pipe, _ = initialize_models()<br>    <br>    try:<br>        # Generate basic caption<br>        outputs = pipe(input_image)<br>        caption = outputs[0][&#39;generated_text&#39;]<br>        <br>        # If there&#39;s a specific question, append it to the response<br>        if input_text and input_text.strip():<br>            response = f&quot;Based on the image which shows {caption}, &quot;<br>            response += f&quot;addressing your question: {input_text}\n&quot;<br>            return response<br>            <br>        return caption<br>    except Exception as e:<br>        print(f&quot;Error in image processing: {str(e)}&quot;)<br>        return &quot;Sorry, I couldn&#39;t process the image properly.&quot;<br><br>def transcribe(audio):<br>    &quot;&quot;&quot;Transcribe audio using Whisper&quot;&quot;&quot;<br>    global audio_model<br>    if audio_model is None:<br>        _, audio_model = initialize_models()<br>    <br>    if audio is None:<br>        return &#39;&#39;<br>    <br>    try:<br>        audio = whisper.load_audio(audio)<br>        audio = whisper.pad_or_trim(audio)<br>        <br>        mel = whisper.log_mel_spectrogram(audio).to(audio_model.device)<br>        result = whisper.decode(audio_model, mel, whisper.DecodingOptions())<br>        <br>        return result.text<br>    except Exception as e:<br>        print(f&quot;Error in transcription: {str(e)}&quot;)<br>        return &quot;Sorry, I couldn&#39;t transcribe the audio properly.&quot;<br><br>def text_to_speech(text):<br>    &quot;&quot;&quot;Convert text to speech using gTTS&quot;&quot;&quot;<br>    try:<br>        with tempfile.NamedTemporaryFile(delete=False, suffix=&quot;.mp3&quot;) as fp:<br>            tts = gTTS(text=text, lang=&#39;en&#39;, slow=False)<br>            tts.save(fp.name)<br>            return fp.name<br>    except Exception as e:<br>        print(f&quot;Error in text-to-speech: {str(e)}&quot;)<br>        return None<br><br>def process_inputs(audio, image):<br>    &quot;&quot;&quot;Main processing function&quot;&quot;&quot;<br>    try:<br>        # Process speech to text<br>        speech_to_text_output = transcribe(audio) if audio is not None else &quot;&quot;<br>        <br>        # Process image and generate response<br>        if image is not None:<br>            query = speech_to_text_output if speech_to_text_output else &quot;Describe this image in detail&quot;<br>            chatgpt_output = img2txt(query, image)<br>        else:<br>            chatgpt_output = &quot;No image provided.&quot;<br>        <br>        # Generate audio response<br>        audio_output = text_to_speech(chatgpt_output)<br>        <br>        return speech_to_text_output, chatgpt_output, audio_output<br>        <br>    except Exception as e:<br>        print(f&quot;Error in process_inputs: {str(e)}&quot;)<br>        return str(e), str(e), None<br><br># Create Gradio interface<br>demo = gr.Interface(<br>    fn=process_inputs,<br>    inputs=[<br>        gr.Audio(sources=[&quot;microphone&quot;], type=&quot;filepath&quot;, label=&quot;Voice Input&quot;),  # Fixed Audio component syntax<br>        gr.Image(type=&quot;pil&quot;, label=&quot;Image Input&quot;)  # Specified image type<br>    ],<br>    outputs=[<br>        gr.Textbox(label=&quot;Speech to Text&quot;),<br>        gr.Textbox(label=&quot;Image Analysis&quot;),<br>        gr.Audio(label=&quot;AI Response&quot;, type=&quot;filepath&quot;)<br>    ],<br>    title=&quot;Image Analysis with Voice Interface&quot;,<br>    description=&quot;Upload an image and ask questions using your voice. The AI will analyze the image and respond with both text and speech.&quot;<br>)<br><br>if __name__ == &quot;__main__&quot;:<br>    print(&quot;Starting Gradio interface...&quot;)<br>    # Initialize models before launching the interface<br>    initialize_models()<br>    # Launch with minimal GPU memory usage<br>    demo.launch(<br>        share=True,<br>        debug=True,<br>        server_name=&quot;0.0.0.0&quot;,<br>        server_port=7860,<br>    )</pre><h4>Key Challenges and Learnings</h4><ol><li><strong>Model Integration</strong>: Combining multiple AI models into a single workflow required careful handling of dependencies and ensuring compatibility between frameworks.</li><li><strong>Latency Optimization</strong>: Achieving a response time of under 2 seconds involved optimizing model loading and inference pipelines.</li><li><strong>User Experience</strong>: Designing an intuitive interface with Gradio was crucial to making the system accessible to non-technical users.</li></ol><h4>Why This Matters</h4><p>This project is a step toward building AI systems that can understand and interact with the world the way humans do — by combining multiple senses (vision, hearing, and language). Such systems have wide-ranging applications, from <strong>assistive technologies</strong> for the visually impaired to <strong>smart home devices</strong> that can understand and respond to complex commands.</p><h4>What’s Next?</h4><p>I’m excited to continue exploring the possibilities of multimodal AI. Future improvements could include:</p><ul><li>Integrating more advanced vision-language models like <strong>LLaVA</strong> for richer image understanding.</li><li>Adding support for real-time video analysis.</li><li>Expanding the system’s capabilities to handle more complex queries and tasks.</li></ul><p>If you’re interested in AI, computer vision, or multimodal systems, I’d love to connect and hear your thoughts!</p><p><a href="https://mr-sharath.github.io/">https://mr-sharath.github.io/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8ffc5db7cbf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Movie Recommender System with Wikipedia Links ]]></title>
            <link>https://medium.com/@mr.sharath.7/building-a-movie-recommender-system-with-wikipedia-links-0bba1a218f14?source=rss-3795b55ce854------2</link>
            <guid isPermaLink="false">https://medium.com/p/0bba1a218f14</guid>
            <category><![CDATA[movie-recommendation]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[predictions]]></category>
            <category><![CDATA[embedding]]></category>
            <category><![CDATA[svm]]></category>
            <dc:creator><![CDATA[Sharath Reddy]]></dc:creator>
            <pubDate>Thu, 25 Jul 2024 05:01:46 GMT</pubDate>
            <atom:updated>2024-07-25T05:01:46.319Z</atom:updated>
            <content:encoded><![CDATA[<p>Ever wondered how to build a movie recommender system using a unique source of data? In this post, we’ll explore how to create a recommender system based on outgoing Wikipedia links. It’s an innovative approach that leverages the vast network of connections on Wikipedia to suggest movies you might enjoy. Let’s dive in! 🌟</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_l7-r7vF0mdobwjBogqWQg.png" /></figure><h3>📊 What We’ll Do</h3><ol><li><strong>Extract Training Data</strong>: We start by extracting a dataset from Wikipedia, focusing on outgoing links related to movies.</li><li><strong>Train Embeddings</strong>: Next, we’ll use these links to train movie embeddings that capture the relationships between different films.</li><li><strong>Implement a Classifier</strong>: We’ll then build a Support Vector Machine (SVM) classifier to recommend movies based on these embeddings.</li><li><strong>Predict Review Scores</strong>: Finally, we’ll use our trained embeddings to predict movie review scores, enhancing the recommendation process.</li></ol><h3>🔧 Setting Up</h3><p>First, clone the repository and install the required libraries. Make sure you have Python 3.x and the necessary packages:</p><pre>git clone https://github.com/yourusername/movie-recommender-system.git<br>cd movie-recommender-system<br>pip install -r requirements.txt</pre><h3>🏗️ How It Works</h3><h3>Data Extraction</h3><p>We extract movie data from Wikipedia, focusing on outgoing links. This data forms the foundation of our movie embeddings.</p><h3>Training Embeddings</h3><p>Using the outgoing links, we train embeddings that represent the relationships between movies. This helps us understand which movies are closely related.</p><h3>Building the Classifier</h3><p>We implement an SVM classifier that uses these embeddings to provide recommendations. This model learns to suggest movies based on their connections.</p><h3>Predicting Review Scores</h3><p>With our trained embeddings, we predict and evaluate review scores for movies, enhancing our recommendations.</p><h3>🤔 Why This Approach?</h3><p>Using Wikipedia links provides a unique and rich source of data for recommendations. Unlike traditional methods, this approach captures the intricate relationships between movies, offering a fresh perspective on recommendations.</p><h3>🤝 Contributing</h3><p>We welcome contributions! Fork the repository and submit pull requests to improve the project.</p><h3>🖥️Code</h3><p><em>GitHub link: </em><a href="https://github.com/mr-sharath/Movie-Recommender-System/tree/main"><em>https://github.com/mr-sharath/Movie-Recommender-System/</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0bba1a218f14" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Real-time Document Interaction System with Python RAG Application and Generative AI]]></title>
            <link>https://medium.com/@mr.sharath.7/building-a-real-time-document-interaction-system-with-python-rag-application-and-generative-ai-62ed7be81a6c?source=rss-3795b55ce854------2</link>
            <guid isPermaLink="false">https://medium.com/p/62ed7be81a6c</guid>
            <category><![CDATA[streamlit]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[geminipro]]></category>
            <category><![CDATA[text-processing]]></category>
            <dc:creator><![CDATA[Sharath Reddy]]></dc:creator>
            <pubDate>Sun, 19 May 2024 13:37:59 GMT</pubDate>
            <atom:updated>2024-05-19T13:37:59.054Z</atom:updated>
            <content:encoded><![CDATA[<h4>Large Language Models — Gemini Pro</h4><h4>Introduction:</h4><p>During my first semester in Masters, I undertook a project to develop a real-time document interaction system. This system aimed to facilitate seamless upload and analysis of multiple PDFs, leveraging generative AI to provide intelligent responses to user inquiries. This project not only enhanced my understanding of generative AI but also showcased the potential of such technologies in streamlining document analysis.I personally used this project during my exams to easily glance at any part of my textbooks, making it an invaluable tool for quick and effective studying.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HWzD5HJ7FZD4JNtGMQTAzA.png" /></figure><h4>Project Overview:</h4><p>The real-time document interaction system is an end-to-end web application built using Streamlit. It allows users to upload and analyze more than ten PDFs simultaneously, providing the capability to inquire about diverse topics within these documents through APIs from the advanced large language model, Gemini Pro. I personally used this project during my exams to easily glance at any part of my textbooks, making it an invaluable tool for quick and effective studying.</p><h4>Technical Foundations:</h4><p>The project utilized several key technologies and methods:</p><ul><li>Streamlit: For building an interactive and user-friendly web interface.</li><li>Google Gemini Pro: A sophisticated large language model providing quick and accurate answers.</li><li>Python: The scripting language used for developing the backend and integrating various components.</li></ul><p>Building the System:</p><ol><li>Web Server Development:</li></ol><ul><li><em>Streamlit</em>: Leveraged Streamlit to create an intuitive and responsive web interface that allows users to upload multiple PDF documents effortlessly.</li></ul><p>2. PDF Analysis:</p><ul><li><em>PDF Processing</em>: Implemented functionality to extract and process text from the uploaded PDFs, ensuring efficient handling of large documents.</li></ul><p>3. Generative AI Integration:</p><ul><li><em>Google Gemini Pro API</em>: Enhanced the interaction with the Gemini Pro API to enable real-time queries and responses. The system ensured a response time of around 3 seconds, providing users with prompt and highly accurate answers.</li></ul><p>4. <em>User Interaction</em>:</p><ul><li><em>Query Handling</em>: Enabled users to inquire about various topics within the PDFs, with the system effectively coordinating with the Gemini Pro model to deliver insightful responses.</li></ul><h4>Challenges and Learnings:</h4><p>One of the primary challenges was ensuring seamless integration between the web server and the AI model, maintaining low response times and high accuracy. Through this project, I gained valuable experience in handling real-time data processing and integrating generative AI models into practical applications.</p><h4>Conclusion:</h4><p>The real-time document interaction system project demonstrated the potential of generative AI in automating and enhancing document analysis. By leveraging the capabilities of Google Gemini Pro and Streamlit, I was able to create a system that simplifies the process of extracting and querying information from multiple PDFs. This project is a testament to the practical application of advanced AI technologies in real-world scenarios, and personally, it was a game-changer in my exam preparations.</p><h4>Call to Action:</h4><p>Explore the project on GitHub to see the implementation details, test the system with your documents, and provide feedback. Your insights and contributions are highly valued as we continue to refine and expand the capabilities of AI-driven document interaction systems.</p><blockquote><a href="https://github.com/sharathkumarreddy7/chat_with_pdf">https://github.com/sharathkumarreddy7/chat_with_pdf</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=62ed7be81a6c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting Stock Prices with Time Series Forecasting and XGBoost]]></title>
            <link>https://medium.com/@mr.sharath.7/predicting-stock-prices-with-time-series-forecasting-and-xgboost-e0c33aaec8e3?source=rss-3795b55ce854------2</link>
            <guid isPermaLink="false">https://medium.com/p/e0c33aaec8e3</guid>
            <category><![CDATA[xgboost]]></category>
            <category><![CDATA[time-series-forecasting]]></category>
            <category><![CDATA[stock-market]]></category>
            <dc:creator><![CDATA[Sharath Reddy]]></dc:creator>
            <pubDate>Sun, 19 May 2024 13:30:38 GMT</pubDate>
            <atom:updated>2024-05-19T13:30:38.981Z</atom:updated>
            <content:encoded><![CDATA[<h4>Introduction:</h4><p>Predicting stock prices is a complex yet intriguing challenge in the financial market. During my exploration of time series forecasting, I embarked on a project to forecast stock prices using the powerful XGBoost algorithm. This project involved retrieving historical data, analyzing trends, and applying machine learning techniques to predict future stock prices, showcasing the potential of data-driven investment strategies.</p><h4>Technical Foundations:</h4><p>The project utilized several key technologies and methods:</p><ul><li><em>Time Series Forecasting</em>: Analyzing historical data to predict future values in a series.</li><li><em>XGBoost</em>: An optimized gradient boosting algorithm that excels in predictive modeling tasks.</li><li><em>Python and Pandas</em>: Used for data retrieval, preprocessing, and analysis.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8udOXjweDjBrLoG69y2a7w.jpeg" /></figure><p>Building the System:</p><ol><li>Data Collection:</li></ol><ul><li>Historical Data Retrieval: Collected five years of historical closing price data using Pandas data reader, ensuring a comprehensive dataset for analysis.</li></ul><p>2. Data Preprocessing:</p><ul><li>Cleaning and Formatting: Processed the data to handle missing values and ensure consistency.</li><li><strong>Feature Engineering</strong>: Created relevant features such as moving averages and lagged values to enhance the predictive power of the model.</li></ul><p>3. Model Development:</p><ul><li><strong>XGBoostRegressor</strong>: Implemented the XGBoostRegressor algorithm to build a predictive model. This involved training the model on historical data and fine-tuning hyperparameters to optimize performance.</li></ul><p>4. Prediction:</p><ul><li>Forecasting: Used the trained XGBoost model to forecast the closing prices for the next month, evaluating the model’s accuracy and reliability.</li></ul><h4>Challenges and Learnings:</h4><p>The primary challenge was dealing with the inherent volatility and unpredictability of stock prices. Fine-tuning the XGBoost model to accurately capture the underlying trends without overfitting was crucial. Through this project, I deepened my understanding of time series analysis, feature engineering, and the application of machine learning algorithms in financial forecasting.</p><h4>Conclusion:</h4><p>The stock price prediction project highlighted the potential of machine learning in financial markets. By leveraging XGBoost and historical data, I was able to develop a model capable of forecasting future stock prices with a reasonable degree of accuracy. This project not only showcased my proficiency in Python scripting and data analysis but also demonstrated the practical application of machine learning in making informed investment decisions.</p><h4>Call to Action:</h4><p>I invite you to explore the project on GitHub, delve into the code, and experiment with your datasets. Your feedback and contributions are welcome as we continue to refine predictive models for financial markets.</p><blockquote><a href="https://github.com/sharathkumarreddy7/Time-series-Forecasting">https://github.com/sharathkumarreddy7/Time-series-Forecasting</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e0c33aaec8e3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Elevating Image Quality with Deep Learning: An ESRGAN-based Approach]]></title>
            <link>https://medium.com/@mr.sharath.7/elevating-image-quality-with-deep-learning-an-esrgan-based-approach-0e66953bfd0b?source=rss-3795b55ce854------2</link>
            <guid isPermaLink="false">https://medium.com/p/0e66953bfd0b</guid>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <category><![CDATA[esrgan]]></category>
            <category><![CDATA[image-super-resolution]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[generative-adversarial]]></category>
            <dc:creator><![CDATA[Sharath Reddy]]></dc:creator>
            <pubDate>Sun, 19 May 2024 10:06:30 GMT</pubDate>
            <atom:updated>2024-05-19T14:04:10.425Z</atom:updated>
            <content:encoded><![CDATA[<h4>Neural Networks, Generative AI</h4><h4>Introduction:</h4><p>In the pursuit of enhancing image quality, I embarked on a project during my early days of exploring deep learning techniques. This project focused on developing an image super-resolution model using Enhanced Super Resolution Generative Adversarial Networks (ESRGANs). The aim was to significantly improve the resolution of low-quality images, providing a powerful tool for applications in photography, medical imaging, and more.</p><h4>Technical Foundations:</h4><p>The project leveraged several key technologies:</p><ul><li><em>ESRGAN</em>: An advanced version of GAN designed for super-resolution tasks, providing superior image quality.</li><li><em>Neural Networks</em>: Utilized deep learning techniques to train the ESRGAN model, focusing on enhancing image details and resolution.</li><li><em>Python and PyTorch</em>: Employed for implementing the neural network models and managing the data processing workflows.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SEKRXvWEZ96AcudcJbjRMw.jpeg" /></figure><h4>Building the System:</h4><ol><li>Model Development:</li></ol><ul><li>ESRGAN Architecture: Implemented an ESRGAN model, which includes a generator to upscale low-resolution images and a discriminator to differentiate between high-resolution images and upscaled images.</li><li>Training the Model: Trained the ESRGAN model on a dataset of high-resolution images, ensuring the generator learns to produce high-quality details that closely match the original high-resolution images.</li></ul><p>2. Data Processing:</p><ul><li>Input Preparation: Processed low-resolution images to feed into the ESRGAN model.</li><li>Output Generation: Generated high-resolution images from the low-resolution inputs, demonstrating significant quality improvements.</li></ul><p>3. Workflow Streamlining:</p><ul><li>Test Script: Developed a test.py script to facilitate the seamless extraction of high-resolution images. This script enables users to run the model on custom low-resolution images and obtain enhanced results efficiently.</li><li>GitHub Repository: Created a comprehensive and easily replicable repository on GitHub, providing clear instructions and necessary code for users to replicate the project and achieve similar results.</li></ul><h4>Conclusion:</h4><p>The deep learning-powered image super-resolution project was a pivotal experience in my exploration of neural networks and their applications. By leveraging ESRGANs, I was able to develop a model that delivers remarkable improvements in image quality, showcasing the potential of deep learning techniques in image enhancement tasks.</p><h4>Call to Action:</h4><p>I invite you to explore the project on GitHub, try out the test script on your low-resolution images, and experience the transformation firsthand. Your feedback and contributions are welcome as we continue to push the boundaries of image super-resolution technology.</p><blockquote><a href="https://github.com/sharathkumarreddy7/Image-Super-Resolution-Project-using-ESRGAN">https://github.com/sharathkumarreddy7/Image-Super-Resolution-Project-using-ESRGAN</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0e66953bfd0b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>