<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by bhavya sharma on Medium]]></title>
        <description><![CDATA[Stories by bhavya sharma on Medium]]></description>
        <link>https://medium.com/@18bhavyasharma?source=rss-7496da9f1e68------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*_DkQrXX_4PwdN1I3.jpg</url>
            <title>Stories by bhavya sharma on Medium</title>
            <link>https://medium.com/@18bhavyasharma?source=rss-7496da9f1e68------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 17:27:29 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@18bhavyasharma/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[I Was About to Pay More for Claude Code. Then I Found 5 Frontier Models Behind One API For Free]]></title>
            <link>https://medium.com/@18bhavyasharma/i-was-about-to-pay-more-for-claude-code-then-i-found-5-frontier-models-behind-one-api-for-free-42cfb0ea0138?source=rss-7496da9f1e68------2</link>
            <guid isPermaLink="false">https://medium.com/p/42cfb0ea0138</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[api-key]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[agent-router]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[bhavya sharma]]></dc:creator>
            <pubDate>Wed, 26 Aug 2026 17:50:12 GMT</pubDate>
            <atom:updated>2026-08-27T10:37:52.942Z</atom:updated>
            <content:encoded><![CDATA[<h3>Claude Opus 5, Claude Opus 4.8, GPT-5.6 Sol, GLM 5.3 and DeepSeek V4 Flash — and why AI coding may be changing</h3><p>Use your github account to sign up on agent router and get 125$ credit token for free.</p><p>Use this invitation link while logging in to get extra 50$ making total 175$</p><p><a href="https://agentrouter.org/register?aff=7Fzq">Try AgentRouter — my invitation link</a></p><p><a href="https://agentrouter.org/register?aff=7Fzq">https://agentrouter.org/register?aff=7Fzq</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FTDn8HyyIATJUwMNeYSpKA.png" /></figure><p><strong>You will see these five models currently available in your account</strong>:</p><ul><li><strong>Claude Opus 5</strong></li><li><strong>Claude Opus 4.8</strong></li><li><strong>GPT-5.6 Sol</strong></li><li><strong>GLM 5.3</strong></li><li><strong>DeepSeek V4 Flash</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*5x7cAYHQXf_M5lOprj1N8g.png" /></figure><h3>The Problem With Choosing One AI Model</h3><p>AI coding has changed dramatically.</p><p>A few years ago, picking an AI assistant was relatively simple.</p><p>You chose a product.</p><p>Today, the model underneath that product can matter just as much.</p><p>You might prefer one model for:</p><ul><li>large codebases</li><li>architecture</li><li>debugging</li><li>refactoring</li><li>reasoning</li><li>terminal tasks</li><li>generating tests</li><li>quick coding changes</li><li>long-context analysis</li></ul><p>And there is no guarantee that one model will be the best at everything.</p><p>That’s why model routers are becoming interesting.</p><p>Instead of building your workflow around a single provider, you put a routing layer between your application and the models.</p><p>The architecture becomes:</p><p><strong>Claude Code / Cline / Codex</strong></p><p>↓</p><p><strong>AgentRouter</strong></p><p>↓</p><p><strong>Model</strong></p><p>The model can change.</p><p>The workflow doesn’t have to.</p><p>AgentRouter’s documentation says one API key can be used across supported agents such as Claude Code, Codex, Cline and Roo Code, with usage measured under the same account.</p><h3>Use the heavyweight model when the task deserves it.</h3><h3>Use another model when it doesn’t.</h3><p>That’s potentially much more powerful than simply searching for the cheapest model.</p><h3>One API Instead of Multiple Provider Setups</h3><p>AgentRouter’s current documentation provides separate OpenAI-compatible and Anthropic-compatible endpoints.</p><p>For OpenAI-compatible models, the documented base URL is:</p><p><a href="https://co.agentrouter.org/v1">https://co.agentrouter.org/v1</a></p><p>For Anthropic-compatible Claude models:</p><p><a href="https://co.agentrouter.org/">https://co.agentrouter.org</a></p><p>The documentation specifically warns not to mix the two endpoint formats.</p><p>The practical idea is simple.</p><p>Your tool talks to AgentRouter.</p><p>AgentRouter handles the model connection.</p><p>You select the model.</p><p>That means your development environment doesn’t necessarily need to become a collection of unrelated provider configurations.</p><h3>But Is It Actually Free?</h3><p>This is where I want to be precise.</p><p>You’ve probably seen headlines saying things like:</p><blockquote><em>“Free Claude Code.”</em></blockquote><blockquote><em>“Free Opus.”</em></blockquote><blockquote><em>“$125 free credits.”</em></blockquote><blockquote><strong><em>Unlimited free frontier AI forever.</em></strong></blockquote><p>That’s not a safe assumption.</p><p>AgentRouter’s current pricing page describes the service as usage-based rather than an unlimited free service, with balances that don’t expire and pricing positioned as lower than official API pricing.</p><p>Promotional credits or signup offers can also change.</p><p>So my advice is simple:</p><p><strong>Use any current promotional credits while they’re available, but evaluate the service based on its ongoing economics rather than assuming the promotion will last forever.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z3HH76urmaYwfNNNJCxfpw.png" /></figure><h3>The Part Nobody Talks About: Model Availability</h3><p>There’s another detail worth knowing.</p><p>AgentRouter currently advertises <strong>15+ available models</strong> on its homepage.</p><p>But that does <strong>not</strong> necessarily mean every account sees the same models.</p><p>Its own documentation says:</p><blockquote><em>the models accessible to an agent depend on the resource pool associated with the API key.</em></blockquote><p>That’s important.</p><p>The five models in this article are <strong>the five I currently see</strong>, not a guarantee that every AgentRouter account will have exactly the same five.</p><p>Model availability can change.</p><p>Resource pools can change.</p><p>New models can appear.</p><p>Older models can disappear.</p><p>That’s normal in a rapidly changing AI infrastructure market.</p><h3>Should You Try It?</h3><p>If you’re already using:</p><ul><li>Claude Code</li><li>Cline</li><li>Codex</li><li>Roo Code</li><li>kilo code</li><li>another configurable AI coding agent</li></ul><p>then I think it’s worth experimenting with a model router.</p><p>Not because you should immediately abandon your current provider.</p><p>Instead, use it as an experiment.</p><p>Take the same coding task.</p><h3>Using AgentRouter with Kilo Code</h3><p>Kilo Code is a VS Code coding agent(extension). You can connect AgentRouter through a <strong>Custom Provider</strong>. Kilo Code supports OpenAI-compatible providers and Anthropic Messages.</p><p><strong>1. Install &amp; get your key</strong></p><ul><li>Install <strong>Kilo Code</strong> from the VS Code Extensions marketplace.</li><li>Get your AgentRouter API key from the AgentRouter console.</li></ul><p><strong>2. Configure Kilo Code</strong><br> Go to <strong>Kilo Code → Settings → Providers → Agentrouter</strong>.</p><p>Or</p><p>Go to <strong>Kilo Code → Settings → Providers → Connect Custom Provider</strong>.</p><p><strong>Use OpenAI Compatible</strong></p><ul><li><strong>Provider ID:</strong> agentrouter</li><li><strong>Display Name:</strong> AgentRouter</li><li><strong>API:</strong> OpenAI Compatible</li><li><strong>Base URL:</strong> <a href="https://agentrouter.org/v1">https://agentrouter.org/v1</a></li><li><strong>API Key:</strong> Your AgentRouter key</li><li><strong>Model:</strong> gpt-5.6 sol, glm-5.3, etc</li></ul><h3>Final Thought</h3><p>The AI industry is moving incredibly quickly.</p><p>A few months ago, a list containing Claude Opus, GPT, GLM and DeepSeek would have been interesting.</p><p>Now we’re reaching a point where the bigger challenge isn’t <strong>finding a powerful model</strong>.</p><p>It’s managing all of them.</p><p>That’s why I’m paying attention to AI routers.</p><p><strong>The future may not be one AI model beating every other model.</strong></p><p>It may be a world where developers have access to dozens of highly capable models and simply choose the right one for each job.</p><p>And if that happens, the most important piece of infrastructure may not be the model itself.</p><p>It may be the <strong>router connecting everything together</strong>.</p><p><a href="https://agentrouter.org/register?aff=7Fzq">Try AgentRouter — my invitation link</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=42cfb0ea0138" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What You Can Do with Gemini API and Vertex AI]]></title>
            <link>https://medium.com/@18bhavyasharma/what-you-can-do-with-gemini-api-and-vertex-ai-5a4ed7351117?source=rss-7496da9f1e68------2</link>
            <guid isPermaLink="false">https://medium.com/p/5a4ed7351117</guid>
            <category><![CDATA[gemini-api]]></category>
            <category><![CDATA[vertex-ai]]></category>
            <category><![CDATA[genaiacademy]]></category>
            <category><![CDATA[aiforviksitbharat]]></category>
            <category><![CDATA[genai-exchange]]></category>
            <dc:creator><![CDATA[bhavya sharma]]></dc:creator>
            <pubDate>Tue, 06 May 2025 20:54:34 GMT</pubDate>
            <atom:updated>2025-05-06T20:57:06.440Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HJXB6rUL72CTBKSB2scPNA.jpeg" /></figure><p>Google’s Gemini API, integrated within Vertex AI, offers a robust platform for developers and data scientists to build advanced AI applications. Leveraging function calling, developers can define custom functions that the model suggests invoking based on user prompts, facilitating seamless interactions with external APIs, databases, and services. This approach allows the model to generate structured outputs specifying function names and arguments, which developers can use to call external systems and incorporate the results back into the model’s responses. Such capabilities enable the creation of dynamic, context-aware applications across various domains, including customer support, data analysis, and workflow automation. With support for multimodal inputs — text, images, and more — and enterprise-grade tools for customization and deployment, Gemini and Vertex AI provide the necessary infrastructure to bring innovative AI solutions to life</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*aRXENZXNy2-H7aiiAgGXEA.png" /></figure><p>Recently, I completed the <strong>“Explore Generative AI with the Gemini API in Vertex AI”</strong> course offered by Google Cloud Skills Boost. which provided me practical experience in connecting to AI services, sending various types of prompts, and fine-tuning responses from the Gemini model. I also explored more advanced capabilities such as managing multi-turn interactions, caching context for efficient conversations, and working with embeddings for enhanced semantic understanding.</p><p>The training further introduced me to multimodal AI, where I learned how to generate meaningful outputs from both text and image inputs. Additionally, I acquired hands-on experience in function calling, using the Gemini 2.0 model to automatically trigger external functions based on user queries — such as retrieving product data, geocoding addresses, and extracting structured information from logs, In the final challenge lab, I applied these skills to real-world scenarios, including generating text, creating function calls, and even describing video content. This practical experience demonstrated how to ensure the reliability and production readiness of AI-driven applications using Gemini APIs.</p><h3>Understanding Gemini API and Vertex AI</h3><p><strong>Gemini API</strong> is a suite of generative AI models developed by Google DeepMind, designed for multimodal use cases. It allows developers to build applications that can understand and generate human-like text, process images, and more.</p><p><strong>Vertex AI</strong> is Google Cloud’s machine learning platform that enables the deployment and management of ML models at scale. By integrating Gemini API with Vertex AI, developers can leverage powerful AI capabilities within a robust infrastructure.</p><h3>What is Function Calling?</h3><p>Function Calling is a feature that allows the Gemini model to generate structured outputs specifying which function to call and with what arguments, based on a user’s natural language input. Instead of parsing unstructured text responses, developers receive clear instructions on function invocations, making integrations more reliable and efficient.</p><h3>Getting Started with Gemini and Vertex AI</h3><h4>1. Install the Google Gen AI SDK for Python</h4><p>Begin by installing the necessary SDK:</p><pre>%pip install --upgrade --quiet google-genai</pre><h4>2. Set Up Your Google Cloud Project</h4><p>Ensure you have a Google Cloud project with billing enabled and the Vertex AI API activated, Set your project and region.</p><pre>import os<br><br>PROJECT_ID = &quot;[your-project-id]&quot;<br>if not PROJECT_ID or PROJECT_ID == &quot;[your-project-id]&quot;:<br>    PROJECT_ID = str(os.environ.get(&quot;GOOGLE_CLOUD_PROJECT&quot;))<br><br>LOCATION = os.environ.get(&quot;GOOGLE_CLOUD_REGION&quot;, &quot;us-central1&quot;)</pre><h4>3. Initialize the Gemini Client</h4><pre>from google import genai<br><br>client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)</pre><h3>Defining Functions for Gemini</h3><p>Functions are defined using the <strong>FunctionDeclaration</strong> class, specifying the function&#39;s name, description, and parameters. These declarations guide the Gemini model in understanding which functions are available and how to use them.</p><pre>from google.genai.types import FunctionDeclaration<br><br>get_weather = FunctionDeclaration(<br>    name=&quot;get_weather&quot;,<br>    description=&quot;Retrieve weather information for a specified location and date.&quot;,<br>    parameters={<br>        &quot;type&quot;: &quot;OBJECT&quot;,<br>        &quot;properties&quot;: {<br>            &quot;location&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;City and state&quot;},<br>            &quot;date&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;Date in YYYY-MM-DD format&quot;},<br>        },<br>        &quot;required&quot;: [&quot;location&quot;, &quot;date&quot;],<br>    },<br>)</pre><h3>Implementing Function Calling: Code Examples</h3><h4>Example 1: Weather Information Retrieval</h4><p><strong>Objective</strong>: Fetch weather data for a given location and date.</p><p><strong>Function Declaration</strong>:</p><pre>get_weather = FunctionDeclaration(<br>    name=&quot;get_weather&quot;,<br>    description=&quot;Retrieve weather information for a specified location and date.&quot;,<br>    parameters={<br>        &quot;type&quot;: &quot;OBJECT&quot;,<br>        &quot;properties&quot;: {<br>            &quot;location&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;City and state&quot;},<br>            &quot;date&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;Date in YYYY-MM-DD format&quot;},<br>        },<br>        &quot;required&quot;: [&quot;location&quot;, &quot;date&quot;],<br>    },<br>)</pre><p><strong>Function Implementation</strong>:</p><pre>import requests<br><br>def get_weather(location: str, date: str) -&gt; dict:<br>    # Placeholder implementation<br>    return {<br>        &quot;location&quot;: location,<br>        &quot;date&quot;: date,<br>        &quot;temperature&quot;: &quot;22°C&quot;,<br>        &quot;condition&quot;: &quot;Sunny&quot;,<br>    }</pre><p><strong>Using the Function with Gemini</strong>:</p><pre>from google.genai.types import Tool, GenerateContentConfig, Part<br><br>weather_tool = Tool(function_declarations=[get_weather])<br><br>chat = client.chats.create(<br>    model=&quot;gemini-2.0-flash-001&quot;,<br>    config=GenerateContentConfig(<br>        temperature=0,<br>        tools=[weather_tool],<br>    ),<br>)<br><br>prompt = &quot;What&#39;s the weather like in San Francisco on 2025-05-10?&quot;<br>response = chat.send_message(prompt)<br>function_call = response.function_calls[0]<br><br># Execute the function<br>weather_info = get_weather(**function_call.args)<br><br># Send the function response back to Gemini<br>response = chat.send_message(<br>    Part.from_function_response(<br>        name=&quot;get_weather&quot;,<br>        response={&quot;content&quot;: weather_info},<br>    ),<br>)<br>print(response.text)</pre><h3>Example 2: Currency Conversion</h3><p><strong>Objective</strong>: Convert an amount from one currency to another.</p><p><strong>Function Declaration</strong>:</p><pre>convert_currency = FunctionDeclaration(<br>    name=&quot;convert_currency&quot;,<br>    description=&quot;Convert amount from one currency to another.&quot;,<br>    parameters={<br>        &quot;type&quot;: &quot;OBJECT&quot;,<br>        &quot;properties&quot;: {<br>            &quot;amount&quot;: {&quot;type&quot;: &quot;NUMBER&quot;, &quot;description&quot;: &quot;Amount to convert&quot;},<br>            &quot;from_currency&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;Source currency code&quot;},<br>            &quot;to_currency&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;Target currency code&quot;},<br>        },<br>        &quot;required&quot;: [&quot;amount&quot;, &quot;from_currency&quot;, &quot;to_currency&quot;],<br>    },<br>)</pre><p><strong>Function Implementation</strong>:</p><pre>def convert_currency(amount: float, from_currency: str, to_currency: str) -&gt; dict:<br>    # Placeholder implementation<br>    conversion_rate = 0.85  # Example rate<br>    converted_amount = amount * conversion_rate<br>    return {<br>        &quot;converted_amount&quot;: converted_amount,<br>        &quot;from_currency&quot;: from_currency,<br>        &quot;to_currency&quot;: to_currency,<br>        &quot;rate&quot;: conversion_rate,<br>    }</pre><p><strong>Using the Function with Gemini</strong>:</p><pre>currency_tool = Tool(function_declarations=[convert_currency])<br><br>chat = client.chats.create(<br>    model=&quot;gemini-2.0-flash-001&quot;,<br>    config=GenerateContentConfig(<br>        temperature=0,<br>        tools=[currency_tool],<br>    ),<br>)<br><br>prompt = &quot;Convert 100 USD to EUR.&quot;<br>response = chat.send_message(prompt)<br>function_call = response.function_calls[0]<br><br># Execute the function<br>conversion_result = convert_currency(**function_call.args)<br><br># Send the function response back to Gemini<br>response = chat.send_message(<br>    Part.from_function_response(<br>        name=&quot;convert_currency&quot;,<br>        response={&quot;content&quot;: conversion_result},<br>    ),<br>)<br>print(response.text)</pre><h3>Example 3: Scheduling Meetings</h3><p><strong>Objective</strong>: Schedule a meeting with specified attendees, date, time, and topic.<a href="https://ai.google.dev/gemini-api/docs/function-calling?utm_source=chatgpt.com">s</a></p><p><strong>Function Declaration</strong>:</p><pre>schedule_meeting = FunctionDeclaration(<br>    name=&quot;schedule_meeting&quot;,<br>    description=&quot;Schedule a meeting with attendees at a specified date and time.&quot;,<br>    parameters={<br>        &quot;type&quot;: &quot;OBJECT&quot;,<br>        &quot;properties&quot;: {<br>            &quot;attendees&quot;: {<br>                &quot;type&quot;: &quot;ARRAY&quot;,<br>                &quot;items&quot;: {&quot;type&quot;: &quot;STRING&quot;},<br>                &quot;description&quot;: &quot;List of attendees&quot;,<br>            },<br>            &quot;date&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;Date in YYYY-MM-DD format&quot;},<br>            &quot;time&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;Time in HH:MM format&quot;},<br>            &quot;topic&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;Meeting topic&quot;},<br>        },<br>        &quot;required&quot;: [&quot;attendees&quot;, &quot;date&quot;, &quot;time&quot;, &quot;topic&quot;],<br>    },<br>)</pre><p><strong>Function Implementation</strong>:</p><pre>def schedule_meeting(attendees: list, date: str, time: str, topic: str) -&gt; dict:<br>    # Placeholder implementation<br>    return {<br>        &quot;attendees&quot;: attendees,<br>        &quot;date&quot;: date,<br>        &quot;time&quot;: time,<br>        &quot;topic&quot;: topic,<br>        &quot;status&quot;: &quot;Scheduled&quot;,<br>    }</pre><p><strong>Using the Function with Gemini</strong>:</p><pre>meeting_tool = Tool(function_declarations=[schedule_meeting])<br><br>chat = client.chats.create(<br>    model=&quot;gemini-2.0-flash-001&quot;,<br>    config=GenerateContentConfig(<br>        temperature=0,<br>        tools=[meeting_tool],<br>    ),<br>)<br><br>prompt = &quot;Schedule a meeting with Alice and Bob on 2025-05-15 at 14:00 about project updates.&quot;<br>response = chat.send_message(prompt)<br>function_call = response.function_calls[0]<br><br># Execute the function<br>meeting_details = schedule_meeting(**function_call.args)<br><br># Send the function response back to Gemini<br>response = chat.send_message(<br>    Part.from_function_response(<br>        name=&quot;schedule_meeting&quot;,<br>        response={&quot;content&quot;: meeting_details},<br>    ),<br>)<br>print(response.text)</pre><h3>Advanced Features of Gemini API and Vertex AI</h3><p>Building upon the foundational understanding of function calling with the Gemini API and Vertex AI, let’s delve into the advanced capabilities that empower developers to create more sophisticated and responsive applications.</p><h4>1. Compositional Function Calling</h4><p>Compositional function calling allows the Gemini model to chain multiple function calls together to fulfill complex user requests. For instance, if a user asks, “What’s the weather like at my current location?”, the model can:</p><ul><li>Invoke a <strong>get_current_location()</strong> function to determine the user&#39;s location.</li><li>Use the result to call a <strong>get_weather(location)</strong> function to fetch the weather information.</li></ul><p>This sequential execution enables the model to handle multi-step tasks seamlessly.</p><h4>2. Parallel Function Calling</h4><p>In scenarios where multiple independent pieces of information are required, the Gemini model can perform parallel function calls. For example, when a user inquires, “Do you have the Pixel 9 in stock, and where’s the nearest store?”, the model can simultaneously:</p><ul><li>Call <strong>get_product_info(“Pixel 9”)</strong> to check stock availability.</li><li>Call <strong>get_store_location(“user’s location”)</strong> to find the nearest store.</li></ul><p>This parallel processing reduces latency and provides quicker responses to the user.</p><h4>3. Multimodal Function Calling</h4><p>Gemini’s multimodal capabilities allow it to process and understand various input types, including text, images, audio, and video. For instance, a user can upload a photo of a product, and the model can:</p><ul><li>Analyze the image to identify the product.</li><li>Invoke <strong>get_product_info(product_name)</strong> to retrieve details about the product.</li></ul><p>This feature enhances user interaction by allowing more natural and diverse inputs.</p><h4>4. Integration with External APIs</h4><p>Function calling enables seamless integration with external APIs, allowing the Gemini model to fetch real-time data. For example, to provide weather updates:</p><pre>from google import genai<br>from google.genai.types import FunctionDeclaration, GenerateContentConfig, Part, Tool<br><br># Define the function declaration<br>get_weather = FunctionDeclaration(<br>    name=&quot;get_weather&quot;,<br>    description=&quot;Fetches weather information for a given city and date.&quot;,<br>    parameters={<br>        &quot;type&quot;: &quot;OBJECT&quot;,<br>        &quot;properties&quot;: {<br>            &quot;city&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;Name of the city&quot;},<br>            &quot;date&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;Date in YYYY-MM-DD format&quot;},<br>        },<br>        &quot;required&quot;: [&quot;city&quot;, &quot;date&quot;],<br>    },<br>)<br><br># Initialize the tool with the function declaration<br>weather_tool = Tool(function_declarations=[get_weather])<br><br># Configure the model<br>config = GenerateContentConfig(temperature=0, tools=[weather_tool])<br><br># Initialize the client<br>client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)<br><br># Send a prompt to the model<br>prompt = &quot;What&#39;s the weather in Jaipur on 2025-05-10?&quot;<br>response = client.models.generate_content(<br>    model=&quot;gemini-2.0-flash-001&quot;,<br>    contents=prompt,<br>    config=config,<br>)<br><br># Extract the function call from the response<br>function_call = response.candidates[0].content.parts[0].function_call<br>print(f&quot;Function to call: {function_call.name}&quot;)<br>print(f&quot;Arguments: {function_call.args}&quot;)</pre><p>This code snippet demonstrates how the model identifies the need to call the <strong>get_weather</strong> function with the appropriate arguments extracted from the user&#39;s prompt.</p><h3>Integrating Gemini API with Vertex AI</h3><h4>Prerequisites</h4><ul><li>Google Cloud Project with Vertex AI API enabled.</li><li>Python environment with the <strong>google-genai</strong> SDK installed.</li></ul><h4>Installation</h4><pre>pip install --upgrade google-genai</pre><h4>Authentication</h4><pre>from google.colab import auth<br>auth.authenticate_user()</pre><h4>Initialize Client</h4><pre>from google import genai<br>PROJECT_ID = &quot;[your-project-id]&quot;<br>LOCATION = &quot;[your-region]&quot;<br>client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)</pre><h4>Choose Model</h4><pre>MODEL_ID = &quot;gemini-2.0-flash-001&quot;</pre><h4>Send Text Prompt</h4><pre>response = client.models.generate_content(<br>    model=MODEL_ID, contents=&quot;What&#39;s the largest planet in our solar system?&quot;<br>)<br>print(response.text)</pre><h4>Send Multimodal Prompt</h4><pre>from PIL import Image<br>import requests<br><br>image = Image.open(<br>    requests.get(<br>        &quot;https://storage.googleapis.com/cloud-samples-data/generative-ai/image/meal.png&quot;,<br>        stream=True,<br>    ).raw<br>)<br><br>response = client.models.generate_content(<br>    model=MODEL_ID,<br>    contents=[image, &quot;Write a short and engaging blog post based on this picture.&quot;]<br>)<br>print(response.text)</pre><h3>Advanced Features</h3><h4>Function Calling</h4><p>Define external functions that Gemini can invoke during its execution.</p><pre>from google.genai.types import FunctionDeclaration, Tool<br><br>def get_destination(destination: str):<br>    return f&quot;User wants to travel to {destination}&quot;<br><br>destination_tool = Tool(<br>    function_declarations=[<br>        FunctionDeclaration(<br>            name=&quot;get_destination&quot;,<br>            description=&quot;Get the destination that the user wants to go to&quot;,<br>            parameters={&quot;type&quot;: &quot;OBJECT&quot;, &quot;properties&quot;: {&quot;destination&quot;: {&quot;type&quot;: &quot;STRING&quot;}}},<br>        )<br>    ]<br>)<br><br>response = client.models.generate_content(<br>    model=MODEL_ID,<br>    contents=&quot;I&#39;d like to travel to Paris.&quot;,<br>    config=GenerateContentConfig(tools=[destination_tool])<br>)<br>print(response.candidates[0].content.parts[0].function_call)</pre><h4>Context Caching</h4><p>Store frequently used input tokens in a dedicated cache to improve efficiency.</p><pre>from google.genai.types import CreateCachedContentConfig<br><br>system_instruction = &quot;&quot;&quot;<br>  You are an expert researcher who has years of experience in conducting systematic literature surveys and meta-analyses of different topics.<br>  You pride yourself on incredible accuracy and attention to detail. You always stick to the facts in the sources provided, and never make up new facts.<br>  Now look at the research paper below, and answer the following questions in 1-2 sentences.<br>&quot;&quot;&quot;<br><br>pdf_parts = [<br>    Part.from_uri(<br>        file_uri=&quot;gs://cloud-samples-data/generative-ai/pdf/2312.11805v3.pdf&quot;,<br>        mime_type=&quot;application/pdf&quot;,<br>    ),<br>    Part.from_uri(<br>        file_uri=&quot;gs://cloud-samples-data/generative-ai/pdf/2403.05530.pdf&quot;,<br>        mime_type=&quot;application/pdf&quot;,<br>    ),<br>]<br><br>cached_content = client.caches.create(<br>    model=&quot;gemini-2.0-flash-001&quot;,<br>    config=CreateCachedContentConfig(<br>        system_instruction=system_instruction,<br>        contents=pdf_parts,<br>        ttl=&quot;3600s&quot;,<br>    ),<br>)<br><br>response = client.models.generate_content(<br>    model=&quot;gemini-2.0-flash-001&quot;,<br>    contents=&quot;What is the research goal shared by these research papers?&quot;,<br>    config=GenerateContentConfig(cached_content=cached_content.name),<br>)<br>print(response.text)</pre><h3>Batch Prediction</h3><p>Process large datasets by sending a large number of requests in a single batch.</p><pre>from google.genai.types import CreateBatchJobConfig<br><br>INPUT_DATA = &quot;gs://cloud-samples-data/generative-ai/batch/batch_requests_for_multimodal_input_2.jsonl&quot;<br>BUCKET_URI = &quot;[your-cloud-storage-bucket]&quot;<br><br>batch_job = client.batches.create(<br>    model=MODEL_ID,<br>    src=INPUT_DATA,<br>    config=CreateBatchJobConfig(dest=BUCKET_URI),<br>)<br><br>batch_job.name</pre><h3>Use Cases of Gemini API and Vertex AI</h3><h4>1. Content Generation</h4><p>Automatically generate articles, blog posts, or marketing content based on input prompts or existing materials.</p><h4>2. Chatbots and Virtual Assistants</h4><p>Build conversational agents that can understand and respond to user queries across multiple modalities.</p><h4>3. Document Analysis</h4><p>Analyze and extract information from documents, including PDFs, images, and videos.</p><h4>4. Code Generation and Debugging</h4><p>Generate code snippets or debug existing code based on natural language descriptions.</p><h4>5. Data Analysis and Visualization</h4><p>Analyze datasets and generate visualizations or summaries to aid in decision-making.</p><h3>Conclusion</h3><p>Google’s Gemini API, integrated with Vertex AI, empowers developers to build intelligent, multimodal, and production-ready applications. Through features like <strong>function calling</strong>, developers can seamlessly connect the model to external systems, enabling structured and context-aware interactions. The platform supports <strong>compositional</strong>, <strong>parallel</strong>, and <strong>multimodal function calling</strong>, significantly enhancing responsiveness and flexibility. With practical tools for fine-tuning, multi-turn conversation management, and real-world deployment, Gemini and Vertex AI offer a comprehensive environment for creating cutting-edge generative AI solutions. Whether it’s for customer support, data processing, or workflow automation, these tools provide the infrastructure and intelligence to deliver impactful results at scale.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5a4ed7351117" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Developing AI-Powered Applications Using Gemini and Streamlit]]></title>
            <link>https://medium.com/@18bhavyasharma/developing-ai-powered-applications-using-gemini-and-streamlit-c163ba07bc63?source=rss-7496da9f1e68------2</link>
            <guid isPermaLink="false">https://medium.com/p/c163ba07bc63</guid>
            <category><![CDATA[streamlit]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[genaiacademy]]></category>
            <category><![CDATA[genai-exchange]]></category>
            <category><![CDATA[aiforviksitbharat]]></category>
            <dc:creator><![CDATA[bhavya sharma]]></dc:creator>
            <pubDate>Mon, 05 May 2025 21:03:56 GMT</pubDate>
            <atom:updated>2025-05-05T21:03:56.968Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/507/1*caj3GCqMfCiKcz7MWUiXPg.png" /></figure><p>Google’s Gemini API, integrated with Streamlit, offers developers a powerful toolkit to build intelligent, interactive applications. This comprehensive guide walks you through the process of developing AI-powered applications using Gemini’s function calling capabilities and deploying them with Streamlit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*xt9rUkHe0GSWnqw80qMj7w.png" /></figure><p>Recently, I completed the <strong>“Develop GenAI Apps with Gemini and Streamlit”</strong> course offered by Google Cloud Skills Boost. I have developed a comprehensive understanding of integrating generative AI models into cloud-based applications, In the initial lab, I learned to interact with the Gemini 2.0 Flash model using the Vertex AI SDK for Python. This involved generating text from various inputs, including text prompts, images, and videos. I explored features like streaming responses, adjusting safety filters, and handling multi-turn conversations, which are crucial for building responsive and secure AI applications, The subsequent lab focused on deploying a Streamlit application integrated with the Gemini API on Cloud Run. I gained hands-on experience in setting up a Python virtual environment, configuring environment variables, and using Docker to containerize the application. Deploying the application to Cloud Run demonstrated how to build scalable and interactive AI-powered web applications, These labs have equipped me with practical skills in leveraging Google’s generative AI capabilities, enabling me to develop and deploy sophisticated AI applications that can process and generate content across multiple modalities.</p><h3>Introduction to Gemini and Streamlit</h3><p><strong>Gemini</strong> is a family of generative AI models developed by Google, designed for multimodal use cases. It supports function calling, allowing developers to define functions that the model can invoke based on user inputs.</p><p><strong>Streamlit</strong> is an open-source Python library that enables the creation of interactive web applications for data science and machine learning projects with minimal effort.</p><p>By combining Gemini’s AI capabilities with Streamlit’s user-friendly interface, developers can build robust AI-powered applications.</p><h3>1. Setting Up the Environment</h3><h4>Install Required Libraries</h4><p>Ensure you have the necessary libraries installed:</p><pre>!pip install --quiet google-generativeai streamlit requests</pre><h4>Set Up Google Cloud Project</h4><p>Before using Vertex AI, set up your Google Cloud project and enable the Vertex AI API.</p><pre>import os<br>from google import generativeai as genai<br><br># Set your Google Cloud project ID<br>PROJECT_ID = &quot;your-project-id&quot;<br>LOCATION = &quot;us-central1&quot;<br><br># Initialize the Gemini client<br>client = genai.Client(project=PROJECT_ID, location=LOCATION)</pre><h3>2. Creating a Basic Chatbot Interface</h3><p>With your environment set up, you can now create a simple chatbot interface using Streamlit. This interface will allow users to interact with the Gemini model through a web-based chat.</p><pre>import streamlit as st<br><br>st.title(&quot;Gemini Chatbot&quot;)<br><br>user_input = st.text_input(&quot;You: &quot;, &quot;&quot;)<br><br>if user_input:<br>    response = client.models.generate_content(<br>        model=&quot;gemini-2.0-flash-001&quot;,<br>        contents=user_input<br>    )<br>    st.write(&quot;Gemini: &quot;, response.text)</pre><p>This code snippet creates a basic chat interface where user inputs are sent to the Gemini model, and the responses are displayed on the web page.</p><h3>3. Implementing Function Calling with Gemini</h3><p>To enhance the capabilities of your application, you can implement function calling with Gemini. This allows the model to execute predefined functions based on user inputs, enabling dynamic interactions.</p><h4>Defining Functions</h4><p>First, define the functions that Gemini can call.</p><pre>from google.genai.types import FunctionDeclaration<br><br>def get_product_info(product_name: str):<br>    # Function to fetch product information<br>    pass<br><br>def get_store_location(location: str):<br>    # Function to fetch store location<br>    pass<br><br>def place_order(product: str, address: str):<br>    # Function to place an order<br>    pass<br><br>function_declarations = [<br>    FunctionDeclaration(<br>        name=&quot;get_product_info&quot;,<br>        description=&quot;Fetch product information&quot;,<br>        parameters={&quot;type&quot;: &quot;OBJECT&quot;, &quot;properties&quot;: {&quot;product_name&quot;: {&quot;type&quot;: &quot;STRING&quot;}}}<br>    ),<br>    FunctionDeclaration(<br>        name=&quot;get_store_location&quot;,<br>        description=&quot;Fetch store location&quot;,<br>        parameters={&quot;type&quot;: &quot;OBJECT&quot;, &quot;properties&quot;: {&quot;location&quot;: {&quot;type&quot;: &quot;STRING&quot;}}}<br>    ),<br>    FunctionDeclaration(<br>        name=&quot;place_order&quot;,<br>        description=&quot;Place an order&quot;,<br>        parameters={&quot;type&quot;: &quot;OBJECT&quot;, &quot;properties&quot;: {&quot;product&quot;: {&quot;type&quot;: &quot;STRING&quot;}, &quot;address&quot;: {&quot;type&quot;: &quot;STRING&quot;}}}<br>    )<br>]</pre><h3>4. Integrating Functions with Streamlit</h3><p>Next, integrate these functions into your Streamlit application.</p><pre>import streamlit as st<br>from google.genai.types import GenerateContentConfig, Part, Tool<br><br>tool = Tool(function_declarations=function_declarations)<br><br>chat = client.chats.create(<br>    model=&quot;gemini-2.0-flash-001&quot;,<br>    config=GenerateContentConfig(temperature=0, tools=[tool])<br>)<br><br>st.title(&quot;Gemini Retail Assistant&quot;)<br><br>user_input = st.text_input(&quot;Ask me anything about products or stores:&quot;)<br><br>if user_input:<br>    response = chat.send_message(user_input)<br>    for function_call in response.function_calls:<br>        if function_call.name == &quot;get_product_info&quot;:<br>            product_info = get_product_info(function_call.arguments[&quot;product_name&quot;])<br>            st.write(&quot;Product Info: &quot;, product_info)<br>        elif function_call.name == &quot;get_store_location&quot;:<br>            store_location = get_store_location(function_call.arguments[&quot;location&quot;])<br>            st.write(&quot;Store Location: &quot;, store_location)<br>        elif function_call.name == &quot;place_order&quot;:<br>            order_status = place_order(function_call.arguments[&quot;product&quot;], function_call.arguments[&quot;address&quot;])<br>            st.write(&quot;Order Status: &quot;, order_status)</pre><p>In this setup, the chatbot identifies the appropriate function to call based on the user’s input and displays the corresponding information.</p><h3>5. Deploying Your Application</h3><p>Once your application is ready, you can deploy it using Google Cloud Run. This service allows you to run your application in a fully managed environment.</p><h4>Dockerizing the Application</h4><p>Create a <strong>dockerfile</strong> to containerize your application.</p><pre>FROM python:3.9-slim<br><br>WORKDIR /app<br><br>COPY . /app<br><br>RUN pip install --no-cache-dir -r requirements.txt<br><br>CMD [&quot;streamlit&quot;, &quot;run&quot;, &quot;app.py&quot;]</pre><h3>Building and Deploying</h3><p>Build and deploy your application to Cloud Run.</p><pre>gcloud builds submit --tag gcr.io/your-project-id/gemini-streamlit-app<br>gcloud run deploy gemini-streamlit-app --image gcr.io/your-project-id/gemini-streamlit-app --platform managed --region us-central1 --allow-unauthenticated</pre><p>After deployment, your application will be accessible via a public URL provided by Cloud Run.</p><h3>Here is an example on How you can Build a Production-Ready Streamlit App Using Gemini 2.0 Flash for AI Interactions</h3><h4>Environment Setup</h4><p><strong>Install Prerequisites</strong></p><p>Create a Python virtual environment and install required packages using bash:</p><pre>python3 -m venv genai-streamlit<br>source genai-streamlit/bin/activate<br>pip install --upgrade google-genai streamlit</pre><h4>Initialize Google GenAI SDK</h4><p>Set your Google Cloud project and region, then initialize the GenAI client:</p><pre>import os<br>from google import genai<br><br>PROJECT_ID = os.getenv(&quot;GOOGLE_CLOUD_PROJECT&quot;, &quot;your-project-id&quot;)<br>LOCATION   = os.getenv(&quot;GOOGLE_CLOUD_REGION&quot;, &quot;us-central1&quot;)<br><br>client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)</pre><h4>Building a Basic Chat Interface</h4><p>Let’s start with a simple Streamlit app that sends user questions to Gemini and displays responses.</p><p><strong>app.py</strong></p><pre>import streamlit as st<br><br>st.title(&quot;EcoTips Chatbot&quot;)<br><br>user_input = st.text_input(&quot;Ask me for eco-friendly tips:&quot;)<br><br>if user_input:<br>    response = client.models.generate_content(<br>        model=&quot;gemini-2.0-flash-001&quot;,<br>        contents=user_input<br>    )<br>    st.markdown(f&quot;**Gemini:** {response.text}&quot;)</pre><p>Run locally with:</p><pre>streamlit run app.py</pre><ul><li>We create a text input box for the user.</li><li>On submission, we call Gemini’s <strong>generate_content</strong> method.</li><li>The AI’s reply is rendered via Markdown.</li></ul><h3>Implementing Function Calling</h3><p>Function calling enables structured interactions: the model suggests which function to invoke and with what arguments. We’ll build an <strong>event recommendation assistant</strong>.</p><h4>1. Declare Functions</h4><pre>from google.genai.types import FunctionDeclaration<br><br>def fetch_events(city: str, date: str):<br>    # In production, call an events API.<br>    return [{&quot;name&quot;:&quot;Green Market&quot;,&quot;date&quot;:date,&quot;city&quot;:city}]<br><br># Describe available tools for Gemini<br>event_tool = FunctionDeclaration(<br>    name=&quot;fetch_events&quot;,<br>    description=&quot;Retrieve eco-friendly events by city and date&quot;,<br>    parameters={<br>        &quot;type&quot;:&quot;OBJECT&quot;,<br>        &quot;properties&quot;:{<br>            &quot;city&quot;: {&quot;type&quot;:&quot;STRING&quot;,&quot;description&quot;:&quot;City name&quot;},<br>            &quot;date&quot;: {&quot;type&quot;:&quot;STRING&quot;,&quot;description&quot;:&quot;Event date e.g. 2025-06-01&quot;}<br>        },<br>        &quot;required&quot;:[&quot;city&quot;,&quot;date&quot;]<br>    }<br>)</pre><h4>2. Integrate in Streamlit</h4><pre>from google.genai.types import Tool, GenerateContentConfig, Part<br><br># Wrap in a Tool<br>tool = Tool(function_declarations=[event_tool])<br><br># Start a chat session with function-calling enabled<br>chat = client.chats.create(<br>    model=&quot;gemini-2.0-flash-001&quot;,<br>    config=GenerateContentConfig(temperature=0, tools=[tool])<br>)<br><br>st.title(&quot;Eco Event Finder&quot;)<br>query = st.text_input(&quot;Find events (e.g. &#39;Events in Boston on 2025-06-15&#39;):&quot;)<br><br>if query:<br>    response = chat.send_message(query)<br>    # The model suggests the function call<br>    call = response.function_calls[0]<br>    args = call.arguments<br>    events = fetch_events(args[&quot;city&quot;], args[&quot;date&quot;])<br>    # Provide function output back to Gemini for natural summary<br>    summary = chat.send_message(<br>        Part.from_function_response(name=&quot;fetch_events&quot;, response={&quot;content&quot;: events})<br>    )<br>    st.write(summary.text)</pre><h3>Multimodal Text Generation</h3><p>Gemini Flash can process images alongside text. Let’s ask it to describe a nature photo.</p><h4>Helper Functions</h4><pre>from google.genai.types import Image, Part<br>import requests<br>from PIL import Image as PILImage<br><br>def load_image_from_url(/service/url: str) -&gt; Image:<br>    data = requests.get(url).content<br>    return Image.from_bytes(data)</pre><h4>Streamlit Integration</h4><pre>st.header(&quot;Image Description&quot;)<br><br>image_url = st.text_input(&quot;Enter an image URL:&quot;)<br>if image_url:<br>    img = load_image_from_url(https://medium.com/image_url)<br>    prompt = &quot;Describe the main features of this scene.&quot;<br>    contents = [img, prompt]<br>    <br>    response = client.models.generate_content(<br>        model=&quot;gemini-2.0-flash-001&quot;,<br>        contents=contents<br>    )<br>    st.image(PILImage.open(requests.get(image_url, stream=True).raw), caption=&quot;Input Image&quot;)<br>    st.write(response.text)</pre><h3>Here is the complete app.py code</h3><pre>import os<br>import requests<br>import streamlit as st<br>from google import genai<br>from google.genai.types import FunctionDeclaration, Tool, GenerateContentConfig, Part<br>from PIL import Image as PILImage<br><br># Set up Google Cloud project and region<br>PROJECT_ID = os.getenv(&quot;GOOGLE_CLOUD_PROJECT&quot;, &quot;your-project-id&quot;)<br>LOCATION = os.getenv(&quot;GOOGLE_CLOUD_REGION&quot;, &quot;us-central1&quot;)<br><br># Initialize the GenAI client<br>client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)<br><br># Declare function for event fetching<br>def fetch_events(city: str, date: str):<br>    # In production, you would call an events API.<br>    return [{&quot;name&quot;: &quot;Green Market&quot;, &quot;date&quot;: date, &quot;city&quot;: city}]<br><br># Define the event tool function declaration<br>event_tool = FunctionDeclaration(<br>    name=&quot;fetch_events&quot;,<br>    description=&quot;Retrieve eco-friendly events by city and date&quot;,<br>    parameters={<br>        &quot;type&quot;: &quot;OBJECT&quot;,<br>        &quot;properties&quot;: {<br>            &quot;city&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;City name&quot;},<br>            &quot;date&quot;: {&quot;type&quot;: &quot;STRING&quot;, &quot;description&quot;: &quot;Event date e.g. 2025-06-01&quot;}<br>        },<br>        &quot;required&quot;: [&quot;city&quot;, &quot;date&quot;]<br>    }<br>)<br><br># Wrap in a Tool<br>tool = Tool(function_declarations=[event_tool])<br><br># Start a chat session with function-calling enabled<br>chat = client.chats.create(<br>    model=&quot;gemini-2.0-flash-001&quot;,<br>    config=GenerateContentConfig(temperature=0, tools=[tool])<br>)<br><br># Streamlit UI for EcoTips Chatbot<br>st.title(&quot;EcoTips Chatbot&quot;)<br><br>user_input = st.text_input(&quot;Ask me for eco-friendly tips:&quot;)<br><br>if user_input:<br>    response = client.models.generate_content(<br>        model=&quot;gemini-2.0-flash-001&quot;,<br>        contents=user_input<br>    )<br>    st.markdown(f&quot;**Gemini:** {response.text}&quot;)<br><br># Streamlit UI for Eco Event Finder<br>st.title(&quot;Eco Event Finder&quot;)<br>query = st.text_input(&quot;Find events (e.g. &#39;Events in Boston on 2025-06-15&#39;):&quot;)<br><br>if query:<br>    response = chat.send_message(query)<br>    # The model suggests the function call<br>    call = response.function_calls[0]<br>    args = call.arguments<br>    events = fetch_events(args[&quot;city&quot;], args[&quot;date&quot;])<br>    # Provide function output back to Gemini for natural summary<br>    summary = chat.send_message(<br>        Part.from_function_response(name=&quot;fetch_events&quot;, response={&quot;content&quot;: events})<br>    )<br>    st.write(summary.text)<br><br># Multimodal Text Generation - Image Description<br>st.header(&quot;Image Description&quot;)<br><br>def load_image_from_url(/service/url: str) -&gt; Part:<br>    data = requests.get(url).content<br>    return Part.from_bytes(data, mime_type=&quot;image/jpeg&quot;)<br><br>image_url = st.text_input(&quot;Enter an image URL:&quot;)<br>if image_url:<br>    img = load_image_from_url(https://medium.com/image_url)<br>    prompt = &quot;Describe the main features of this scene.&quot;<br>    contents = [img, prompt]<br>    <br>    response = client.models.generate_content(<br>        model=&quot;gemini-2.0-flash-001&quot;,<br>        contents=contents<br>    )<br>    st.image(PILImage.open(requests.get(image_url, stream=True).raw), caption=&quot;Input Image&quot;)<br>    st.write(response.text)<br><br></pre><h3>Deploying to Cloud Run</h3><p>Finally, we’ll containerize and deploy.</p><h4>Dockerfile</h4><pre>FROM python:3.9-slim<br>WORKDIR /app<br>COPY . /app<br>RUN pip install --no-cache-dir --upgrade google-genai streamlit<br>CMD [&quot;streamlit&quot;,&quot;run&quot;,&quot;app.py&quot;,&quot;--server.port=8080&quot;,&quot;--server.enableCORS=false&quot;]</pre><h4>Build &amp; Deploy</h4><pre># Build and push<br>gcloud builds submit --tag gcr.io/$PROJECT_ID/ecotips-app<br><br># Deploy<br>gcloud run deploy ecotips-app \<br>  --image gcr.io/$PROJECT_ID/ecotips-app \<br>  --platform managed \<br>  --region $LOCATION \<br>  --allow-unauthenticated \<br>  --set-env-vars=GOOGLE_CLOUD_PROJECT=$PROJECT_ID,GOOGLE_CLOUD_REGION=$LOCATION</pre><p>After deployment, visit the provided service URL to interact with your EcoTips Chatbot at scale.</p><h3>Conclusion</h3><p>Through my recent completion of the “Develop GenAI Apps with Gemini and Streamlit” course on Google Cloud Skills Boost, I gained hands-on experience in leveraging Google’s generative AI models, specifically Gemini 2.0 Flash, to create intelligent applications that interact with users through dynamic content generation and function calling. integrating Google’s Gemini API with Streamlit provides a robust framework for building interactive, AI-powered applications. Through hands-on labs, I gained practical experience in using Gemini’s function calling capabilities, enabling dynamic interactions and responses based on user inputs. The combination of multimodal text generation and function invocation allows developers to create sophisticated applications that process and generate content across multiple formats, such as text, images, and events. The process of deploying these applications on Google Cloud using Cloud Run and Docker showcases the scalability and ease of managing AI-powered services.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c163ba07bc63" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Extract Value from Rich Documents Using gemini Multimodality and multimodal RAG]]></title>
            <link>https://medium.com/@18bhavyasharma/how-to-extract-value-from-rich-documents-using-gemini-multimodality-and-multimodal-rag-b135002c3c8b?source=rss-7496da9f1e68------2</link>
            <guid isPermaLink="false">https://medium.com/p/b135002c3c8b</guid>
            <category><![CDATA[genaiacademy]]></category>
            <category><![CDATA[google-cloud]]></category>
            <category><![CDATA[multimodal-ai]]></category>
            <category><![CDATA[genai-exchange]]></category>
            <category><![CDATA[aiforviksitbharat]]></category>
            <dc:creator><![CDATA[bhavya sharma]]></dc:creator>
            <pubDate>Sun, 04 May 2025 19:23:50 GMT</pubDate>
            <atom:updated>2025-05-17T12:57:20.745Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h28TYV4aneLPn9D9KfawZg.jpeg" /></figure><p>In today’s data-driven world, businesses and professionals frequently deal with large volumes of unstructured data, such as PDFs, scanned images, or multimedia content. The ability to extract meaningful insights from these complex documents is crucial. Google’s Gemini 2.0, a cutting-edge multimodal AI model designed to process and understand text, images, audio, and video in an integrated way. This article explores how Gemini 2.0 can be used to extract valuable information from rich documents and improve retrieval-augmented generation (RAG) systems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*w5gwETYkz11ObeT6U5g80g.png" /></figure><p>Recently, I completed the “Inspect Rich Documents with Gemini Multimodality and Multimodal RAG” course by google cloud skill boost, I explored how to leverage Google Cloud’s Gemini models to extract and analyze information from complex documents containing both text and images. By utilizing multimodal prompts, I learned to generate comprehensive descriptions of visual content and retrieve supplementary information beyond what’s explicitly presented. The lab also introduced me to building metadata for documents with mixed content and employing Multimodal Retrieval-Augmented Generation (RAG) techniques to fetch relevant text segments and provide citations. This hands-on experience enhanced my understanding of integrating multimodal AI capabilities into document analysis workflows, enabling more enriched and context-aware information retrieval.</p><h3>Understanding Gemini 2.0</h3><p>Gemini 2.0 is a family of generative AI models developed by Google DeepMind, built to be natively multimodal. Unlike its predecessors, Gemini 2.0 can process and generate content across various modalities, including text, images, audio, and video. This capability makes it particularly suited for applications requiring a deep understanding of diverse data types.</p><h3>Key Features of Gemini 2.0</h3><ul><li><strong>Multimodal Input and Output</strong>: Supports a combination of text, images, audio, and video inputs and can generate responses in multiple formats.</li><li><strong>Advanced Reasoning</strong>: Capable of complex reasoning across different modalities, making it ideal for tasks like document analysis and RAG.</li><li><strong>Real-Time Interaction</strong>: With the Gemini 2.0 Flash model, users can engage in real-time conversations, leveraging the model’s low-latency capabilities.</li></ul><h3>Understanding Multimodal RAG (Retrieval-Augmented Generation)</h3><h4>What is RAG?</h4><p><strong>Retrieval-Augmented Generation (RAG)</strong> refers to the process of enhancing model responses by first retrieving relevant information from external sources, such as <strong>documents</strong>, <strong>databases</strong>, or <strong>websites</strong>, and then generating an output based on that retrieved knowledge. In multimodal RAG, the system can retrieve both <strong>textual and visual information</strong> before generating its response.</p><p>For example, if a user asks about a product’s suitability, the system can retrieve images and descriptions of that product and its alternatives, then use the multimodal model to generate a recommendation.</p><h4>RAG Process:</h4><ol><li><strong>Retrieve</strong> relevant data (e.g., text or images) from a knowledge base or external source.</li><li><strong>Combine</strong> the retrieved data with a <strong>prompt</strong> (text or image) to generate a response.</li><li><strong>Generate</strong> an output that is augmented with the retrieved data, providing a more informed and accurate response.</li></ol><h3>Extracting Value from Rich Documents</h3><h4>1. Document Ingestion</h4><p>The first step involves ingesting various document formats into a system that Gemini 2.0 can process. This may include PDFs, scanned images, or multimedia files. Tools like Optical Character Recognition (OCR) can be employed to convert non-text content into machine-readable formats.</p><h4>2. Preprocessing</h4><p>Once the documents are ingested, preprocessing steps are necessary to structure the data. This may involve segmenting documents into sections, extracting metadata, and identifying key entities. For instance, in a legal document, preprocessing might highlight clauses, definitions, and references.</p><h4>3. Semantic Understanding with Gemini 2.0</h4><p>With the data preprocessed, Gemini 2.0 can be utilized to gain a deeper understanding of the content. By feeding both text and images (or other modalities) into the model, it can generate comprehensive summaries, answer questions, and even provide explanations for its reasoning.</p><p><strong>Example</strong>:</p><p>Given a scanned image of a handwritten note, Gemini 2.0 can transcribe the text, interpret the context, and generate a summary or action items.</p><h4>4. Integration into RAG Systems</h4><p>For applications like chatbots or virtual assistants, integrating Gemini 2.0’s capabilities into a RAG system can significantly enhance performance. By combining real-time retrieval of relevant documents with Gemini’s generative capabilities, systems can provide more accurate and context-aware responses.</p><h3>Environment Setup</h3><p>Install the Vertex AI and genai SDKs. In a Jupyter notebook or Colab cell run</p><pre>pip install --upgrade google-cloud-aiplatform google-genai<br>!pip install --upgrade google-genai</pre><p>Restart the kernel if required.</p><p>After installing the SDK, you must restart the runtime to apply the changes.</p><pre>import IPython<br><br>app = IPython.Application.instance()<br>app.kernel.do_shutdown(True)</pre><h3>Authentication (for Google Colab)</h3><p>If you’re using <strong>Google Colab</strong>, authenticate your environment to access Google Cloud resources:</p><pre>import sys<br><br>if &quot;google.colab&quot; in sys.modules:<br>    # Authenticate user to Google Cloud<br>    from google.colab import auth<br>​<br>    auth.authenticate_user()</pre><p>Initialize your project and region</p><pre>from google import genai<br>from google.cloud import aiplatform<br><br>PROJECT_ID = &quot;your-project-id&quot;<br>LOCATION   = &quot;us-central1&quot;<br><br>aiplatform.init(project=PROJECT_ID, location=LOCATION)<br><br>client = genai.Client(vertexai=True,<br>                      project=PROJECT_ID,<br>                      location=LOCATION)</pre><h3>Basic gemini Multimodal Calls</h3><p>Load the gemini Flash model</p><pre>MODEL_ID = &quot;gemini-2.0-flash&quot;<br>model = client.models.get(MODEL_ID)</pre><p>Define a helper to load an image as a Part</p><pre>from google.genai.types import Part<br><br>def image_part(url):<br>    return Part.from_uri(file_uri=url, mime_type=&quot;image/jpeg&quot;)</pre><p>Example 1: describe a document page</p><pre>page_image = image_part(<br>  &quot;gs://my-bucket/docs/contract-page1.jpg&quot;<br>)<br>prompt = &quot;Summarize the key obligations on this page.&quot;<br>resp = client.generate_content(<br>  model=MODEL_ID,<br>  contents=[page_image, prompt]<br>)<br>print(resp.text)</pre><p>Example 2: extract data from a chart image</p><pre>chart_image = image_part(<br>  &quot;gs://my-bucket/docs/sales-chart.png&quot;<br>)<br>prompt = &quot;&quot;&quot;<br>Read the bar chart and list the top three selling regions,<br>including their sales figures.<br>&quot;&quot;&quot;<br>resp = client.generate_content(<br>  model=MODEL_ID,<br>  contents=[chart_image, prompt]<br>)<br>print(resp.text)</pre><h3>Building a Multimodal RAG Pipeline</h3><h4>Step 1 Document preprocessing</h4><p>Split your PDF or scanned multipage document into pages. For each page:</p><ul><li>generate a text embedding via Vertex AI Embeddings</li><li>if the page contains images you wish to retrieve, extract those and compute image embeddings</li></ul><p>Store embeddings and metadata (page number, image coordinates) in a vector database (for example, Vertex AI Matching Engine or a third-party like Pinecone).</p><h4>Step 2 Query time retrieval</h4><p>When a user asks a question, embed the question text and retrieve the top k matching passages or page images. For example:</p><pre>from google.cloud import aiplatform<br><br># create text embedding<br>embed_model = aiplatform.EmbeddingModel(<br>    &quot;textembedding-gecko@001&quot;<br>)<br>query_embed = embed_model.get_embedding(<br>    [&quot;What are the termination clauses?&quot;]<br>)[0]<br><br># retrieve top 3<br>results = my_vector_db.query(<br>  vector=query_embed,<br>  top_k=3<br>)</pre><p>Each result includes either a page image URI or extracted text snippet.</p><h4>Step 3 Prompt construction</h4><p>Build a prompt that interleaves retrieved content. For a multimodal question:</p><pre>prompt = &quot;Answer the question based on the following document passages and images.\nQuestion: What are the notice periods for termination?\n&quot;<br>contents = [prompt]<br><br>for r in results:<br>  if r.type == &quot;text&quot;:<br>    contents.append(r.snippet)<br>  else:<br>    contents.append(Part.from_uri(<br>      file_uri=r.image_uri,<br>      mime_type=&quot;image/jpeg&quot;<br>    ))<br><br>resp = client.generate_content(<br>  model=MODEL_ID,<br>  contents=contents,<br>  config={&quot;maxOutputTokens&quot;: 512}<br>)<br>print(resp.text)</pre><h3>Citation-Aware Answering</h3><p>To build answers with explicit citations, you can:</p><ol><li>assign each retrieved passage or image a label: [Passage 1], [Image A]</li><li>instruct gemini to reference those labels in its output</li></ol><p>Example:</p><pre>prompt = &quot;&quot;&quot;<br>Use the following labelled content to answer.<br>Include citations like (Passage 2) or (Image A).<br>Question: Who is liable for damages under section 4?<br>&quot;&quot;&quot;<br>contents = [prompt]<br>for idx, r in enumerate(results):<br>  label = f&quot;[Passage {idx+1}]&quot; if r.type==&quot;text&quot; else f&quot;[Image {chr(65+idx)}]&quot;<br>  contents.append(label)<br>  if r.type==&quot;text&quot;:<br>    contents.append(r.snippet)<br>  else:<br>    contents.append(Part.from_uri(file_uri=r.image_uri, mime_type=&quot;image/jpeg&quot;))<br><br>resp = client.generate_content(model=MODEL_ID, contents=contents)<br>print(resp.text)</pre><h3>Automating Metadata Extraction</h3><p>You can use gemini to extract structured metadata (dates, amounts, names) from forms or invoices. After spotting fields with OCR, ask:</p><pre>invoice_img = image_part(&quot;gs://my-bucket/invoices/inv123.jpg&quot;)<br>prompt = &quot;&quot;&quot;<br>Extract the following fields in JSON:<br>- invoice_number<br>- invoice_date<br>- total_amount<br>- payer_name<br>&quot;&quot;&quot;<br>resp = client.generate_content(<br>  model=MODEL_ID,<br>  contents=[invoice_img, prompt],<br>  config={&quot;responseMimeType&quot;:&quot;application/json&quot;}<br>)<br>print(resp.text)</pre><p>You can parse the JSON and store it in your database.</p><h3>Here is another example for Extracting Insights from Rich Documents Using Gemini</h3><p>Now, let’s see how to extract insights from rich documents using <strong>Gemini</strong> by combining <strong>text</strong> and <strong>images</strong>.</p><h4>Understanding Images in Context</h4><p>To leverage Gemini’s capabilities, you can provide it with a <strong>room image</strong> and ask it to describe the scene and analyze its contents. This example showcases how Gemini understands and generates a detailed description of an image.</p><pre>from google import genai<br>from google.genai.types import Part<br><br>client = genai.Client(vertexai=True, project=&quot;[your-project-id]&quot;, location=&quot;[your-region]&quot;)<br>MODEL_ID = &quot;gemini-2.0-flash&quot;<br><br># URL for the room image for example<br>room_image_url = &quot;https://storage.googleapis.com/github-repo/img/gemini/retail-recommendations/rooms/spacejoy-c0JoR_-2x3E-unsplash.jpg&quot;<br>room_image = Part.from_uri(file_uri=room_image_url, mime_type=&quot;image/jpeg&quot;)<br><br># Prompt to describe the room<br>prompt = &quot;Describe what&#39;s visible in this room and the overall atmosphere:&quot;<br><br>contents = [room_image, prompt]<br><br># Generate response from Gemini<br>responses = client.models.generate_content_stream(model=MODEL_ID, contents=contents)<br><br>for response in responses:<br>    print(response.text, end=&quot;&quot;)</pre><p>This code provides a room image and asks Gemini to describe the visible elements, such as furniture, lighting, and overall ambiance.</p><h3>Building a Multimodal Recommendation System</h3><p>Let’s now use <strong>Gemini 2.0</strong> to recommend furniture based on a <strong>room image</strong>. This scenario simulates a <strong>customer’s decision-making process</strong> when choosing a suitable piece of furniture.</p><h4>Scenario: Recommending Furniture Based on Room Images</h4><p>You have a room image, and your task is to recommend a suitable piece of furniture (like a chair). The model analyzes the room’s style and recommends the best piece.</p><pre># Image URLs for the chairs for example<br>furniture_image_urls = [<br>    &quot;https://storage.googleapis.com/github-repo/img/gemini/retail-recommendations/furnitures/cesar-couto-OB2F6CsMva8-unsplash.jpg&quot;,<br>    &quot;https://storage.googleapis.com/github-repo/img/gemini/retail-recommendations/furnitures/daniil-silantev-1P6AnKDw6S8-unsplash.jpg&quot;,<br>    &quot;https://storage.googleapis.com/github-repo/img/gemini/retail-recommendations/furnitures/ruslan-bardash-4kTbAMRAHtQ-unsplash.jpg&quot;,<br>    &quot;https://storage.googleapis.com/github-repo/img/gemini/retail-recommendations/furnitures/scopic-ltd-NLlWwR4d3qU-unsplash.jpg&quot;<br>]<br><br># Load chair images<br>furniture_images = [Part.from_uri(file_uri=url, mime_type=&quot;image/jpeg&quot;) for url in furniture_image_urls]<br><br># Prompt to recommend furniture<br>contents = [<br>    &quot;Consider the following chairs:&quot;,<br>    &quot;chair 1:&quot;, furniture_images[0],<br>    &quot;chair 2:&quot;, furniture_images[1],<br>    &quot;chair 3:&quot;, furniture_images[2],<br>    &quot;chair 4:&quot;, furniture_images[3],<br>    &quot;room:&quot;, room_image,<br>    &quot;You are an interior designer. For each chair, explain whether it would be appropriate for the style of the room:&quot;<br>]<br><br>responses = client.models.generate_content_stream(model=MODEL_ID, contents=contents)<br><br>for response in responses:<br>    print(response.text, end=&quot;&quot;)<br></pre><p>This code provides four chair options and a room image. Gemini will evaluate each chair and explain which would fit best based on the room’s style.</p><h3>7. Multimodal RAG in Action</h3><p>In this section, we’ll implement <strong>Multimodal RAG</strong>. The idea is to enhance the model’s ability to make <strong>informed decisions</strong> by retrieving relevant images and textual data from external sources and feeding it into the Gemini model.</p><h4>Implementing a RAG System</h4><p>Let’s simulate how a <strong>RAG system</strong> could work for a furniture store. We’ll retrieve images of available chairs from the store’s inventory, ask the user to upload a room image, and then use Gemini to recommend the most appropriate chair.</p><pre># Room image and chair options for recommendation<br>contents = [<br>    &quot;Consider the following chairs:&quot;,<br>    &quot;chair 1:&quot;, furniture_images[0],<br>    &quot;chair 2:&quot;, furniture_images[1],<br>    &quot;chair 3:&quot;, furniture_images[2],<br>    &quot;chair 4:&quot;, furniture_images[3],<br>    &quot;room:&quot;, room_image,<br>    &quot;You are an interior designer. Return in JSON, for each chair, whether it would fit in the room, with an explanation:&quot;<br>]<br><br>responses = client.models.generate_content_stream(<br>    model=MODEL_ID,<br>    contents=contents,<br>    config=GenerateContentConfig(response_mime_type=&quot;application/json&quot;),<br>)<br><br>for response in responses:<br>    print(response.text, end=&quot;&quot;)</pre><p>This implementation uses the <strong>RAG</strong> technique by feeding both <strong>visual context (images)</strong> and <strong>furniture options</strong> into Gemini. The model processes the input and generates recommendations with explanations in <strong>JSON format</strong>.</p><h3>Now, lets talk some more about Multimodal Retrieval-Augmented Generation (RAG) using the Gemini API in Vertex AI</h3><p>In recent years, the integration of multimodal data (such as text, images, and even videos) into machine learning models has gained significant attention. Multimodal Retrieval-Augmented Generation (RAG) is one such advancement that combines both retrieval and generation techniques to solve complex tasks by leveraging diverse data sources. This article explores how to implement a multimodal RAG pipeline using the <strong>Gemini API</strong> in <strong>Vertex AI</strong>, utilizing text and image data for search, comparison, and reasoning tasks.</p><p>We will demonstrate how to create a powerful multimodal system capable of answering complex queries based on both textual and visual data. This system can handle various use cases, such as document analysis, comparative reasoning, and content generation from both images and text.</p><h3>Understanding the Key Concepts: RAG, Gemini API, and Vertex AI</h3><p>Before diving into the code and implementation, let’s take a brief moment to understand the core concepts that form the foundation of this system:</p><ul><li><strong>Retrieval-Augmented Generation (RAG)</strong>: RAG is a hybrid model that combines two critical components:</li><li><strong>Retrieval</strong>: The model retrieves relevant documents or data points from a large corpus based on a query. This retrieval process can be based on either text or image embeddings.</li><li><strong>Generation</strong>: Once the relevant data is retrieved, a generative model (like Gemini or GPT-3) uses this information to answer questions or generate text.</li><li><strong>Gemini API</strong>: The Gemini API is a powerful tool that enables integration with <strong>Google Cloud’s Vertex AI</strong>. It is built to handle multimodal data, including images and text. By using <strong>Gemini 2.0 Flash</strong>, we can leverage the model’s capabilities to perform comparative reasoning and answer complex queries across various data types.</li><li><strong>Vertex AI</strong>: A comprehensive machine learning platform provided by Google Cloud. It allows for the training and deployment of models, including those utilizing the Gemini API for advanced reasoning and retrieval tasks.</li></ul><h3>Setting Up Your Environment</h3><p>Before we dive into the details of the pipeline, ensure that you have the following prerequisites set up:</p><ul><li><strong>Google Cloud Platform (GCP)</strong> account with Vertex AI enabled.</li><li><strong>Gemini API</strong> access, which is part of the Google Cloud AI.</li><li><strong>Python</strong> environment with the necessary libraries installed, including:</li></ul><p>You can install the required libraries with the following commands:</p><pre>pip install --upgrade google-cloud-aiplatform<br>pip install openai<br>pip install matplotlib<br>pip install pillow<br>pip install pandas</pre><h3>Step-by-Step Workflow for Multimodal RAG using Gemini API</h3><h4>Step 1: Define the User Query</h4><p>The first step in the RAG pipeline is receiving a user query. This query can be either entirely textual or multimodal (i.e., it could include an image and text). For our example, let’s consider the following user query:</p><pre>Questions:<br>- What are the critical differences between various graphs for Google Class A Share?<br>- Which index best matches Class A share performance closely where Google is not already a part? Explain the reasoning.<br>- Identify key chart patterns for Google Class A shares.<br>- What is cost of revenues, operating expenses, and net income for 2020? Do mention the percentage change.<br>- What was the effect of Covid on the 2020 financial year?<br>- What are the total revenues for APAC and USA for 2021?</pre><p>This query requests comparative analysis of graphs related to Google Class A shares, financial metrics for 2020, and geographic revenue breakdowns.</p><h4>Step 2: Retrieve Relevant Text Chunks</h4><p>To answer the query, we must first search the document for relevant text chunks that may contain information about <strong>Google Class A shares</strong> and other financial aspects. We’ll use the get_similar_text_from_query() function to search for these text chunks based on text embeddings.</p><p>Here’s how we can implement the text retrieval process:</p><pre>from google.cloud import vertex_ai<br>from google.cloud import storage<br><br># Initialize Vertex AI client<br>client = vertex_ai.PipelineServiceClient()<br><br># Query to retrieve relevant text chunks<br>query = &quot;What is the cost of revenues, operating expenses, and net income for 2020?&quot;<br><br># Function to retrieve relevant text chunks<br>def get_similar_text_from_query(query, text_metadata_df, column_name=&quot;text_embedding_chunk&quot;, top_n=5):<br>    &quot;&quot;&quot;<br>    This function retrieves relevant text chunks from a corpus based on a user&#39;s query.<br>    &quot;&quot;&quot;<br>    # Assuming embeddings are stored in the column &#39;column_name&#39;<br>    # Perform similarity search using the query and metadata embeddings<br>    matching_results = text_metadata_df.loc[<br>        text_metadata_df[column_name].apply(lambda x: similarity(query, x) &gt; 0.7)<br>    ].head(top_n)<br>    <br>    return matching_results<br><br># Example usage<br>matching_text = get_similar_text_from_query(query, text_metadata_df)<br>print(matching_text)</pre><p>This code retrieves the top 5 text chunks based on the similarity between the query and the stored text embeddings. The similarity function compares the query’s text embedding with the embeddings stored in the text metadata dataframe.</p><h4>Step 3: Retrieve Similar Images</h4><p>Once we’ve retrieved relevant text chunks, the next step is to search for images that may visually represent data related to the query. For example, if the user queries about financial performance, we may want to find graphs or charts depicting the cost of revenues, operating expenses, and net income.</p><p>We use the get_similar_image_from_query() function to search for relevant images from a dataset of images stored in the metadata.</p><pre>from PIL import Image<br>import numpy as np<br><br># Function to retrieve similar images based on text embedding<br>def get_similar_image_from_query(text_metadata_df, image_metadata_df, query, column_name=&quot;text_embedding_from_image_description&quot;, top_n=3):<br>    &quot;&quot;&quot;<br>    This function retrieves the most relevant images based on the user&#39;s query.<br>    &quot;&quot;&quot;<br>    # Assuming image descriptions are stored in the column &#39;column_name&#39;<br>    matching_results = image_metadata_df.loc[<br>        image_metadata_df[column_name].apply(lambda x: similarity(query, x) &gt; 0.7)<br>    ].head(top_n)<br>    <br>    return matching_results<br><br># Example usage<br>image_query = &quot;Google Class A cumulative 5-year total return&quot;<br>matching_images = get_similar_image_from_query(text_metadata_df, image_metadata_df, image_query)</pre><p>In the code above, we search for images related to “Google Class A cumulative 5-year total return” based on text embeddings extracted from image descriptions. These results are returned as the top 3 matching images.</p><h4>Step 4: Combine Text and Image Context for Gemini</h4><p>At this point, we have both textual and visual data that are relevant to the user’s query. The next step is to combine these text and image results into a unified context that can be passed to the <strong>Gemini 2.0 Flash model</strong> for further analysis.</p><pre># Combine text and image results<br>final_context_text = &quot;\n&quot;.join(matching_text[&#39;chunk_text&#39;].values)<br>context_images = []<br><br>for idx, image_row in matching_images.iterrows():<br>    context_images.append(f&quot;Image: {image_row[&#39;image_object&#39;]}\nCaption: {image_row[&#39;image_description&#39;]}&quot;)<br><br># Prepare the final prompt<br>prompt = f&quot;&quot;&quot;<br>Instructions: Compare the images and the text provided as Context: to answer multiple questions.<br>Make sure to think thoroughly before answering and put the necessary steps to arrive at the answer in bullet points.<br><br>Context:<br> - Text Context:<br>{final_context_text}<br><br> - Image Context:<br>{&#39;&#39;.join(context_images)}<br><br>{query}<br>&quot;&quot;&quot;</pre><p>The context is now a combination of text and images that will provide Gemini with all the necessary information to generate a relevant response.</p><h4>Step 5: Pass Context to Gemini for Multimodal Reasoning</h4><p>Now that we have our <strong>context</strong> (both textual and visual), we can pass this to the <strong>Gemini 2.0 Flash model</strong> for generating answers. Here’s how you can send the context to Gemini for reasoning:</p><pre>from google.cloud import vertex_ai<br><br># Initialize the Gemini client<br>gemini_client = vertex_ai.PipelineServiceClient()<br><br># Function to get response from Gemini model<br>def get_gemini_response(prompt):<br>    response = gemini_client.generate_text(<br>        model_name=&quot;projects/my-project/locations/us-central1/models/gemini-2.0-flash&quot;,<br>        prompt=prompt,<br>        temperature=1.0<br>    )<br>    <br>    return response.result<br><br># Get Gemini&#39;s response<br>gemini_response = get_gemini_response(prompt)<br>print(gemini_response)</pre><p>This function sends the combined context to Gemini, which will generate a detailed answer to the user’s query based on both the images and the text.</p><h4>Step 6: Display Citations and References</h4><p>Once the answer is generated, it’s important to display citations for transparency and reference purposes. These citations help users understand where the information came from.</p><pre>def print_text_to_image_citation(matching_results_image):<br>    for idx, result in enumerate(matching_results_image):<br>        print(f&quot;Image {idx+1} Citation: {result[&#39;image_description&#39;]}&quot;)<br><br>def print_text_to_text_citation(matching_results_text):<br>    for idx, result in enumerate(matching_results_text):<br>        print(f&quot;Text Citation: {result[&#39;chunk_text&#39;]}&quot;)<br><br># Display citations for both text and images<br>print(&quot;---------------Image Citations------------------&quot;)<br>print_text_to_image_citation(matching_images)<br><br>print(&quot;---------------Text Citations------------------&quot;)<br>print_text_to_text_citation(matching_text)</pre><p>This will print the citations for both the images and text that were used to generate the final response, ensuring that the sources are clear and verifiable.</p><h3>Conclusion</h3><p>leveraging Gemini 2.0’s multimodal capabilities significantly enhances the ability to extract meaningful insights from rich documents. By combining text, images, and even video, Gemini provides a powerful tool for document analysis and retrieval-augmented generation (RAG). The <strong>Inspect Rich Documents with Gemini Multimodality and Multimodal RAG</strong> course by google cloud offers a comprehensive learning experience for professionals looking to utilize Gemini models and Vertex AI in document analysis. It covers advanced techniques for processing and analyzing complex documents containing both text and visual elements, providing hands-on practice in multimodal prompting, metadata extraction, embedding generation, and RAG.</p><p>The course is designed for applications such as retail recommendations, financial document analysis, and content creation, making it highly relevant for AI-driven projects. By completing the course, learners gain practical skills to implement advanced AI techniques for document inspection, enhance search and retrieval accuracy, and improve decision-making processes. Multimodal RAG systems, particularly when integrating text and visual content, allow for richer, more context-aware responses, transforming how we interact with and analyze documents.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b135002c3c8b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Learning Gemini & Imagen Can Make You a More Competitive Developer]]></title>
            <link>https://medium.com/@18bhavyasharma/how-learning-gemini-imagen-can-make-you-a-more-competitive-developer-33f6293cbe61?source=rss-7496da9f1e68------2</link>
            <guid isPermaLink="false">https://medium.com/p/33f6293cbe61</guid>
            <category><![CDATA[aiforviksitbharat]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[genaiacademy]]></category>
            <category><![CDATA[genai-exchange]]></category>
            <category><![CDATA[gemini]]></category>
            <dc:creator><![CDATA[bhavya sharma]]></dc:creator>
            <pubDate>Thu, 24 Apr 2025 19:02:14 GMT</pubDate>
            <atom:updated>2025-05-04T19:38:25.181Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6yqixLIPXXSHi3k8OorIOg.jpeg" /></figure><p>In an era where artificial intelligence is reshaping industries, understanding and applying generative AI models is no longer optional for software professionals. Two of Google’s most powerful tools— Gemini for multimodal text and chat, and Imagen for high-quality image generation — open entirely new avenues for innovation. By mastering these tools, you gain the ability to build applications that can converse naturally, analyze or describe visual content, and generate realistic image on demand.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*MjTNCVva31DXVVi7yy0LcA.png" /></figure><p>Recently, I completed the Build Real World AI Applications with Gemini and Imagen Skill Badge by google cloud in which I have learned how to use the Vertex AI Python SDK to connect to Google’s AI platform and load pre-trained generative models — Gemini for multi-modal text and image understanding and Imagen for high-quality image generation — sending both text and image prompts with streaming and non-streaming methods to build chat and recognition applications, These skills are important because generative AI offers new and disruptive opportunities to increase revenue, reduce costs, improve productivity, and better manage risk by automating content creation and enriching user experiences Integrating large language and vision models into production workflows accelerates development, lowers infrastructure overhead, and can unlock trillions of dollars in value across industries from finance to healthcare</p><h3>Why Generative AI Matters for Developers</h3><p>Developers who integrate generative AI move beyond static features and into products that learn, adapt, and interact more naturally. Key competitive advantages include,</p><ul><li><strong>Better user experiences</strong>: Chatbots powered by generative AI can remember past conversations, answer follow-up questions, and even understand images. That’s way more helpful than boring menus or basic FAQs.</li><li><strong>Faster prototyping</strong>: Need a mockup or placeholder image? You can use tools like Imagen to generate one in minutes.</li><li><strong>Smarter insights from data</strong>: Tools like Gemini can summarize long documents, pull out key points, or translate different languages. That means less manual work and quicker results.</li><li><strong>Stand out from the crowd</strong>: As AI becomes more common, being able to customize and fine-tune your own AI features can really set your product apart from the rest.</li></ul><h3>Understanding Gemini and Imagen</h3><h3>What Is Gemini?</h3><p>Gemini is Google’s state-of-the-art generative AI model family that excels at natural language understanding, multi-turn conversations, and multimodal inputs (combining text and images). Through the Vertex AI platform, you access these capabilities via a unified SDK that abstracts away infrastructure details, allowing both synchronous and streaming chat interactions.</p><h3>What Is Imagen?</h3><p>Imagen is Google’s high-fidelity image generation model that transforms text prompts into photorealistic images. Hosted on Vertex AI, Imagen provides on-demand image synthesis, optional upscaling, and mask-based editing. The same SDK interface lets you call Imagen models alongside Gemini models, ensuring a consistent development experience.</p><h3>Getting Started with the Google Gen AI SDK</h3><h4>Installing the Libraries</h4><p>To begin, install the core SDKs:</p><pre>pip install google-genai                  # for Gemini Developer API &amp; Vertex AI Gen AI<br>pip install google-cloud-aiplatform&gt;=1.38 # for Vertex AI client and high-level utilities<br>pip install --upgrade google-cloud-aiplatform # to install the latest version , you can use this</pre><p>These commands readies both text and vision pipelines in Python.</p><h3>Authenticating and Initializing</h3><p>Initialize your environment and authenticate via the Google Cloud CLI</p><pre>gcloud auth login<br>gcloud config set project YOUR_PROJECT_ID<br>gcloud config set ai/region YOUR_REGION</pre><p>In Python, create a client instance that works for both Gemini and Imagen:</p><pre>from google import genai<br>from google.genai.types import HttpOptions<br><br>client = genai.Client(<br>    vertexai=True,<br>    project=&quot;YOUR_PROJECT_ID&quot;,<br>    location=&quot;YOUR_REGION&quot;,<br>    http_options=HttpOptions(api_version=&quot;v1&quot;)<br>)</pre><p>This unified client lets you call text-based and image-based generative endpoints with the same interface.</p><h3>Building Text-Based Generative Apps with Gemini</h3><h4>Synchronous Chat</h4><p>Here’s a custom example to create a Q&amp;A assistant:</p><pre>from google.genai.types import Part, UserContent, ModelContent<br><br># Start a chat session with initial context<br>chat = client.chats.create(model=&quot;gemini-2.0-flash-001&quot;)<br><br># Add user and system messages<br>chat.send_message(&quot;Hello, Gemini! I&#39;m planning a road trip.&quot;)<br>response = chat.send_message(&quot;What are three must-visit sights in California?&quot;)<br>print(&quot;Gemini says:&quot;, response.text)</pre><p>This code demonstrates a simple back-and-forth conversation, with all messages batched until response time.</p><h4>Streaming Responses</h4><p>To display tokens as they arrive, enabling a “live” feel:</p><pre>stream = client.chats.create(model=&quot;gemini-2.0-flash-001&quot;)<br>print(&quot;Listening...&quot;)<br>for chunk in stream.send_message_stream(&quot;Tell me a short story about a time-traveling cat&quot;):<br>    print(chunk.text, end=&quot;&quot;, flush=True)</pre><p>Streaming improves user engagement in chatbots or live-update interfaces.</p><h3>Building Image Generation Apps with Imagen</h3><h4>Basic Image Synthesis</h4><p>Generate images from text prompts with minimal code:</p><pre>from vertexai.preview.vision_models import ImageGenerationModel<br>import uuid<br><br>def create_image(prompt: str, count: int = 1):<br>    # Initialize within the function for standalone use<br>    ImageGenerationModel.init(project=&quot;YOUR_PROJECT_ID&quot;, location=&quot;YOUR_REGION&quot;)<br>    model = ImageGenerationModel.from_pretrained(&quot;imagen-3.0-generate-002&quot;)<br>    <br>    results = model.generate_images(prompt=prompt, number_of_images=count)<br>    for img in results:<br>        filename = f&quot;{uuid.uuid4()}.png&quot;<br>        img.save(filename)<br>        print(&quot;Saved:&quot;, filename)</pre><p>This snippet showcases generating multiple images, each saved with a unique name.</p><h4>Upscaling and Inpainting</h4><p>To refine or edit an existing image:</p><pre># Upscale a generated image<br>upscaled = model.upscale_image(image=results[0])<br>upscaled.save(&quot;upscaled.png&quot;)<br><br># Inpaint a masked area<br>mask = Image.load_from_file(&quot;mask.png&quot;)<br>inpainted = model.inpaint(image=results[0], mask=mask, prompt=&quot;Add a sunset sky&quot;)<br>inpainted.save(&quot;inpainted.png&quot;)</pre><p>Use upscaling to enhance resolution and inpainting for targeted edits, leveraging Imagen’s advanced capabilities.</p><h3>Best Practices and Tips</h3><h4>Prompt Engineering</h4><ul><li><strong>Be specific</strong>: Clearly define the style, context, and constraints in your prompt.</li><li><strong>Iterate quickly</strong>: Adjust prompts on the fly to refine outputs.</li><li><strong>Use system messages</strong>: In chat apps, set expectations (e.g., “You are a travel guide.”).</li></ul><h4>Cost Management</h4><ul><li><strong>Use smaller models</strong> for prototyping (e.g., gemini-1.5 pro before 2.0 flash ).</li><li><strong>Cache responses</strong> when possible to reduce repeat calls.</li></ul><h4>Error Handling</h4><ul><li><strong>Retry on 500 errors</strong>: Transient server issues can be resolved with simple retries.</li><li><strong>Validate inputs</strong>: Ensure image URIs or prompt lengths meet API requirements.</li></ul><h3>Generate and Analyze Images with Imagen and Gemini</h3><p>let’s walk through a complete example where we generate an image with Imagen and then analyze it using Gemini’s multimodal capabilities.</p><h4>Step 1: Generate an Image Using Imagen</h4><p>In this example, we’ll generate an image of a fantasy landscape using the Imagen model:</p><pre>import vertexai<br>from vertexai.preview.vision_models import ImageGenerationModel<br><br>def generate_fantasy_landscape_image(prompt: str, output_file: str = &quot;fantasy_landscape.png&quot;):<br>    &quot;&quot;&quot;Generate an image using the Imagen model based on the provided prompt.&quot;&quot;&quot;<br>    vertexai.init(project=&quot;Your project id&quot;, location=&quot;us-west1&quot;)<br><br>    # Load the pre-trained Imagen model<br>    model = ImageGenerationModel.from_pretrained(&quot;imagen-3.0-generate-002&quot;)<br><br>    # Generate the image<br>    images = model.generate_images(<br>        prompt=prompt,<br>        number_of_images=1,<br>        seed=42,<br>        add_watermark=False,<br>    )<br><br>    # Save the generated image locally<br>    images[0].save(output_file)<br>    print(f&quot;Image saved as {output_file}&quot;)<br><br># Example usage<br>generate_fantasy_landscape_image(&quot;Create a magical forest landscape with glowing mushrooms and a purple sky&quot;)</pre><p>This script initializes the Imagen model and generates a visually rich fantasy environment, saving the image for further analysis.</p><h4>Step 2: Analyze the Generated Image Using Gemini</h4><p>Once we’ve generated our fantasy image, we can now analyze it with Gemini to generate a creative story description based on what the image contains:</p><pre># Import necessary modules<br>import vertexai<br>from vertexai.generative_models import GenerativeModel, Part, Image<br><br># Set your project ID and location<br>PROJECT_ID = &quot;project id&quot;<br>LOCATION = &quot;us-west1&quot;<br><br># Function to analyze the landscape image and generate a story<br>def analyze_landscape_image(image_path: str) -&gt; str:<br>    # Initialize the Vertex AI client with the given project and location<br>    vertexai.init(project=PROJECT_ID, location=LOCATION)<br>    <br>    # Load the pre-trained Gemini model (gemini-2.0-flash-001)<br>    multimedia_model = GenerativeModel(&#39;gemini-2.0-flash-001&#39;)<br><br>    # Create the parts for the prompt and the image<br>    text_part = Part.from_text(&#39;Describe a short fantasy story based on this image&#39;)<br>    image_part = Part.from_image(Image.load_from_file(image_path))<br>    <br>    # Generate the content by passing the text and image parts<br>    response = multimedia_model.generate_content([text_part, image_part])<br>    <br>    # Return the generated text (story)<br>    return response.text<br><br># Example usage - replace with the correct image file path<br>print(analyze_landscape_image(&#39;./fantasy_landscape.png&#39;))</pre><p>This showcases the full pipeline — from image generation to multimodal interpretation — using Imagen and Gemini. Whether you’re building an AI art assistant, educational tool, or creative content generator, this workflow forms the backbone of powerful generative applications.</p><h3>Real-World Applications of Imagen and Gemini</h3><h4>Virtual Product Try-On and Customization</h4><ul><li><strong>Description:</strong> Combine Imagen for rendering lifelike product visuals (e.g., apparel, eyewear, furniture) in user-specific contexts (body shape, room layout) with Gemini-powered chat guidance for style choices and fit advice.</li><li><strong>Impact:</strong> Reduces return rates, enhances customer engagement, and accelerates purchase decisions through personalized, interactive experiences.</li></ul><h4>Interactive Art Therapy Platform</h4><ul><li><strong>Description:</strong> Develop a multimodal wellness app where users describe their emotional state to Gemini, which suggests therapeutic prompts, while Imagen generates calming or symbolic imagery tailored to individual needs.</li><li><strong>Impact:</strong> Expands access to mental health support, offering cost-effective, scalable art therapy interventions.</li></ul><h4>Cultural Heritage Reconstruction and Storytelling</h4><ul><li><strong>Description:</strong> Use Imagen to recreate historical landmarks or artifacts from old photos and textual descriptions; integrate Gemini to provide dynamically narrated tours, answering visitor questions in multiple languages.</li><li><strong>Impact:</strong> Enhances virtual museum experiences and democratizes access to cultural education worldwide.</li></ul><h4>AI-Driven Architectural Concept Generator</h4><ul><li><strong>Description:</strong> Architects input high-level requirements (e.g., “three-story sustainable office building”) into Gemini to refine specifications, then Imagen produces conceptual renderings and material visualizations.</li><li><strong>Impact:</strong> Speeds up early-stage design iterations and client feedback loops, reducing time and cost in the architectural process.</li></ul><h4>Supply Chain Digital Twin Visualizations</h4><ul><li><strong>Description:</strong> Render real-time digital twins of factory floors and logistics networks using Imagen, while Gemini provides natural-language alerts and recommendations on bottlenecks and inventory levels.</li><li><strong>Impact:</strong> Improves operational visibility, anticipates disruptions, and optimizes resource allocation.</li></ul><h4>Adaptive E-Commerce Product Catalogs</h4><ul><li><strong>Description:</strong> Automatically generate on-demand product variants (color, pattern, style) via Imagen, coupled with Gemini chat agents that guide customers through product discovery and customization.</li><li><strong>Impact:</strong> Creates highly personalized shopping experiences and expands product lineups without photography overhead.</li></ul><h4>Real-Time Emergency Response Coordination</h4><ul><li><strong>Description:</strong> Deploy Imagen to simulate disaster scenarios (wildfires, floods) based on geospatial data, while Gemini serves as a command-center chatbot that suggests evacuation routes and resource deployment plans.</li><li><strong>Impact:</strong> Enhances situational awareness and decision-making for first responders and relief agencies.</li></ul><h4>Environmental Monitoring and Simulation</h4><ul><li><strong>Description:</strong> Feed satellite or drone imagery into Gemini for real-time analysis of deforestation, pollution hotspots, or wildlife movements, and generate annotated reports and forecast visuals via Imagen.</li><li><strong>Impact:</strong> Supports conservation efforts and policy planning with actionable insights.</li></ul><h4>Personalized Educational Content Creation</h4><ul><li><strong>Description:</strong> Teachers use Gemini to draft tailored lesson narratives and quizzes for different learning levels, and Imagen to generate illustrative diagrams and interactive flashcards on the fly.</li><li><strong>Impact:</strong> Enables differentiated instruction at scale and boosts student engagement through multimedia learning materials.</li></ul><h3><strong>Conclusion</strong></h3><p>Mastering tools like Gemini and Imagen empowers developers to build intelligent, multimodal applications that go far beyond traditional software capabilities. These models enable seamless interactions through natural language, image recognition, and generation — helping developers create products that are more engaging, responsive, and impactful. From prototyping to production, the ability to harness generative AI through the Vertex AI platform opens doors to innovation in diverse sectors such as e-commerce, healthcare, education, and architecture. As AI continues to shape the future of technology, developers who adopt these tools now will hold a significant competitive edge in delivering next-generation experiences.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33f6293cbe61" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Generative AI with Google Vertex AI Studio and Gemini]]></title>
            <link>https://medium.com/@18bhavyasharma/mastering-generative-ai-with-google-vertex-ai-studio-and-gemini-792f9dc4e204?source=rss-7496da9f1e68------2</link>
            <guid isPermaLink="false">https://medium.com/p/792f9dc4e204</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[genai-exchange]]></category>
            <category><![CDATA[aiforviksitbharat]]></category>
            <category><![CDATA[vertex-ai]]></category>
            <category><![CDATA[genaiacademy]]></category>
            <dc:creator><![CDATA[bhavya sharma]]></dc:creator>
            <pubDate>Tue, 22 Apr 2025 18:10:00 GMT</pubDate>
            <atom:updated>2025-05-04T19:35:04.125Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wdd7PeIG-ehI2TK7REis6w.jpeg" /></figure><p>Prompt design in Vertex AI is the art of crafting clear, well-structured requests that guide Google’s generative models to produce useful, reliable outputs. Effective prompts start with a precise objective and include just the right amount of context — no more, no less — and are organized logically with labels, delimiters, and examples. In Vertex AI, prompt engineering follows an iterative, test‑driven workflow: you define your goal, draft a prompt with components such as instructions, context, and persona, systematically test variations, and refine based on generated responses. Moreover, Vertex AI offers specialized tools like the Prompt Optimizer to automate improvement and advanced techniques such as chain‑of‑thought or ReAct prompting for complex reasoning tasks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*AAsuiMNB04ZvPXrrM22Pmw.png" /></figure><p>As AI continues to revolutionize the tech world, one skill that’s become essential is Prompt Engineering. With the growth of Generative AI, our interaction with machines has evolved from writing code to creating prompts. Recently, I completed the Prompt Design in Vertex AI Skill Badge by Google Cloud, and here’s a breakdown of everything I learned.</p><p>In this course, I learned how to build structured prompts that guide large language models like Gemini to produce accurate, useful outputs for tasks like text generation, classification, summarization, and more. It covered key concepts such as prompt components (instructions, context, examples), few-shot prompting, and advanced techniques like chain-of-thought reasoning. I also had the opportunity to work with Vertex AI Studio to experiment with different prompts and understand how small changes in wording or structure can impact the model’s response.</p><h3>What is a prompt?</h3><p>A prompt is simply a natural‑language request you submit to a language model to get it to generate text, images, code, or other outputs. Google Cloud It can include instructions, contextual background, few‑shot examples, or partial inputs for the model to complete. Google Cloud</p><h3>Why prompt design matters in Vertex AI</h3><p>Well‑crafted prompts help models understand exactly what you want, reducing irrelevant or “hallucinated” responses. Poorly defined prompts often lead to vague or incorrect outputs, forcing extra steps of correction. Google Cloud Vertex AI’s performance on tasks — from summarization to code generation depends on prompt clarity and structure.</p><h3>Why Use Vertex AI Studio and Gemini 2.0 Flash?</h3><ul><li><strong>Rapid Prototyping</strong><br> Vertex AI Studio offers a no‑code prompt playground where you can instantly test sample prompts, adjust model settings, and iterate on prompt design without writing infrastructure code.</li><li><strong>Enterprise‑Grade Governance</strong><br> Data used for tuning and inference remains under your control; neither prompts nor weights leave your project, and customer data never retrains Google’s base models.</li><li><strong>Unified Multimodal Foundation Models</strong><br> Gemini 2.0 Flash handles text, images, audio, and video in a single API. It can generate text, images, code, and even structured data like JSON or tables — all with low latency and high throughput.</li><li><strong>Seamless Python Integration</strong><br> The Vertex AI SDK for Python lets you bring generative AI into notebooks and scripts, enabling automated pipelines, scheduled jobs, or integration into web backends .</li></ul><h3>Key Features of Gemini 2.0 Flash</h3><ol><li><strong>Multimodal Inputs &amp; Outputs</strong><br> Supports image, audio, video, and text inputs; can return text responses or generate images and audio streams natively .</li><li><strong>Million‑Token Context Window</strong><br> Enables long‑form generation, document summarization, and extended conversations without losing context .</li><li><strong>Native Tool Use</strong><br> Can invoke grounding tools (e.g., Google Search, code execution) within a prompt to reduce hallucinations and enhance factual accuracy.</li><li><strong>Configurable “Thinking Budget”</strong><br> Control computational reasoning intensity per request — balancing cost, latency, and reasoning depth .</li><li><strong>Safety Filters</strong><br> Built‑in content filters across multiple categories (hate, violence, self‑harm) with adjustable thresholds to meet compliance and brand safety requirements .</li></ol><h3>Getting Started with the Vertex AI SDK for Python</h3><h4>Installation and Initialization</h4><pre>pip install --upgrade --user google-cloud-aiplatform vertexai</pre><pre>import os, vertexai<br>from vertexai.generative_models import GenerativeModel<br><br># Initialize your project<br>PROJECT_ID = os.getenv(&quot;GOOGLE_CLOUD_PROJECT&quot;, &quot;your-project-id&quot;)<br>LOCATION   = os.getenv(&quot;GOOGLE_CLOUD_REGION&quot;, &quot;us-central1&quot;)<br>vertexai.init(project=PROJECT_ID, location=LOCATION)  # SDK v2 style :contentReference[oaicite:14]{index=14}</pre><h4>Loading the Gemini Model</h4><pre>model = GenerativeModel(&quot;gemini-2.0-flash&quot;)</pre><h3>Prompt Engineering Best Practices</h3><h4>1. Be Concise</h4><p>Trim extraneous phrasing to focus the model on core intent, reducing ambiguity and noise .</p><p><strong>Example:</strong></p><pre># Less effective<br>prompt = &quot;Could you possibly suggest some really creative and unique names for a brand new coffee shop that focuses on artisanal beans and sustainable practices?&quot;<br><br># More effective<br>prompt = &quot;Suggest three names for an artisanal, sustainable coffee shop.&quot;</pre><h4>2. Be Specific</h4><p>Define exact requirements — format, tone, list length — to guide output structure .</p><p><strong>Example:</strong></p><pre>prompt = (<br>  &quot;List five benefits of solar energy for homeowners, each in one sentence.&quot;<br>)</pre><h4>3. One Task at a Time</h4><p>Separate distinct asks into individual prompts for clarity and better accuracy.</p><h4>4. Use Examples (Few‑Shot)</h4><p>Provide 1–5 in‑context examples to teach the model desired patterns; watch out for over‑fitting if too many.</p><p><strong>Example (One‑Shot):</strong></p><pre># Example provided for sentiment classification<br>prompt = (<br>    &quot;Classify sentiment as positive, neutral, or negative.\n\n&quot;<br>    &quot;Example:\n&quot;<br>    &quot;Text: The new park is beautiful and peaceful.\n&quot;<br>    &quot;Sentiment: positive\n\n&quot;<br>    &quot;Text: I waited an hour for no reason.\n&quot;<br>    &quot;Sentiment:&quot;<br>)</pre><h4>5. Classification vs. Generation</h4><p>For predictable output, frame tasks as classification (choose an option) rather than open‑ended generation.</p><h4>Generating Text from Simple Prompts</h4><pre># Basic text generation<br>response = model.generate_content(&quot;Explain quantum entanglement in two sentences.&quot;)<br>print(response.text)</pre><h4>Streaming Responses</h4><pre>for chunk in model.generate_content(&quot;Write a haiku about autumn.&quot;, stream=True):<br>    print(chunk.text, end=&quot;&quot;)</pre><h3>Multimodal Prompt Examples</h3><h4>Image Captioning</h4><pre>from vertexai.generative_models import Image<br><br># Assume &#39;sunset.jpg&#39; is a local image<br>image   = Image.load_from_file(&quot;sunset.jpg&quot;)<br>prompt  = &quot;Provide a poetic caption for this sunset image.&quot;<br>response = model.generate_content([image, prompt])<br>print(response.text)</pre><h4>Video Summarization</h4><pre>from vertexai.generative_models import Part<br><br>video = Part.from_uri(&quot;gs://my‑bucket/travel.mp4&quot;, mime_type=&quot;video/mp4&quot;)<br>prompt = &quot;Summarize the main events of this travel video in bullet points.&quot;<br>response = model.generate_content([video, prompt])<br>print(response.text)</pre><h4>Direct Web Media Analysis</h4><pre>url_image = Part.from_uri(<br>    &quot;https://example.com/product.png&quot;, mime_type=&quot;image/png&quot;<br>)<br>prompt = &quot;List all objects visible in this image.&quot;<br>response = model.generate_content([url_image, prompt])<br>print(response.text)</pre><h3>Building Custom Tools: Image Analysis &amp; Tagline Generator</h3><ol><li><strong>Image Analysis Tool</strong></li></ol><ul><li><strong>Objective</strong>: Generate short, catchy, and poetic descriptions for marketing assets.</li><li><strong>Approach:</strong> Upload product image, prompt for multiple styles in one call.</li></ul><pre>image = Image.load_from_file(&quot;gear.jpg&quot;)<br>prompt = (<br>    &quot;For this outdoor gear image, generate:\n&quot;<br>    &quot;1. A concise product description (under 8 words).\n&quot;<br>    &quot;2. A catchy ad slogan.\n&quot;<br>    &quot;3. A poetic tagline emphasizing adventure.&quot;<br>)<br>resp = model.generate_content([image, prompt])<br>print(resp.text)</pre><p><strong>2. Tagline Generator</strong></p><ul><li><strong>Objective:</strong> Create diverse taglines based on product attributes and audience.</li><li><strong>Approach:</strong> Use system instructions, few‑shot examples, and parameterized prompts.</li></ul><pre>from vertexai.generative_models import SafetySetting, HarmCategory, HarmBlockThreshold<br><br># System instructions define context<br>system_inst = [<br>  &quot;You are a creative marketing assistant for an outdoor brand.&quot;<br>]<br>model_with_system = model.start_chat(<br>  config={&quot;system_instructions&quot;: system_inst}<br>)<br><br># Few‑shot examples embedded<br>examples = [<br>  {<br>    &quot;input&quot;: &quot;Durable, lightweight backpack for hikers.&quot;,<br>    &quot;output&quot;: &quot;Trail‑Ready: Pack Less, Trek More&quot;<br>  },<br>  {<br>    &quot;input&quot;: &quot;Waterproof jacket for rainy forest expeditions.&quot;,<br>    &quot;output&quot;: &quot;Rain or Shine, Embrace the Wild&quot;<br>  }<br>]<br>for ex in examples:<br>    model_with_system.send_message(ex[&quot;input&quot;])<br>    model_with_system.send_message(f&quot;Tagline: {ex[&#39;output&#39;]}&quot;)<br><br># Generate new tagline<br>response = model_with_system.send_message(<br>  &quot;Tagline: Affordable, eco‑friendly camping tent.&quot;<br>)<br>print(response.text)</pre><h3>Ensuring Quality and Safety</h3><ul><li><strong>Safety Filters</strong><br> Adjust category thresholds (e.g., block low‑severity hate or sexual content) per request.</li></ul><pre>from vertexai.generative_models import SafetySetting, HarmBlockThreshold, HarmCategory<br>safety = [<br>  SafetySetting(HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT, HarmBlockThreshold.BLOCK_LOW)<br>]<br>resp = model.generate_content(&quot;Suggest edgy slogans.&quot;, safety_settings=safety)</pre><ul><li><strong>Gen AI Evaluation Service</strong><br> Define custom metrics, run evaluations on prompts and models, compare parameter settings to optimize performance.</li></ul><h3>Use Cases: Real-World Applications of Vertex AI Studio and Gemini Flash</h3><h4>1. Personalized Marketing Campaigns</h4><p>Incorporating Gemini 2.0 Flash in personalized marketing offers the ability to tailor ads based on user preferences. By analyzing user behavior and crafting dynamic copy or visuals based on this input, businesses can automate creative processes while ensuring the content resonates with their audience. For example, if a user frequently searches for outdoor gear, Gemini can generate tailored promotional slogans, product descriptions, and even personalized emails or landing pages.</p><p><strong>Example Workflow</strong>:</p><ol><li><strong>Data Collection</strong>: Use analytics to gather data about users’ browsing or purchasing behavior.</li><li><strong>Custom Content Generation</strong>: Feed that data into Gemini to create customized headlines, social media posts, and email marketing content.</li><li><strong>A/B Testing</strong>: Test multiple outputs to see which performs best.</li></ol><pre># Custom marketing campaign tailored to user&#39;s interests<br>user_behavior_data = [<br>    {&quot;activity&quot;: &quot;viewed hiking boots&quot;, &quot;timestamp&quot;: &quot;2025-04-22&quot;},<br>    {&quot;activity&quot;: &quot;purchased water bottle&quot;, &quot;timestamp&quot;: &quot;2025-04-21&quot;}<br>]<br>prompt = &quot;Generate a catchy headline and product description for a new hiking boot.&quot;<br>response = model.generate_content([user_behavior_data, prompt])<br>print(response.text)</pre><h4>2. AI-Enhanced Customer Support</h4><p>By using Gemini Flash for building intelligent customer support bots, businesses can automate responses across multiple channels — chat, voice, and email — while keeping interactions natural. For example, you could train a chatbot to understand customer queries related to billing, order status, and troubleshooting, and generate personalized, context-aware responses in real time.</p><p><strong>Example Workflow</strong>:</p><ol><li><strong>User Query</strong>: A customer asks for help regarding an order status.</li><li><strong>AI Response</strong>: Gemini processes the query, searches through the customer database, and responds in a conversational manner.</li></ol><pre># Chatbot response based on a user&#39;s query<br>prompt = &quot;The user is asking about the status of their order number 12345.&quot;<br>response = model.generate_content([prompt])<br>print(response.text)</pre><h3>3. Automated Content Creation for Blogs and Articles</h3><p>Gemini Flash can be used to automatically create blog posts, articles, or even long-form content for websites. The model can generate high-quality, SEO-optimized content based on minimal input, ensuring that businesses maintain a regular flow of fresh content without the need for a human writer. The AI can also adapt to different tones (informal, professional, technical) based on audience requirements.</p><p><strong>Example Workflow</strong>:</p><ol><li><strong>Content Briefing</strong>: Provide a brief describing the topic and desired tone.</li><li><strong>AI Content Generation</strong>: Gemini generates an entire article, providing structured headings and subheadings.</li><li><strong>SEO Optimization</strong>: Utilize AI to suggest keywords and improve SEO.</li></ol><pre># Generating a blog post based on a brief<br>prompt = &quot;Write a blog post about the latest trends in AI for 2025. The tone should be professional.&quot;<br>response = model.generate_content([prompt])<br>print(response.text)</pre><h4>4. Multimodal Video Analysis for Marketing Insights</h4><p>With Gemini 2.0 Flash’s multimodal capabilities, businesses can extract insights from videos such as product reviews, tutorials, and webinars. By analyzing video content, Gemini can generate summaries, highlight key points, and even provide sentiment analysis, helping businesses gain insights into audience reactions, product performance, and engagement levels.</p><p><strong>Example Workflow</strong>:</p><ol><li><strong>Video Processing</strong>: Feed the marketing video into the model.</li><li><strong>Insight Extraction</strong>: Gemini analyzes the content and summarizes key takeaways, emotions, and audience engagement.</li></ol><pre># Analyzing video content for sentiment and key insights<br>video = Part.from_uri(&quot;gs://my-bucket/marketing-video.mp4&quot;, mime_type=&quot;video/mp4&quot;)<br>prompt = &quot;Summarize the key messages and audience sentiment in the video.&quot;<br>response = model.generate_content([video, prompt])<br>print(response.text)</pre><h4>5. AI-Driven Product Recommendations</h4><p>Integrating Gemini 2.0 Flash into an e-commerce platform can enhance product recommendations by analyzing customer reviews, purchase history, and even real-time browsing behavior. The model can then generate dynamic product descriptions or recommend related products in a conversational manner.</p><p><strong>Example Workflow</strong>:</p><ol><li><strong>Customer Behavior Data</strong>: Collect the user’s recent interactions (search queries, purchase history).</li><li><strong>Personalized Recommendations</strong>: Based on this data, Gemini generates a list of recommended products along with tailored descriptions.</li></ol><pre># Product recommendation based on customer&#39;s recent browsing history<br>user_data = {&quot;recently_viewed&quot;: [&quot;hiking shoes&quot;, &quot;waterproof jackets&quot;]}<br>prompt = &quot;Based on the user&#39;s recent browsing, recommend three related products with brief descriptions.&quot;<br>response = model.generate_content([user_data, prompt])<br>print(response.text)</pre><h3>Advanced Techniques for Fine-Tuning the Model</h3><p>While Gemini 2.0 Flash comes pre-trained, you can fine-tune it for specific domains by feeding it domain-specific datasets. This allows the model to better understand jargon, terminology, and niche topics, making it more useful for specialized industries.</p><h4>1. Domain-Specific Fine-Tuning</h4><p>Fine-tuning allows you to adapt the model to understand specific lexicons, technical terms, and tone of a particular field such as legal, medical, or technical industries. By leveraging fine-tuning techniques, businesses can create a highly specialized AI assistant that is optimized for their needs.</p><h4>2. Customizing the Generation Process</h4><p>You can refine the output of Gemini by customizing parameters like creativity level (temperature), randomness (top_p), and response length (max_tokens). These settings allow you to adjust the style, coherence, and detail of the generated text, ensuring that you get the right output for every use case.</p><pre>response = model.generate_content(&quot;Generate a short story about a detective solving a mystery.&quot;, temperature=0.7, max_tokens=150)<br>print(response.text)</pre><h4>3. Real-Time Adaptation and User Feedback Loops</h4><p>By using feedback loops, businesses can continually improve their AI systems. By collecting user feedback on generated content (whether through thumbs up/down or sentiment analysis of customer responses), businesses can fine-tune prompts to guide the model toward even more accurate results.</p><pre># User feedback loop to refine responses<br>feedback = &quot;User rated the product recommendation as highly relevant.&quot;<br>prompt = f&quot;Refine the product recommendation based on the feedback: {feedback}&quot;<br>response = model.generate_content([prompt])<br>print(response.text)</pre><h3>Conclusion</h3><p>In conclusion, mastering generative AI with Google Vertex AI Studio and Gemini 2.0 offers powerful capabilities for businesses and developers looking to harness AI for a variety of applications. The key to leveraging these tools effectively lies in well-crafted prompt design, which enables the models to produce relevant and accurate outputs. The iterative and test-driven approach to prompt engineering, combined with advanced techniques like few-shot prompting and chain-of-thought reasoning, ensures high-quality responses across tasks such as text generation, classification, and multimodal applications.</p><p>Vertex AI Studio simplifies the process by offering a no-code environment for rapid prototyping and model tuning, making it easier to test, refine, and deploy AI models. The integration of Gemini 2.0 Flash enhances this process by providing a unified, multimodal platform that can handle text, images, audio, and video inputs, delivering a wide range of outputs with low latency and high throughput. This flexibility supports numerous real-world applications, from personalized marketing and customer support to automated content creation and product recommendations.</p><p>With advanced features like configurable thinking budgets, safety filters, and seamless Python integration, Vertex AI and Gemini 2.0 Flash are poised to empower businesses to automate tasks, gain valuable insights, and enhance user experiences. By fine-tuning models for specific domains and customizing the generation process, organizations can tailor the AI’s behavior to suit their unique needs, ensuring that generative AI continues to drive innovation across industries.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=792f9dc4e204" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Companies Using Linux: An In-Depth Exploration of Benefits and Adoption]]></title>
            <link>https://medium.com/@18bhavyasharma/companies-using-linux-an-in-depth-exploration-of-benefits-and-adoption-d02c545afbff?source=rss-7496da9f1e68------2</link>
            <guid isPermaLink="false">https://medium.com/p/d02c545afbff</guid>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[scalability]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[bhavya sharma]]></dc:creator>
            <pubDate>Sat, 02 Nov 2024 11:52:19 GMT</pubDate>
            <atom:updated>2024-11-02T11:52:19.381Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FdGp9770wbfy9AQ_DK1Tng.jpeg" /></figure><p>Linux has become a cornerstone operating system for a diverse array of companies, from small startups to global enterprises. This open-source platform is renowned for its flexibility, security, and cost-effectiveness. This article delves into specific companies that leverage Linux, the reasons behind their choice, and the extensive benefits they enjoy from adopting this operating system.</p><h4>Prominent Companies Using Linux</h4><ol><li><strong>Google</strong></li></ol><ul><li><strong>Usage</strong>: Google has integrated Linux into its infrastructure, utilizing it for services like Google Cloud and Android. They have developed their own custom distribution, which forms the backbone of many of their products.</li><li><strong>Benefit</strong>: The customization allows Google to optimize performance and enhance security protocols, crucial for managing their vast data processing needs.</li></ul><p><strong>2. Facebook</strong></p><ul><li><strong>Usage</strong>: Facebook runs its operations on a version of Linux specifically tailored for its needs. The company utilizes its distribution to manage the enormous volumes of data and traffic it handles daily.</li><li><strong>Benefit</strong>: The use of Linux contributes to Facebook’s ability to innovate quickly, allowing for rapid deployment of new features and enhancements while ensuring reliability.</li></ul><p><strong>3. IBM</strong></p><ul><li><strong>Usage</strong>: IBM has long championed Linux, particularly in its cloud offerings and enterprise solutions. The company has invested heavily in the development and support of Linux, providing various distributions like Red Hat Enterprise Linux to its clients.</li><li><strong>Benefit</strong>: This commitment to Linux allows IBM to offer flexible and scalable solutions to enterprises, along with strong support for businesses migrating to open-source platforms.</li></ul><p><strong>4. Red Hat</strong></p><ul><li><strong>Usage</strong>: As a leading provider of open-source software solutions, Red Hat uses its own Linux distribution — Red Hat Enterprise Linux (RHEL) — internally and offers it to clients worldwide.</li><li><strong>Benefit</strong>: Red Hat benefits from its strong community support, continuous development, and the ability to offer customers robust solutions with extensive documentation and assistance.</li></ul><p><strong>5. NASA</strong></p><ul><li><strong>Usage</strong>: NASA employs Linux for a range of applications, including data processing for the Mars rovers and other missions where reliability and real-time processing are critical.</li><li><strong>Benefit</strong>: The stability and performance of Linux ensure that NASA can operate in the high-stakes environment of space exploration, where failures can have significant consequences.</li></ul><p><strong>6. Amazon</strong></p><ul><li><strong>Usage</strong>: Amazon Web Services (AWS) relies heavily on Linux across its cloud services, offering customers various Linux-based instances for their applications.</li><li><strong>Benefit</strong>: Linux’s scalability and performance capabilities allow AWS to provide a reliable cloud platform, supporting millions of businesses and applications globally.</li></ul><h4>Reasons for Choosing Linux</h4><ol><li><strong>Cost-Effectiveness</strong></li></ol><ul><li><strong>No Licensing Fees</strong>: Unlike proprietary operating systems, Linux is free to use. This allows companies to allocate their budgets more efficiently, focusing on development and innovation rather than licensing costs.</li><li><strong>Reduced Hardware Costs</strong>: Linux can often run on older hardware, allowing companies to extend the life of their existing systems.</li></ul><p><strong>2. Flexibility and Customization</strong></p><ul><li><strong>Tailored Solutions</strong>: Companies can modify Linux distributions to meet their specific operational needs. This flexibility extends to everything from user interfaces to security configurations.</li><li><strong>Variety of Distributions</strong>: With numerous distributions available (like Ubuntu, CentOS, Fedora), companies can choose one that best fits their needs.</li></ul><p><strong>3. Security</strong></p><ul><li><strong>Open-Source Advantage</strong>: The open-source nature of Linux means that vulnerabilities can be identified and addressed quickly by the community, resulting in enhanced security.</li><li><strong>Regular Updates</strong>: Many distributions offer frequent updates and patches, ensuring that systems are protected against emerging threats.</li></ul><p><strong>4. Stability and Performance</strong></p><ul><li><strong>Uptime Reliability</strong>: Linux is known for its stability, often running for years without needing a reboot. This is vital for businesses that require constant uptime.</li><li><strong>Efficient Resource Management</strong>: Linux can effectively manage resources, enabling it to handle demanding applications and workloads efficiently.</li></ul><p><strong>5. Community Support</strong></p><ul><li><strong>Extensive Resources</strong>: The large and active Linux community provides a wealth of resources, documentation, and forums where users can seek help and share solutions.</li><li><strong>Collaborative Development</strong>: Companies can contribute to the codebase, allowing them to shape the direction of the software they use.</li></ul><p><strong>6. Compatibility with Cloud Technologies</strong></p><ul><li><strong>Cloud Readiness</strong>: As more businesses migrate to cloud services, Linux’s compatibility with cloud infrastructure makes it an ideal choice for modern IT environments.</li><li><strong>Support for Containerization</strong>: Technologies like Docker and Kubernetes, which are widely used for managing applications in the cloud, are optimized for Linux.</li></ul><h4>Benefits Gained from Using Linux</h4><ol><li><strong>Enhanced Security</strong></li></ol><ul><li><strong>Fewer Vulnerabilities</strong>: Organizations that use Linux experience lower rates of malware infections and security breaches due to its strong security model.</li><li><strong>Role-Based Access Control</strong>: Linux provides robust user and permission management, ensuring that sensitive data is accessible only to authorized users.</li></ul><p><strong>2. Lower Total Cost of Ownership</strong></p><ul><li><strong>Cost Savings</strong>: With no licensing fees and reduced hardware costs, companies can achieve a lower total cost of ownership, which is crucial for startups and budget-conscious enterprises.</li><li><strong>Sustainable IT</strong>: The longevity of Linux systems reduces the need for frequent upgrades, further lowering costs.</li></ul><p><strong>3. Greater Control</strong></p><ul><li><strong>System Customization</strong>: Companies have full control over their operating system, allowing for custom configurations that align with specific business processes.</li><li><strong>Independence from Vendors</strong>: Using Linux reduces dependence on proprietary vendors, enabling companies to choose their hardware and software solutions freely.</li></ul><p><strong>4. Longer Software Lifecycles</strong></p><ul><li><strong>Support for Legacy Systems</strong>: Many Linux distributions offer long-term support (LTS) versions that ensure ongoing updates and security patches for extended periods, allowing companies to maintain stable environments.</li><li><strong>Ease of Migration</strong>: When businesses decide to upgrade their systems, Linux offers easier migration paths compared to some proprietary systems.</li></ul><p><strong>5. Innovation and Development</strong></p><ul><li><strong>Rapid Prototyping</strong>: Companies can quickly develop and deploy applications on Linux, allowing for faster innovation cycles.</li><li><strong>Access to Cutting-Edge Technologies</strong>: With a vibrant community, organizations can experiment with the latest advancements in technology, including artificial intelligence and big data solutions.</li></ul><p><strong>6. Scalability</strong></p><ul><li><strong>Handling Growth</strong>: Linux can scale seamlessly from small applications on individual machines to large enterprise-level solutions across multiple servers, making it suitable for organizations of all sizes.</li><li><strong>Load Balancing</strong>: Advanced features like load balancing ensure optimal performance, even under high traffic conditions.</li></ul><h4>Conclusion</h4><p>The widespread adoption of Linux by companies across various industries highlights its significance as a reliable, secure, and cost-effective operating system. Organizations like Google, Facebook, IBM, and NASA demonstrate that Linux can meet the rigorous demands of modern business operations while providing the flexibility and control that companies need to innovate and grow. As technology continues to evolve, Linux will likely remain a pivotal player in the landscape of operating systems, empowering businesses to thrive in an increasingly digital world.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d02c545afbff" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting Up Alerts for CPU Usage with Prometheus and Grafana]]></title>
            <link>https://medium.com/@18bhavyasharma/setting-up-alerts-for-cpu-usage-with-prometheus-and-grafana-40623e7a60b3?source=rss-7496da9f1e68------2</link>
            <guid isPermaLink="false">https://medium.com/p/40623e7a60b3</guid>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[cpu-usage]]></category>
            <category><![CDATA[alerting]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[prometheus]]></category>
            <dc:creator><![CDATA[bhavya sharma]]></dc:creator>
            <pubDate>Sat, 02 Nov 2024 10:58:27 GMT</pubDate>
            <atom:updated>2024-11-02T10:58:27.000Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*391P1RnonzXAxYuDmIgyhg.png" /></figure><p>In monitoring systems, staying informed about resource utilization like CPU usage is essential for maintaining optimal performance. This guide walks you through configuring Prometheus to alert you when CPU usage exceeds specific thresholds, such as 80%, and then setting up Grafana to display these alerts in a dashboard. The steps covered include creating alert rules in Prometheus, integrating Alertmanager for notifications, and setting up Grafana to visualize active alerts.</p><h3>Overview of Tools</h3><ul><li><strong>Prometheus</strong>: An open-source monitoring and alerting toolkit used for gathering metrics and setting alerts based on threshold conditions.</li><li><strong>Alertmanager</strong>: A companion tool for Prometheus that manages alert notifications, sending them to specified channels like email, Slack, or PagerDuty.</li><li><strong>Grafana</strong>: A visualization and analytics platform for monitoring, which can display Prometheus alerts in custom dashboards.</li></ul><h3>Prerequisites</h3><ol><li><strong>Prometheus</strong> installed and running on a server, collecting metrics from the node_exporter for CPU, memory, and other system metrics.</li><li><strong>Grafana</strong> installed and connected to Prometheus as a data source.</li><li>(Optional) <strong>Alertmanager</strong> for routing alerts to various notification channels.</li></ol><h3>Step 1: Configure Prometheus to Monitor CPU Usage</h3><p>Prometheus collects data from the node_exporter on system metrics, including CPU usage. We&#39;ll define an alert rule to detect high CPU usage.</p><ol><li><strong>Create a Prometheus Alert Rule File</strong>:</li></ol><ul><li>In your Prometheus configuration directory, create a new file called alert.rules.yml (or another preferred name).</li></ul><p><strong>2. Define an Alert Rule</strong>:</p><ul><li>This rule will set an alert if CPU usage is above 80% over a continuous 5-minute period.</li><li>Add the following YAML configuration to alert.rules.yml:</li></ul><pre>groups:<br>  - name: CPU_Usage_Alerts<br>    rules:<br>      - alert: HighCPUUsage<br>        expr: 100 * (1 - avg by(instance) (rate(node_cpu_seconds_total{mode=&quot;idle&quot;}[5m]))) &gt; 80<br>        for: 5m<br>        labels:<br>          severity: critical<br>        annotations:<br>          summary: &quot;High CPU usage detected on {{ $labels.instance }}&quot;<br>          description: &quot;CPU usage on instance {{ $labels.instance }} has been above 80% for the last 5 minutes.&quot;</pre><p><strong>Explanation</strong>:</p><ul><li>expr: The expression calculates CPU usage by measuring the idle time (mode=&quot;idle&quot;) of CPU seconds, subtracting it from 100%. The threshold is set to trigger if usage exceeds 80%.</li><li>for: The alert will only fire if the CPU usage remains above the threshold for 5 minutes, to avoid temporary spikes causing unnecessary alerts.</li><li>labels: severity is set to critical, which can help with routing and prioritization in Alertmanager.</li><li>annotations: Additional information fields for the alert to provide context in notifications and dashboards.</li></ul><p><strong>3. Update Prometheus Configuration</strong>:</p><ul><li>Edit prometheus.yml, Prometheus&#39;s main configuration file, to include the rule file:</li></ul><pre>rule_files:   - &quot;alert.rules.yml&quot;</pre><p><strong>4. Restart Prometheus</strong> to load the new alert rule:</p><pre>sudo systemctl restart prometheus</pre><h3>Step 2: Setting Up Alertmanager (Optional but Recommended)</h3><p>Prometheus can detect alerts, but it doesn’t send notifications directly. Alertmanager handles alert routing to specified channels, allowing you to receive alerts via email, Slack, PagerDuty, etc.</p><ol><li><strong>Configure Alertmanager</strong>:</li></ol><ul><li>Locate and open the alertmanager.yml file, usually in the Alertmanager installation directory.</li><li>Define receivers for different alert channels. Here’s an example for configuring email notifications.</li></ul><pre>route:<br>  receiver: email_receiver<br>  group_wait: 10s<br>  group_interval: 5m<br>  repeat_interval: 1h<br><br>receivers:<br>  - name: email_receiver<br>    email_configs:<br>      - to: &quot;your_email@example.com&quot;<br>        from: &quot;alertmanager@example.com&quot;<br>        smarthost: &quot;smtp.example.com:587&quot;<br>        auth_username: &quot;your_smtp_user&quot;<br>        auth_identity: &quot;alertmanager@example.com&quot;<br>        auth_password: &quot;your_smtp_password&quot;</pre><p><strong>Explanation</strong>:</p><ul><li>route: Configures how alerts are grouped, delayed, and repeated.</li><li>receivers: Defines the notification channel, in this case, an email with SMTP configuration.</li></ul><p><strong>2. Link Prometheus to Alertmanager</strong>:</p><ul><li>In the Prometheus configuration file (prometheus.yml), add the Alertmanager URL:</li></ul><pre>alerting:<br>  alertmanagers:<br>    - static_configs:<br>        - targets:<br>            - &#39;localhost:9093&#39;  # Replace with Alertmanager&#39;s actual host and port if different</pre><p><strong>3. Restart Alertmanager and Prometheus</strong>:</p><ul><li>Restart Alertmanager and Prometheus to apply configurations:</li></ul><pre>sudo systemctl restart alertmanager<br>sudo systemctl restart prometheus</pre><p>Now, when an alert is triggered, Alertmanager will send it to the specified email address.</p><h3>Step 3: Displaying Alerts in Grafana</h3><p>Grafana allows you to visualize active alerts from Prometheus, providing a clear dashboard for system monitoring.</p><ol><li><strong>Create a New Panel for Alerts in Grafana</strong>:</li></ol><ul><li>Log in to Grafana and open the dashboard where you want to add the alert panel.</li><li>Add a new panel, selecting <strong>Prometheus</strong> as the data source.</li></ul><p><strong>2. Query for Active Alerts</strong>:</p><ul><li>In the panel’s <strong>Query</strong> section, use the following query to capture active alerts from Prometheus:</li></ul><pre>ALERTS{alertstate=&quot;firing&quot;}</pre><p>This query filters for only active alerts (alertstate=”firing”) in Prometheus.</p><p><strong>3. Configure the Visualization</strong>:</p><ul><li>Choose a visualization type suited for alerts. <strong>Table</strong> or <strong>Alert List</strong> panels work well for displaying alert information.</li><li>Customize the table to display relevant fields like alertname, severity, instance, and description from the annotations defined in Prometheus.</li></ul><p><strong>4. Set Refresh Intervals</strong>:</p><ul><li>Configure the panel refresh rate to update every few seconds or minutes, depending on your monitoring needs.</li></ul><p><strong>5. Save the Dashboard</strong>:</p><ul><li>Once you’ve configured and styled the panel, save your dashboard.</li></ul><h3>Step 4: Testing the Alert Setup</h3><p>To ensure everything is configured correctly, simulate high CPU usage or temporarily adjust the alert threshold in alert.rules.yml to a lower level, such as 10%. You can then check:</p><ol><li><strong>Prometheus</strong>: Look at the <strong>Alerts</strong> page in Prometheus to see if the alert is firing.</li><li><strong>Alertmanager</strong>: Confirm that notifications are routed to the configured email or other channels.</li><li><strong>Grafana</strong>: Check the alerts panel in Grafana to see if the active alert appears.</li></ol><p>After verifying that the alerts work as expected, remember to revert any temporary changes to thresholds or rules.</p><h3>Conclusion</h3><p>Setting up CPU usage alerts with Prometheus and displaying them in Grafana provides a powerful monitoring solution. This setup ensures you’ll be informed of any performance issues that could impact application stability, and by using Grafana, you can continuously monitor your systems from a single dashboard. As your monitoring needs evolve, you can extend these configurations to alert on other system metrics, customizing them to suit the specific needs of your environment.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=40623e7a60b3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Step-by-Step Guide to Creating a Grafana Dashboard for System Monitoring with Prometheus]]></title>
            <link>https://medium.com/@18bhavyasharma/a-step-by-step-guide-to-creating-a-grafana-dashboard-for-system-monitoring-with-prometheus-5f9735f480d5?source=rss-7496da9f1e68------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f9735f480d5</guid>
            <category><![CDATA[metrics]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[prometheus]]></category>
            <dc:creator><![CDATA[bhavya sharma]]></dc:creator>
            <pubDate>Sat, 02 Nov 2024 10:48:26 GMT</pubDate>
            <atom:updated>2024-11-02T10:48:26.810Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*391P1RnonzXAxYuDmIgyhg.png" /></figure><p>Grafana is a powerful open-source tool used for data visualization and monitoring. It connects seamlessly with Prometheus, a leading open-source monitoring solution, to pull and display system metrics. In this guide, we’ll go through installing Grafana, configuring Prometheus as a data source, and creating a dashboard to monitor CPU and memory metrics in real-time.</p><h3>Step 1: Install Grafana</h3><p>Grafana can be installed on various operating systems. For this guide, we’ll focus on installation for Ubuntu/Debian systems.</p><h4>1. Add the Grafana Repository</h4><p>Open a terminal and add the Grafana repository to your system:</p><pre>sudo apt-get update<br>sudo apt-get install -y software-properties-common<br>sudo add-apt-repository &quot;deb https://packages.grafana.com&quot; #add latest grafana repo link</pre><h4>2. Install Grafana</h4><p>Once the repository is added, install Grafana with:</p><pre>sudo apt-get update<br>sudo apt-get install grafana</pre><h4>3. Start and Enable the Grafana Service</h4><p>After installation, start Grafana and set it to run on startup:</p><pre>sudo systemctl start grafana-server<br>sudo systemctl enable grafana-server</pre><h4>4. Access Grafana</h4><p>To access Grafana, open a web browser and go to http://localhost:3000. You’ll be prompted to log in.</p><ul><li><strong>Default Username</strong>: admin</li><li><strong>Default Password</strong>: admin</li></ul><p>After logging in, you’ll be prompted to change the default password for security.</p><h3>Step 2: Configure Prometheus as a Data Source in Grafana</h3><p>To visualize data collected by Prometheus, we need to configure it as a data source in Grafana.</p><h4>1. Add Prometheus Data Source</h4><ol><li>In the Grafana interface, go to the <strong>Configuration</strong> menu and select <strong>Data Sources</strong>.</li><li>Click <strong>Add data source</strong>, then select <strong>Prometheus</strong> from the list.</li></ol><h4>2. Configure Prometheus Connection</h4><ol><li>Set the <strong>URL</strong> field to http://localhost:9090, which is the default address where Prometheus runs.</li><li>Click <strong>Save &amp; Test</strong> to check if Grafana can successfully connect to Prometheus. A green confirmation message should appear if the connection is successful.</li></ol><p>At this point, Grafana is ready to query Prometheus and pull metric data.</p><h3>Step 3: Create a Grafana Dashboard</h3><p>With Prometheus set up as a data source, you can now create a Grafana dashboard to visualize CPU and memory usage.</p><h4>1. Creating a New Dashboard</h4><ol><li>Go to <strong>Dashboards</strong> in the Grafana sidebar.</li><li>Click <strong>New Dashboard</strong> and select <strong>Add New Panel</strong>.</li></ol><p>This will bring you to the panel editor, where you can configure individual panels to display metrics from Prometheus.</p><h4>2. Setting up CPU Usage Monitoring</h4><p>To monitor CPU usage, follow these steps:</p><ol><li>In the <strong>Query</strong> section of the panel editor, select <strong>Prometheus</strong> as the data source.</li><li>Use the following query to pull CPU usage data:</li></ol><pre>rate(node_cpu_seconds_total{mode=&quot;user&quot;}[5m])</pre><p>This query calculates the average CPU usage over the last five minutes.</p><p>in the <strong>Visualization</strong> tab, set the panel type to <strong>Time Series</strong> for real-time updates.</p><p><strong>3. Adjust Panel Settings</strong>:</p><ul><li>In the <strong>Axes</strong> section, set the <strong>Unit</strong> to <strong>Percent</strong> to make the data easier to interpret.</li><li>Customize the color scheme or set thresholds to help visually indicate different levels of CPU load.</li></ul><p>4. Click <strong>Apply</strong> to add this panel to the dashboard.</p><h4>3. Setting up Memory Usage Monitoring</h4><p>To track memory usage, add another panel:</p><ol><li>Click <strong>Add Panel</strong> in the dashboard view.</li><li>Use the following Prometheus query to get memory usage data:</li></ol><pre>(node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes) / node_memory_MemTotal_bytes * 100</pre><p>This query calculates the percentage of memory being used.</p><p>In the <strong>Axes</strong> section, set the <strong>Unit</strong> to <strong>Percent</strong> for consistency.</p><p>3. Customize other visualization options to fit your preferences and click <strong>Apply</strong>.</p><p>With these panels set up, you’ll have real-time CPU and memory metrics on your Grafana dashboard.</p><h3>Step 4: Customizing the Dashboard</h3><p>You can further customize the dashboard layout to enhance readability and add more data. Below are some customization ideas:</p><h4>1. Organizing Panels with Rows</h4><p>Use rows to group related panels. For example, place CPU and memory metrics in separate rows if you plan to add more panels, such as network or disk usage metrics.</p><ol><li>Click <strong>Settings</strong> (gear icon) on the dashboard and select <strong>Add Row</strong>.</li><li>Move the panels into appropriate rows.</li></ol><h4>2. Setting Alerts</h4><p>Grafana allows you to set up alerts for any panel. This feature can notify you when a metric exceeds a predefined threshold.</p><ol><li>In the panel editor, go to the <strong>Alert</strong> tab.</li><li>Click <strong>Create Alert</strong> and define conditions. For example, you could set a condition to trigger an alert when CPU usage exceeds 80%.</li><li>Set the alert notification method, like email or Slack, under <strong>Alerting &gt; Notification Channels</strong>.</li></ol><h4>3. Applying Custom Themes</h4><p>Grafana offers various themes and display options. You can apply a dark or light theme to the entire dashboard or configure individual panels with unique color schemes.</p><ol><li>Open <strong>Settings</strong> (gear icon) in the dashboard view.</li><li>Choose the theme under <strong>Appearance</strong>.</li></ol><h4>4. Adding Additional Panels</h4><p>To enhance your dashboard, consider adding panels for other system metrics, such as:</p><ul><li><strong>Disk Usage:</strong></li></ul><pre>node_filesystem_size_bytes{fstype!=&quot;tmpfs&quot;} - node_filesystem_free_bytes{fstype!=&quot;tmpfs&quot;}</pre><ul><li><strong>Network Traffic</strong>:</li></ul><pre>rate(node_network_receive_bytes_total[5m])</pre><p>Add these panels using the same steps as for the CPU and memory panels, adjusting queries and visualization options as needed.</p><h3>6. Conclusion</h3><p>Setting up a Grafana dashboard with Prometheus allows you to visualize system metrics and monitor performance at a glance. With the flexibility to customize the dashboard layout, add alerts, and expand with more metrics, this setup provides a powerful monitoring solution for any infrastructure.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f9735f480d5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Set Up Prometheus to Collect Metrics from a Node.js App]]></title>
            <link>https://medium.com/@18bhavyasharma/how-to-set-up-prometheus-to-collect-metrics-from-a-node-js-app-a77d0d16fb32?source=rss-7496da9f1e68------2</link>
            <guid isPermaLink="false">https://medium.com/p/a77d0d16fb32</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[prometheus]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[bhavya sharma]]></dc:creator>
            <pubDate>Sat, 02 Nov 2024 10:31:57 GMT</pubDate>
            <atom:updated>2024-11-02T10:31:57.621Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*FOYgWIymj6auPzOeqZE48g.png" /></figure><p>Prometheus is a powerful open-source monitoring solution, ideal for tracking and alerting based on real-time metrics. This guide will walk you through the process of installing Prometheus, configuring it to monitor a Node.js application, and verifying that metrics are being collected. We’ll also include practical code examples and step-by-step explanations to ensure a seamless setup.</p><h3>Overview of the Steps</h3><ol><li>Install Prometheus on your server.</li><li>Set up a basic Node.js application to expose metrics.</li><li>Configure Prometheus to scrape metrics from the Node.js application.</li><li>Verify that Prometheus is collecting metrics correctly.</li></ol><h3>Step 1: Installing Prometheus</h3><p>Prometheus is straightforward to install on Linux, Mac, or Windows. Here, we’ll focus on the installation for Linux, but the process is similar for other operating systems.</p><ol><li><strong>Download Prometheus</strong></li></ol><p>Visit the Prometheus download page and find the latest stable version for your operating system.</p><p><strong>2. Extract the Tarball</strong></p><p>Once the download completes, extract the files:</p><pre>tar -xvzf prometheus-VERSION.linux-amd64.tar.gz<br>cd prometheus-VERSION.linux-amd64</pre><p><strong>3. Verify the Installation</strong></p><p>Before starting Prometheus, check that the binary runs as expected:</p><pre>./prometheus --version</pre><p>You should see the version details, confirming Prometheus is ready to run.</p><p><strong>. Start Prometheus</strong></p><p>Run Prometheus with its default configuration to verify it launches correctly. This will serve as a baseline before customizing any configurations.</p><pre>./prometheus --config.file=prometheus.yml</pre><p>Prometheus will start on port 9090 by default. You can access it at <a href="http://localhost:9090.">http://localhost:9090.</a></p><h3>Step 2: Set Up the Node.js Application with Prometheus Metrics</h3><p>To have Prometheus monitor a Node.js application, the app needs to expose an endpoint that provides metrics in a format Prometheus understands. We’ll use the prom-client library to handle this.</p><ol><li><strong>Initialize the Node.js Application</strong></li></ol><p>If you haven’t already, initialize a new Node.js project and install the prom-client package.</p><pre>mkdir nodejs-app<br>cd nodejs-app<br>npm init -y<br>npm install express prom-client</pre><p><strong>2. Create the Metrics Endpoint</strong></p><p>Inside your Node.js project, create an index.js file. Add the following code to set up an Express server and configure it to expose metrics at the /metrics endpoint.</p><pre>const express = require(&#39;express&#39;);<br>const client = require(&#39;prom-client&#39;);<br><br>const app = express();<br>const register = new client.Registry();<br><br>// Collect default metrics (CPU, memory usage, etc.)<br>client.collectDefaultMetrics({ register });<br><br>// Create a histogram metric for tracking request durations<br>const httpRequestDurationMicroseconds = new client.Histogram({<br>    name: &#39;http_request_duration_seconds&#39;,<br>    help: &#39;Duration of HTTP requests in seconds&#39;,<br>    labelNames: [&#39;method&#39;, &#39;route&#39;, &#39;status_code&#39;],<br>    registers: [register],<br>});<br><br>// Middleware to record request durations<br>app.use((req, res, next) =&gt; {<br>    const end = httpRequestDurationMicroseconds.startTimer();<br>    res.on(&#39;finish&#39;, () =&gt; {<br>        end({ method: req.method, route: req.route ? req.route.path : req.path, status_code: res.statusCode });<br>    });<br>    next();<br>});<br><br>// Metrics endpoint for Prometheus to scrape<br>app.get(&#39;/metrics&#39;, async (req, res) =&gt; {<br>    res.set(&#39;Content-Type&#39;, register.contentType);<br>    res.end(await register.metrics());<br>});<br><br>// Example route to demonstrate metrics tracking<br>app.get(&#39;/&#39;, (req, res) =&gt; {<br>    res.send(&#39;Hello, Prometheus!&#39;);<br>});<br><br>// Start the application on port 3000<br>app.listen(3000, () =&gt; {<br>    console.log(&#39;Node.js app listening on http://localhost:3000&#39;);<br>});</pre><p>This setup exposes basic metrics such as memory and CPU usage, along with a histogram to measure HTTP request durations for each route. The /metrics endpoint will now serve metrics in a format Prometheus can scrape.</p><p><strong>3. Run the Node.js Application</strong></p><p>Start the Node.js application:</p><pre>node index.js</pre><p>The application should now be accessible at <a href="http://localhost:3000,">http://localhost:3000,</a> and the /metrics endpoint will expose the metrics.</p><h3>Step 3: Configure Prometheus to Scrape the Node.js Application</h3><p>Now that your Node.js application is running and exposing metrics, configure Prometheus to scrape these metrics.</p><ol><li><strong>Open the prometheus.yml</strong> <strong>Configuration File</strong></li></ol><p>Locate the prometheus.yml file in your Prometheus installation directory. Open it in a text editor and add a new scrape configuration under scrape_configs.</p><p><strong>2. Add the Node.js Application as a Scrape Target</strong></p><p>Below is a sample prometheus.yml configuration that instructs Prometheus to scrape metrics from the Node.js application every 15 seconds.</p><pre>global:<br>  scrape_interval: 15s<br><br>scrape_configs:<br>  - job_name: &#39;nodejs_app&#39;<br>    scrape_interval: 5s<br>    static_configs:<br>      - targets: [&#39;localhost:3000&#39;]</pre><ul><li>job_name identifies the scrape target.</li><li>scrape_interval sets the frequency with which Prometheus will scrape the metrics (5 seconds here).</li><li>targets specifies the address where Prometheus can find the Node.js application’s metrics endpoint.</li></ul><p><strong>3. Restart Prometheus</strong></p><p>For changes to take effect, restart Prometheus:</p><pre>./prometheus --config.file=prometheus.yml</pre><h3>Step 4: Verify Metrics Collection</h3><p>With both Prometheus and the Node.js application running, we can now verify that metrics are being collected successfully.</p><ol><li><strong>Access the Prometheus Dashboard</strong></li></ol><p>Open your browser and navigate to http://localhost:9090. This brings up the Prometheus web interface.</p><p><strong>2. Check Targets</strong></p><p>In the Prometheus UI, go to <strong>Status</strong> &gt; <strong>Targets</strong>. Here, you should see an entry for nodejs_app with the target localhost:3000. The status should be “UP,” indicating Prometheus can access the metrics endpoint.</p><p>If the status is “DOWN,” double-check that the Node.js application is running and the prometheus.yml configuration is correct.</p><p><strong>3. Query Metrics</strong></p><p>To confirm that metrics are being collected, go to the <strong>Graph</strong> tab in the Prometheus UI. In the query bar, type the name of a metric, such as http_request_duration_seconds_count, and click <strong>Execute</strong>.</p><p>You should see a graph displaying the collected metrics over time. The histogram metric, for example, will show counts and durations of HTTP requests, broken down by route and status code.</p><h3>Additional Customization</h3><p>With Prometheus collecting basic metrics, you can expand your setup:</p><ul><li><strong>Create Additional Metrics</strong>: Use the prom-client library to add custom metrics like counters, gauges, and more histograms to track specific application behaviors.</li><li><strong>Set Up Alerts</strong>: Add alerting rules to the prometheus.yml file to receive notifications when certain thresholds are reached.</li><li><strong>Visualize with Grafana</strong>: Use Grafana to create dashboards that visualize the metrics, making them easier to monitor.</li></ul><h3>Conclusion</h3><p>Setting up Prometheus to monitor a Node.js application provides you with valuable insights into application performance and reliability. This setup is adaptable and can be scaled to monitor multiple applications and services across a system. With additional tools like Grafana, you can enhance the visualization and alerting capabilities, making Prometheus a comprehensive monitoring solution for any development stack.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a77d0d16fb32" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>