<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yevhen Lebid on Medium]]></title>
        <description><![CDATA[Stories by Yevhen Lebid on Medium]]></description>
        <link>https://medium.com/@ealebed?source=rss-bb66d0ee848b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*eHgyCppVAiu8bprZr9t85w.jpeg</url>
            <title>Stories by Yevhen Lebid on Medium</title>
            <link>https://medium.com/@ealebed?source=rss-bb66d0ee848b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 08:44:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ealebed/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Zero-Trust access: Connecting GKE workloads to AWS resources without static credentials]]></title>
            <link>https://medium.com/@ealebed/zero-trust-access-connecting-gke-workloads-to-aws-resources-without-static-credentials-134994147404?source=rss-bb66d0ee848b------2</link>
            <guid isPermaLink="false">https://medium.com/p/134994147404</guid>
            <category><![CDATA[gke]]></category>
            <category><![CDATA[workload-identity]]></category>
            <category><![CDATA[gcp]]></category>
            <category><![CDATA[oidc]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Yevhen Lebid]]></dc:creator>
            <pubDate>Sun, 27 Oct 2024 20:04:58 GMT</pubDate>
            <atom:updated>2024-10-27T20:04:58.226Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1021/1*eKhLFuYg5kZk76nFrI_qmg.png" /></figure><p>In today’s multi-cloud world, organizations often need to run workloads that span multiple cloud providers. A common scenario is applications running on Google Kubernetes Engine (GKE) that need to access AWS services. While this used to require managing long-lived AWS credentials, modern workload identity federation capabilities offer a more secure approach.</p><h4>The Challenge</h4><p>When deploying applications that need cross-cloud access, several security challenges emerge:</p><ul><li>Managing static AWS credentials in Kubernetes secrets</li><li>Regular rotation of access keys</li><li>Compliance with security best practices</li><li>Risk of credential exposure</li><li>Operational overhead of credential management</li></ul><h4>Google Cloud Workload Identity</h4><p><a href="https://cloud.google.com/iam/docs/workload-identity-federation">Workload Identity</a> is Google Cloud’s recommended authentication method for GKE applications. It creates a secure binding between Kubernetes service accounts and Google Cloud IAM service accounts, allowing pods to automatically authenticate to Google Cloud services without managing secrets.</p><h4>AWS IAM Roles for Service Accounts</h4><p>Similarly, AWS provides <a href="https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html">IAM Roles for Service Accounts</a> functionality. This feature associates IAM roles with Kubernetes service accounts in Amazon EKS clusters, enabling fine-grained AWS permissions for pod containers.</p><h4>Proposed solution</h4><p>We can leverage OpenID Connect (OIDC) federation to bridge these two worlds. Here’s how it works:</p><ul><li>AWS IAM supports OIDC identity providers for role assumption</li><li>Google implements OIDC provider and integrates it tightly with GKE through Workload Identity feature, providing a valid OIDC access token to GKE pod, running under Kubernetes Service Account linked to a Google Cloud Service Account</li><li>AWS Security Token Service (STS) exchanges valid OIDC ID tokens for temporary AWS credentials</li></ul><p>The solution also implements strict TLS enforcement for all communication between the <a href="https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#contacting-the-webhook">Kubernetes API server and the webhook service</a>. To handle certificate generation and management, a dedicated tool (<a href="https://github.com/ealebed/admission-webhook-certificator">admission-webhook-certificator</a>) is used.</p><p>This CLI tool helps to create CertificateSigningRequest (CSR) with a client certificate which is approved by this CSR with CA belonging to the Kubernetes cluster itself, then creates a Kubernetes Secret containing private key and a client certificate. The entire process is automated via a Kubernetes Job.</p><h4>Implementation</h4><p>The solution uses a <a href="https://dev-k8sref-io.web.app/docs/extend/mutatingwebhookconfiguration-v1/">Kubernetes mutating admission webhook</a> called <a href="https://github.com/ealebed/token-injector">token-injector</a> that:</p><ol><li>Injects an init container to generate a valid GCP OIDC ID token</li><li>Adds a sidecar container to handle token refresh</li><li>Configures required AWS environment variables for application container</li></ol><h4>How it works for GKE workload</h4><ol><li>Application pod is created under an annotated Kubernetes service account and labeled with</li></ol><pre>admission.token-injector/enabled: &quot;true&quot;</pre><p>2. The mutating webhook intercepts pod creation</p><p>3. Init container generates the initial OIDC token</p><p>4. Sidecar container handles token refresh before expiration</p><p>5. The AWS SDK automatically requests temporary AWS credentials from the STS service, using:</p><ul><li>AWS_WEB_IDENTITY_TOKEN_FILE - the path to the web identity token file (OIDC ID token)</li><li>AWS_ROLE_ARN - the ARN of the role to assume by Pod containers</li><li>AWS_ROLE_SESSION_NAME - the name applied to this assume-role session</li></ul><p>6. Pod containers can now access AWS services securely</p><h4>Benefits</h4><p>This approach offers several advantages:</p><ul><li><strong>Zero static credentials</strong>: No long-lived AWS access keys to manage</li><li><strong>Automatic rotation</strong>: Temporary credentials are automatically refreshed</li><li><strong>Fine-grained control</strong>: AWS IAM roles can be scoped to specific permissions</li><li><strong>Audit trail</strong>: All access is logged and traceable</li><li><strong>Simplified operations</strong>: Reduced operational overhead for credential management</li></ul><h4>Codebase</h4><p>The complete solution, including HELM-charts and Terraform scripts, is available in GitHub repositories:</p><ul><li><a href="https://github.com/ealebed/admission-webhook-certificator">admission-webhook-certificator</a></li><li><a href="https://github.com/ealebed/token-injector">token-injector</a></li></ul><p><em>I’ve inspired the initial mutating admission webhook code from </em><a href="https://github.com/doitintl/gtoken/tree/master"><em>doitintl/gtoken</em></a><em> repository. Big thanks to Alexei Ledenev!</em></p><h4>Conclusion</h4><p>By leveraging OIDC federation, we can establish secure, automated authentication between GKE workloads and AWS services. This approach eliminates the need for static credentials while providing fine-grained access control and simplified operations.</p><p>The solution aligns with modern security practices and provides a robust foundation for multi-cloud architectures. As organizations continue to adopt multi-cloud strategies, implementing secure cross-cloud authentication becomes increasingly important.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=134994147404" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying a Generative AI application with vector store to Cloud Run]]></title>
            <link>https://medium.com/@ealebed/deploying-a-generative-ai-application-with-vector-store-to-cloud-run-efe2417ce769?source=rss-bb66d0ee848b------2</link>
            <guid isPermaLink="false">https://medium.com/p/efe2417ce769</guid>
            <category><![CDATA[gemini]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[firestore]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[google-cloud-run]]></category>
            <dc:creator><![CDATA[Yevhen Lebid]]></dc:creator>
            <pubDate>Sun, 13 Oct 2024 15:26:33 GMT</pubDate>
            <atom:updated>2024-10-13T15:26:33.425Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*26y0m8-GVGIIyRslqJ701w.png" /></figure><p>In the blog post from <a href="https://medium.com/u/dbebfddcaeee">Suds Kumar</a>, called <a href="https://medium.com/google-cloud/building-a-rag-application-with-vector-search-in-firestore-71da2e6e7e77">“Building a RAG application with vector search in Firestore”</a>, he described a process of creating a Retrieval-Augmented Generation (RAG) system using Firestore as a vector store.</p><p>Today, we’re taking the next step: building and deploying a Generative AI-powered application using Google Cloud Run. This guide will walk through creating an Artifact Registry, containerizing the application with Docker, and deploying it to Cloud Run for a scalable, serverless solution.</p><h4>Overview of the application</h4><p>Our application will:</p><ul><li>Use <strong>Firestore</strong> to search for vector embeddings.</li><li>Employ <strong>Vertex AI</strong> models to generate text responses using the <strong>Gemini model</strong>.</li><li>Be containerized using <strong>Docker</strong> and deployed to <strong>Cloud Run</strong> for serverless scalability.</li><li>Use <strong>Cloud Logging</strong> to capture relevant application logs.</li></ul><h4>Prerequisites</h4><p>Before we begin, ensure you have:</p><ul><li>Billing enabled on your Google Cloud project.</li><li>Firestore and your vector store configured (as per the Kumar’s <a href="https://medium.com/google-cloud/building-a-rag-application-with-vector-search-in-firestore-71da2e6e7e77">post</a>).</li><li><a href="https://cloud.google.com/sdk/docs/install">Google Cloud SDK</a> (version 369.0.0 or later) installed and authenticated to your project.</li></ul><h4>Enable Google APIs</h4><p>To enable the required APIs, you can use the following command:</p><pre>gcloud services enable artifactregistry.googleapis.com run.googleapis.com aiplatform.googleapis.com</pre><h4>Create an Artifact Registry</h4><p>First, let’s create an Artifact Registry repository to store our Docker image. Artifact Registry provides a single location for managing container images and language packages.</p><pre>gcloud artifacts repositories create test-repo \<br>    --repository-format=docker \<br>    --location=us-central1 \<br>    --description=&quot;Docker repository for food-safety image&quot;</pre><p>Authenticate your Docker client:</p><pre>gcloud auth configure-docker us-central1-docker.pkg.dev</pre><h4>Prepare the application code</h4><p>requirements.txt file lists the Python dependencies required by application:</p><pre>flask<br>Jinja2<br>pytest<br>pyyaml<br>google-cloud-aiplatform&gt;=1.59.0<br>google-cloud-logging<br>firebase-admin<br>langchain-google-vertexai&gt;=1.0.7</pre><p>The Dockerfile contains all the commands a user could call on the command line to assemble a container image for application:</p><pre>FROM python:3.9.16<br>WORKDIR /app<br>COPY . .<br>RUN pip install gunicorn==20.1.0<br>RUN pip install -r requirements.txt<br>ENV PORT=8080<br>CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 main:app</pre><p>main.py is the core of application, integrating Firestore, Vertex AI, and Flask.</p><pre>import os<br>import yaml<br>import logging<br>import google.cloud.logging<br>from flask import Flask, render_template, request<br><br>from google.cloud import firestore<br>from google.cloud.firestore_v1.vector import Vector<br>from google.cloud.firestore_v1.base_vector_query import DistanceMeasure<br><br>import vertexai<br>from vertexai.generative_models import (<br>    GenerativeModel,<br>    GenerationConfig,<br>    HarmCategory,<br>    HarmBlockThreshold,<br>    SafetySetting,<br>)<br>from vertexai.language_models import TextEmbeddingInput, TextEmbeddingModel<br>from langchain_google_vertexai import VertexAIEmbeddings<br><br># Configure Cloud Logging<br>logging_client = google.cloud.logging.Client()<br>logging_client.setup_logging()<br>logging.basicConfig(level=logging.INFO)<br><br># Read application variables<br>BOTNAME = &quot;FoodBot&quot;<br>SUBTITLE = &quot;Restaurant safety expert bot&quot;<br><br># Set up the dangerous content safety config to block only high-probability dangerous content<br>safety_config = [<br>    SafetySetting(<br>        category=HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,<br>        threshold=HarmBlockThreshold.BLOCK_ONLY_HIGH,<br>    ),<br>]<br><br>app = Flask(__name__)<br><br># Initializing the Firebase client<br>db = firestore.Client()<br><br># Instantiate a collection reference<br>collection = db.collection(&#39;food-safety&#39;)<br><br># Instantiate an embedding model here<br>embedding_model = VertexAIEmbeddings(model_name=&quot;text-embedding-004&quot;)<br><br># Instantiate a Generative AI model here<br>gen_model = GenerativeModel(&quot;gemini-1.5-pro-001&quot;,<br>    generation_config=GenerationConfig(temperature=0),<br>    safety_settings=safety_config,)<br><br># Function to return relevant context from vector database<br>def search_vector_database(query: str):<br>    # Initialize context as an empty string<br>    context = &quot;&quot;<br><br>    # Step 1: Generate the embedding of the query<br>    query_embedding = embedding_model.embed_query(query)<br><br>    # Step 2: Get the 5 nearest neighbors from collection using Firestore&#39;s find_nearest operation<br>    vector_query = collection.find_nearest(<br>    vector_field=&quot;embedding&quot;,<br>    query_vector=Vector(query_embedding),<br>    distance_measure=DistanceMeasure.EUCLIDEAN,<br>    limit=5,<br>    )<br><br>    # Step 3: Iterate over document snapshots and combine them into a single context string<br>    docs = vector_query.stream()<br>    context = [result.to_dict()[&#39;content&#39;] for result in docs]<br><br>    # Don&#39;t delete this logging statement.<br>    logging.info(<br>        context, extra={&quot;labels&quot;: {&quot;service&quot;: &quot;food-safety-service&quot;, &quot;component&quot;: &quot;context&quot;}}<br>    )<br>    return context<br><br><br># Pass Gemini the context data, generate a response, and return the response text.<br>def ask_gemini(question):<br><br>    # 1. Create a prompt_template with instructions to the model<br>    # to use provided context info to answer the question.<br>    prompt_template = &quot;&quot;&quot;<br>    You are a restaurant safety expert bot. Use the following context to answer the user&#39;s question.<br>    <br>    Context: {context}<br>    <br>    Question: {question}<br>    <br>    Answer as accurately and helpfully as possible.<br>    &quot;&quot;&quot;<br><br>    # 2. Use search_vector_database function to retrieve context relevant to the question.<br>    context = search_vector_database(question)<br><br>    # 3. Format the prompt template with the question &amp; context<br>    prompt = prompt_template.format(context=context, question=question)<br><br>    # 4. Pass the complete prompt template to Gemini and get the text of its response.<br>    response = gen_model.generate_content(<br>        prompt,<br>    ).text<br><br>    return response<br><br><br># The Home page route<br>@app.route(&quot;/&quot;, methods=[&quot;POST&quot;, &quot;GET&quot;])<br>def main():<br><br>    # The user clicked on a link to the Home page<br>    # They haven&#39;t yet submitted the form<br>    if request.method == &quot;GET&quot;:<br>        question = &quot;&quot;<br>        answer = &quot;Hi, I&#39;m FoodBot, what can I do for you?&quot;<br><br>    # The user asked a question and submitted the form<br>    # The request.method would equal &#39;POST&#39;<br>    else:<br>        question = request.form[&quot;input&quot;]<br>        # Do not delete this logging statement.<br>        logging.info(<br>            question,<br>            extra={&quot;labels&quot;: {&quot;service&quot;: &quot;food-safety-service&quot;, &quot;component&quot;: &quot;question&quot;}},<br>        )<br>        <br>        # Ask Gemini to answer the question using the data<br>        # from the database<br>        answer = ask_gemini(question)<br><br>    # Do not delete this logging statement.<br>    logging.info(<br>        answer, extra={&quot;labels&quot;: {&quot;service&quot;: &quot;food-safety-service&quot;, &quot;component&quot;: &quot;answer&quot;}}<br>    )<br>    print(&quot;Answer: &quot; + answer)<br><br>    # Display the home page with the required variables set<br>    config = {<br>        &quot;title&quot;: BOTNAME,<br>        &quot;subtitle&quot;: SUBTITLE,<br>        &quot;botname&quot;: BOTNAME,<br>        &quot;message&quot;: answer,<br>        &quot;input&quot;: question,<br>    }<br><br>    return render_template(&quot;index.html&quot;, config=config)<br><br>if __name__ == &quot;__main__&quot;:<br>    app.run(debug=True, host=&quot;0.0.0.0&quot;, port=int(os.environ.get(&quot;PORT&quot;, 8080)))</pre><h4>Build and push the Docker Image</h4><p>Now, let’s build our Docker image and push it to Artifact Registry:</p><pre>export PROJECT_ID=$(gcloud config get-value project)<br><br>docker build -t us-central1-docker.pkg.dev/$PROJECT_ID/test-repo/food-safety:latest -f Dockerfile .<br>docker push us-central1-docker.pkg.dev/$PROJECT_ID/test-repo/food-safety:latest</pre><h4>Deploy the application to Cloud Run</h4><pre>gcloud run deploy food-safety-service \<br>    --image=us-central1-docker.pkg.dev/$PROJECT_ID/test-repo/food-safety:latest \<br>    --region=us-central1 \<br>    --allow-unauthenticated \<br>    --min-instances=0 \<br>    --max-instances=1</pre><p>Note: the --allow-unauthenticated flag makes the service publicly accessible. For production environments, consider implementing authentication.</p><p>To allow all users to invoke the service:</p><pre>gcloud beta run services add-iam-policy-binding \<br>    --region=us-central1 \<br>    --member=allUsers \<br>    --role=roles/run.invoker \<br>    food-safety-service</pre><h4>Verify the deployment</h4><ul><li>Visit the <a href="https://console.cloud.google.com/run">Cloud Run console</a>.</li><li>Find your deployed service and click on it.</li><li>You’ll see the service URL. Click on it to access your application.</li><li>Test it by making sure it can answer the following question: “What temperature range do Mesophilic Bacteria grow best in?”</li><li>You should receive a response with a temperature range of 50 to 110 degrees Fahrenheit</li></ul><h4>Conclusion</h4><p>We’ve successfully built and deployed a Generative AI application using Firestore, Vertex AI, and Cloud Run. This serverless architecture leverages the power of RAG to provide context-aware responses.</p><p>Remember, while this setup offers great scalability, it’s important to monitor your usage and costs. The current configuration allows for up to one instance, but you can adjust this based on your needs and budget.</p><p>For production use, consider implementing authentication, rate limiting, and other security measures.</p><p>Example codebase for this application is available in <a href="https://github.com/ealebed/gen-ai-cloud-run-example">GitHub repository</a>.</p><p>We’d love to hear about your experience implementing this solution. Share your thoughts in the comments!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=efe2417ce769" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Enhancing LLMs with function calling: A Gemini case study]]></title>
            <link>https://medium.com/@ealebed/enhancing-llms-with-function-calling-a-gemini-case-study-5dd0ce372e28?source=rss-bb66d0ee848b------2</link>
            <guid isPermaLink="false">https://medium.com/p/5dd0ce372e28</guid>
            <category><![CDATA[gemini]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[colab-enterprise]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[vertex-ai]]></category>
            <dc:creator><![CDATA[Yevhen Lebid]]></dc:creator>
            <pubDate>Sat, 28 Sep 2024 17:39:18 GMT</pubDate>
            <atom:updated>2024-09-28T17:39:18.130Z</atom:updated>
            <content:encoded><![CDATA[<h3>Enhancing Large Language Models with function calling: A Gemini case study</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e6PYqG11u4xtZBs5mJvFqQ.png" /></figure><p>Large Language Models (LLMs) like Google’s Gemini have revolutionized natural language processing. However, they can face challenges with tasks requiring precise calculations or complex string manipulations. To address this, we can leverage the <strong>ReAct</strong> (Reasoning and Acting) pattern, enabling LLMs to interact with external systems for tasks they’re not optimized to handle directly.</p><p>This blog post explores how to extend Gemini’s capabilities implementing its function calling feature on Colab Enterprise — a powerful, cloud-based platform where developers can write code and interact with Gemini via the Vertex AI SDK.</p><h4>Introduction to Gemini and Vertex AI</h4><p>Before we dive into the implementation details, let’s briefly introduce the key technologies we’ll be working with.</p><p><strong>Gemini</strong> is Google’s most capable and flexible AI model to date. It’s a large language model (LLM) designed to be multimodal, meaning it can understand and generate text, code, audio, image, and video. In this tutorial, we’ll be using Gemini Pro, which offers a great balance of performance and efficiency for our function calling demonstration.</p><p><strong>Vertex AI</strong> is Google Cloud’s unified machine learning platform. It provides tools and services that enable developers and data scientists to build, deploy, and manage machine learning models at scale. Vertex AI offers a range of features, including:</p><ul><li>AutoML and custom model training</li><li>Model deployment and monitoring</li><li>Feature store for managing ML features</li><li>Workbench for collaborative development</li><li>AI agents for building conversational AI applications</li></ul><p>In our case, we’ll be using Vertex AI to access and interact with the Gemini model through its API.</p><h4><strong>Understanding of function calling</strong></h4><p>Function calling allows Gemini to:</p><ol><li>Identify tasks that are better suited for external processing</li><li>Extract relevant parameters from user input</li><li>Call appropriate functions to perform these tasks</li><li>Incorporate the results into its responses</li></ol><p>This approach combines Gemini’s strong reasoning capabilities with the precision of coded functions, resulting in more accurate and reliable outputs.</p><p>To begin, ensure you have access to Google Cloud and are familiar with Vertex AI.</p><h4>Create Jupyter notebook</h4><ol><li>Open your Google Cloud project.</li><li>In the <strong>Google Cloud Console</strong>, use the top search bar to find and select <strong>Colab Enterprise</strong>.</li><li>Once inside <strong>Colab Enterprise</strong>, create a new notebook.</li></ol><h4>Install required libraries</h4><p>Once your notebook is ready, you’ll need to install the required libraries for interacting with Vertex AI:</p><pre>!pip3 install --upgrade --user google-cloud-aiplatform==1.60.0</pre><h4>Restart the kernel</h4><p>After installing the necessary libraries, it’s important to restart the kernel to ensure the environment can properly access the newly installed packages:</p><pre># Restart kernel after installs so that your environment can access the new packages<br>import IPython<br><br>app = IPython.Application.instance()<br>app.kernel.do_shutdown(True)  # Restarts the kernel</pre><h4>Define project variables and initialize Vertex AI SDK</h4><p>Now you’ll need to define the project-specific variables and initialize the Vertex AI SDK to interact with Google Cloud services. Run the following code to set your <strong>Google Cloud project</strong> and <strong>location</strong> for Vertex AI operations:</p><pre>import vertexai<br><br># Get the project ID from gcloud<br>PROJECT_ID = !gcloud config get-value project<br>PROJECT_ID = PROJECT_ID[0]<br>LOCATION = &quot;us-central1&quot;  # Set your desired location<br><br># Print the project ID to confirm<br>print(PROJECT_ID)<br><br># Initialize the Vertex AI SDK<br>vertexai.init(project=PROJECT_ID, location=LOCATION)</pre><h4>Import required libraries</h4><p>Next, import the necessary libraries for working with Vertex AI generative models and other functionality:</p><pre>import requests<br>from vertexai.generative_models import (<br>    Content,<br>    FunctionDeclaration,<br>    GenerationConfig,<br>    GenerativeModel,<br>    Part,<br>    Tool,<br>)</pre><h4>Define functions and FunctionDeclarations</h4><p>Next, create python functions for string manipulations and declare them that the model can use to process the prompt. Function declarations must be provided in a schema format that’s compatible with the <a href="https://spec.openapis.org/oas/v3.0.3#schema">OpenAPI schema</a>.</p><pre># Function to reverse the order of characters in a string<br>def reverse(input_string: str) -&gt; str:<br>    print(&quot;Calling reverse function&quot;)<br>    return input_string[::-1]<br><br># Function to remove all spaces from a string<br>def remove_white_spaces(input_string: str) -&gt; str:<br>    print(&quot;Calling remove_white_spaces function&quot;)<br>    return input_string.replace(&quot; &quot;, &quot;&quot;)<br><br># Define FunctionDeclarations for the reverse function<br>reverse_function = FunctionDeclaration(<br>    name=&quot;reverse&quot;,<br>    description=&quot;Reverses a string&quot;,<br>    parameters={<br>        &quot;type&quot;: &quot;object&quot;,<br>        &quot;properties&quot;: {<br>            &quot;input_string&quot;: {<br>                &quot;type&quot;: &quot;string&quot;,<br>                &quot;description&quot;: &quot;The string to reverse&quot;<br>            }<br>        }<br>    }<br>)<br><br># Define FunctionDeclarations for the remove_white_spaces function<br>remove_white_spaces_function = FunctionDeclaration(<br>    name=&quot;remove_white_spaces&quot;,<br>    description=&quot;Removes spaces from a string&quot;,<br>    parameters={<br>        &quot;type&quot;: &quot;object&quot;,<br>        &quot;properties&quot;: {<br>            &quot;input_string&quot;: {<br>                &quot;type&quot;: &quot;string&quot;,<br>                &quot;description&quot;: &quot;The string to remove whitespaces&quot;<br>            }<br>        }<br>    }<br>)</pre><h4>Define the tool with the created functions</h4><p>Define a tool that includes both of the functions that we declared earlier:</p><pre># Define the tool with the created functions<br>string_tool = Tool(<br>    function_declarations=[reverse_function, remove_white_spaces_function]<br>)</pre><h4>Initialize the model and start chat</h4><pre># Initialize Gemini model<br>model = GenerativeModel(<br>    model_name=&quot;gemini-1.5-pro-001&quot;,<br>    generation_config=GenerationConfig(temperature=0),<br>    system_instruction=[<br>        &quot;Fulfill the user&#39;s instructions.&quot;,<br>        &quot;If asked to reverse a string or remove whitespace, call the provided functions.&quot;,<br>        &quot;You may call one function after the other if needed.&quot;,<br>        &quot;Repeat the result to the user.&quot;,<br>    ],<br>    tools=[string_tool],<br>)<br><br># Start the chat<br>chat = model.start_chat()</pre><p>In the code where we initialize the Gemini model, you’ll notice this line:</p><pre>generation_config=GenerationConfig(temperature=0)</pre><p>The temperature parameter is a crucial setting that affects the creativity and randomness of the model&#39;s outputs. Here&#39;s what you need to know:</p><ul><li>Temperature is typically set between 0 and 1.</li><li>A temperature of 0 makes the model’s outputs more deterministic and focused.</li><li>Higher temperatures (closer to 1) make the outputs more diverse and creative, but potentially less predictable.</li></ul><h4>Define function to handle model responses</h4><pre>def handle_response(response):<br>    # Check if there&#39;s a function call in the response<br>    if response.candidates[0].function_calls:<br>        function_call = response.candidates[0].function_calls[0]<br>    else:<br>        # If no function call, print the response text<br>        print(response.text)<br>        return<br><br><br>    # If the function call is for the reverse_string function<br>    if function_call.name == &quot;reverse&quot;:<br>        # Extract the input argument for the reverse function<br>        input_string = function_call.args[&quot;input_string&quot;]<br><br>        # Call the reverse function<br>        reversed_string = reverse(input_string)  # assuming reverse_string is defined<br>        <br>        # Send the result back to the chat session<br>        response = chat.send_message(<br>            Part.from_function_response(<br>                name=function_call.name,<br>                response={<br>                    &quot;content&quot;: reversed_string,<br>                },<br>            ),<br>        )<br>        # Handle the response recursively<br>        return handle_response(response)<br><br>    # If the function call is for the remove_white_spaces function<br>    elif function_call.name == &quot;remove_white_spaces&quot;:<br>        # Extract the input argument for the remove_white_spaces function<br>        input_string = function_call.args[&quot;input_string&quot;]<br>        <br>        # Call the remove_white_spaces function<br>        no_space_string = remove_white_spaces(input_string)  # assuming remove_white_spaces is defined<br>        <br>        # Send the result back to the chat session<br>        response = chat.send_message(<br>            Part.from_function_response(<br>                name=function_call.name,<br>                response={<br>                    &quot;content&quot;: no_space_string,<br>                },<br>            ),<br>        )<br>        # Handle the response recursively<br>        return handle_response(response)<br>    <br>    else:<br>        # You shouldn&#39;t end up here<br>        print(function_call)</pre><h4>Test the model to call your functions</h4><p>Now, let’s test our model sending different prompts to it:</p><pre>response = chat.send_message(&quot;Tell me a joke.&quot;)<br>handle_response(response)</pre><pre>Why don&#39;t scientists trust atoms? <br><br>Because they make up everything!</pre><pre>response = chat.send_message(&quot;Reverse the following string: Hello world&quot;)<br>handle_response(response)</pre><pre>Calling reverse function<br>dlrow olleH</pre><pre>response = chat.send_message(&quot;Remove the white spaces from the following word: hello @ world. com&quot;)<br>handle_response(response)</pre><pre>Calling remove_white_spaces function<br>hello@world.com</pre><pre>response = chat.send_message(&quot;Reverse the following string and remove spaces: I do not understand the new sentence.&quot;)<br>handle_response(response)</pre><pre>Calling reverse function<br>Calling remove_white_spaces function<br>.ecnetneswenehtdnatsrednutonodI</pre><pre>response = chat.send_message(&quot;Tell me a fact. Then remove the spaces in the fact text and then reverse the fact text.&quot;)<br>print(response.candidates[0].content.parts[0].text)<br>handle_response(response)</pre><pre>A group of owls is called a parliament.<br><br>Calling remove_white_spaces function<br>Calling reverse function<br>.tnemailrapadellacsislwofopuorgA</pre><h4>Conclusion</h4><p>In this blog post, we’ve explored how to enhance Gemini’s capabilities using function calling within the Colab Enterprise environment. By leveraging the Vertex AI SDK, we’ve demonstrated how to seamlessly integrate custom Python functions with Gemini’s powerful language understanding abilities.</p><p>For further exploration, consider diving into more advanced use cases, optimizing your function implementations, and exploring Gemini’s other capabilities through the Vertex AI platform. Always refer to the latest Google Cloud documentation for best practices and updates on working with Gemini and Vertex AI.</p><p>Full Jupiter notebook example is available in <a href="https://github.com/ealebed/Vertex_AI_notebooks/blob/master/Enhancing%20LLMs%20with%20Function%20Calling%20-%20A%20Gemini%20Case%20Study.ipynb">GitHub repository</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5dd0ce372e28" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>