<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lize Raes on Medium]]></title>
        <description><![CDATA[Stories by Lize Raes on Medium]]></description>
        <link>https://medium.com/@lize.raes?source=rss-5db85223188d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*NlyIOA-uGmr2H8vGwzenOA.png</url>
            <title>Stories by Lize Raes on Medium</title>
            <link>https://medium.com/@lize.raes?source=rss-5db85223188d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 09:04:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@lize.raes/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Docker Offload: Local AI Without the Laptop Meltdown]]></title>
            <link>https://medium.com/@lize.raes/docker-offload-local-ai-without-the-laptop-meltdown-634b1f6ea8b3?source=rss-5db85223188d------2</link>
            <guid isPermaLink="false">https://medium.com/p/634b1f6ea8b3</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[gpu]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Lize Raes]]></dc:creator>
            <pubDate>Thu, 10 Jul 2025 10:26:11 GMT</pubDate>
            <atom:updated>2025-07-10T11:10:31.402Z</atom:updated>
            <content:encoded><![CDATA[<p>Local AI models are great. They come with absolute data privacy guarantees required for some industries and are cheap to run. But they may also be frying your laptop and result in low quality responses, issues with tool calling, etc. Docker just launched Docker Offload, a frictionless way to spin up local models in a secure cloud environment at very affordable prices.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/1*4gN0UfiGnb2Crkik6qB_uQ.png" /></figure><p>Docker Offload lets you spin up containers and models in the cloud instead of on your machine. You keep your Docker workflow. You keep your Compose files. But you offload the compute, including to GPUs, for as little as $0.015/min. This allows you to run bigger open source models (better quality) while staying privacy-compliant and without the need to invest $100K in hardware, or run compute-heavy ML pipelines at a very affordable price.</p><h3>How it works under the hood</h3><ul><li>Once you toggle Docker Desktop mode, your containers run in the cloud, no longer on your device.</li><li>A secure SSH tunnel connects your Docker Desktop to the cloud daemon.</li><li>You pay for container uptime, not per inference call like with API services.</li><li>If you enable GPU acceleration, you get access to an NVIDIA L4.</li></ul><h3>Getting Started</h3><p>You need <strong>Docker Desktop 4.43</strong> or newer.<br>And <a href="https://www.docker.com/products/docker-offload/">sign up</a> for Docker Offload.</p><p>You can activate or toggle Offload mode, either via the Docker Desktop UI or via terminal:</p><h3>In Docker Desktop UI</h3><p>Go to Settings &gt; Beta Features &gt; Enable Docker Offload</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NvrQQwc6uVhR5nqf" /></figure><p>Then to activate Offload (paying!), toggle the slider in the top bar from</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/884/0*c0qm63Odeu8fm1yn" /></figure><p>to</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/0*IOMFSy1287Gab42r" /></figure><h3>From CLI</h3><pre>docker offload start</pre><p>You’ll see a welcome screen asking you to:</p><ul><li>Pick your account (if you have more, this is the account that the credits will be counted toward)</li><li>Choose whether you need GPU support (NVIDIA L4 available)</li></ul><h3>Stopping Docker Offload</h3><p>To see current status (basically: on or off):</p><pre>docker offload status</pre><p>You can see the actual containers you have running in the UI, as usual.</p><p>To stop Docker Offload (will stop all your containers) and stop the paying minutes:</p><pre>docker offload stop</pre><p>This is the status you’re looking for when stopped</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PTHjpqNUbnEL85fI" /></figure><p>You cloud service stays up (and keeps billing) until you stop Offload mode or until it shuts down automatically after ~30 minutes of inactivity (no API calls, no port activity).</p><h3>Costs (Beta Pricing)</h3><ul><li>You get 300 minutes free, afterwards</li><li>$0.015/minute with GPU</li><li>$0.01/minute without GPU</li></ul><p>⚠️ Note: Time starts counting as soon as Offload is enabled, not just when you run a model.</p><h3>Hello World</h3><p>To check everything’s working:</p><pre>docker run - rm hello-world</pre><p>If Offload is enabled, you’ll see “Hello from Docker!” printed, but from a container running entirely in the cloud.</p><p>If you like, check your remote GPU details as follows</p><pre>docker run - rm - gpus all nvidia/cuda:12.4.0-runtime-ubuntu22.04 nvidia-smi</pre><p>The output should look like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1eNU0SToMm6HBpp30sCEMw.png" /></figure><h3>Run a TensorFlow Notebook</h3><p>Now let’s do something useful and very cool: run a TensorFlow GPU-enabled Jupyter Notebook:</p><p>The command below will automatically pull the container in the cloud and run it there. For this application, best make sure that you enable GPUs when starting Docker Offload.</p><pre>docker run - rm -p 8888:8888 - gpus all tensorflow/tensorflow:latest-gpu-jupyter</pre><p>Somewhere in the output you should see something like</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wjwNf_hXNEM6OaI5" /></figure><p>Open the second URL (starting with 127.0.0.1), and you’re ready to start training ML models, in the cloud, on a GPU. Here I launched a pre-configured clothing classifier, have fun!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vhQECX3rb5_v3i2Y" /></figure><h3>Offload and AI Models</h3><p>Here’s where it gets extra cool for AI workflows. Docker Offload ties right into Docker’s Model Runner and new Agentic Compose integration.</p><p>Agentic Compose deserves an article on its own, but here is a sneak peek if you want to get an idea of how smooth it becomes to spin up models and run them locally or in the cloud via the usual `compose.yml` as part of a bigger app.</p><pre>services:<br>  chat-app:<br>    image: my-chat-app<br>    models:<br>      llm:<br>        endpoint_var: AI_MODEL_URL<br>        model_var: AI_MODEL_NAME<br><br>models:<br>  llm:<br>    model: ai/smollm2<br>    x-cloud-options:<br>      - &quot;cloud.instance-type=gpu-small&quot;<br>      - &quot;cloud.region=us-west-2&quot;</pre><p>A few quick ending notes:</p><ul><li>Offload is either on or off. You can’t mix local and cloud containers simultaneously.</li><li>Container images must still be uploaded to the cloud before they can run there, so large models may mean longer upload times, counting to your cloud usage.</li><li>You pay for container runtime, even if you’re not actively sending inference requests.</li></ul><p>But in return, your laptop gets a break, and you unlock the power to run bigger models, heavier ML pipelines, and modern agentic apps, all without big hardware investments or building custom cloud infrastructure.</p><p><strong>Docker Offload opens up new possibilities for developers who want privacy, scalability, and serious AI capabilities, in a very frictionless way.</strong></p><p>Curious to see what you’ll build with it!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=634b1f6ea8b3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Local AI in Java: LangChain4j with Docker Model Runner]]></title>
            <link>https://medium.com/@lize.raes/local-ai-in-java-langchain4j-with-docker-model-runner-7d73a57db660?source=rss-5db85223188d------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d73a57db660</guid>
            <category><![CDATA[ai-development]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[langchain4j]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Lize Raes]]></dc:creator>
            <pubDate>Mon, 14 Apr 2025 18:27:26 GMT</pubDate>
            <atom:updated>2025-04-14T18:27:26.994Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Kickstart your AI-powered Java apps with Docker Model Runner</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*itrVE7EKPXO_CJzF.png" /></figure><p>You love Java? You’re already using Docker? And you’re exploring AI? Then this tutorial is for you!</p><p>Docker just launched Docker Model Runner: a model runtime engine that ships with Docker Desktop and model weights that you can pull straight from Docker Hub, just like common images. Think DeepSeek, Gemma3, Phi4, …</p><p>As LangChain4j collaborator and Docker Captain, I had to make the match 🔥 Running language models locally has never been easier, GPU-accelerated, OpenAI-compatible, and fully wired into your existing developer setup. And yes, <strong>it works great with LangChain4j</strong>.</p><h3>🚀 Why is this cool</h3><p>Docker just turned running LLMs into something as easy as docker model run. You don’t need to understand model architecture or inference backends, you just:</p><pre>docker model pull ai/gemma3<br>docker model run ai/gemma3</pre><p>That’s it. It’s the kind of developer experience we’ve been missing in the AI tooling world. Fast, familiar, and frictionless. Especially for teams who already use Docker in their stack. They’re curating and hosting popular models like Gemma, LLaMA, and DeepSeek directly on <a href="http://hub.docker.com/u/ai">Docker Hub</a>, so you don’t have to source weights or worry about sketchy downloads.</p><p>It’s fast. It’s GPU-accelerated. And it plugs directly into your <strong>Java app</strong>.</p><h3>🧪 Try it in under 5 minutes</h3><p><em>Note: At the moment only available on Mac (Windows will follow within a month or so).</em></p><p>First, let’s get your Docker Desktop ready for the Model Runner. Download the latest version (4.40+ required). In settings, scroll down to <strong>Features in Development</strong>, and check</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*aSDp_Ljv31Ot60wD" /></figure><p>The first check enables the Model Runner in CLI, the second one makes sure we can access it with LangChain4j. Click <strong>Apply and Restart</strong>.</p><p>List the available commands and check if the installation was successful by opening a terminal</p><pre>docker model command</pre><p>Now let’s pull and run our first model:</p><pre>docker model pull ai/gemma3:latest<br>docker model run ai/gemma3</pre><p>To use this model directly from Java, I put together a minimal, no-frameworks chatbot with memory using <strong>LangChain4j</strong>: 👉<a href="https://github.com/LizeRaes/docker-model-runner-langchain4j"> github.com/LizeRaes/docker-model-runner-langchain4j</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/0*HMZTFdXfSKSVfTpL.png" /></figure><p>Follow the instructions in the readme, and there you go, you’re now chatting with a local model via Java.</p><p>The magic happens in ChatServer.java:</p><pre>ChatLanguageModel model = OpenAiChatModel.builder()<br>  .apiKey(&quot;not needed&quot;)<br>  .baseUrl(&quot;http://localhost:12434/engines/llama.cpp/v1&quot;)<br>  .modelName(&quot;ai/gemma3&quot;)<br>  .build();</pre><h3>🧠 What models are available?</h3><p>Browse<a href="https://hub.docker.com/u/ai"> Docker’s AI Hub </a>to see what’s already there. More models are added regularly.</p><p>To check file size and resources, just open a model page on Docker Hub — you’ll see the size, architecture, and version. For example, for Gemma3</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-LTy6dEjOFPHvPfD" /></figure><p>Once you have the model running you can check your GPU usage via (MacOS)</p><p>Activity Monitor → Window → GPU History</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jYwtl42j8zrjwSBk" /></figure><h3>🔭 What’s next?</h3><p>Docker’s R&amp;D team proposed a very ambitious roadmap, with things like Windows support, a GUI and Hub uploads that will become available in the next months 🔥</p><p>They already offer containerized MCP servers and are on their way to make all aspects of AI model inference a breeze.</p><p>Your feedback is super valuable for making everything work smoothly, so if you encounter a glitch or miss an essential feature, let the Docker team know via</p><pre>docker feedback</pre><p>Curious to know what you’re gonna build with it!</p><p><strong><em>Note: This article was republished to address visibility and SEO indexing issues affecting the first version.</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d73a57db660" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LangChain4j with Docker Model Runner]]></title>
            <link>https://medium.com/@lize.raes/langchain4j-%EF%B8%8F-docker-model-runner-b5f720a76c85?source=rss-5db85223188d------2</link>
            <guid isPermaLink="false">https://medium.com/p/b5f720a76c85</guid>
            <category><![CDATA[ai-development]]></category>
            <category><![CDATA[langchain4j]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Lize Raes]]></dc:creator>
            <pubDate>Tue, 01 Apr 2025 06:48:13 GMT</pubDate>
            <atom:updated>2025-04-14T18:24:13.664Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Kickstart your AI-powered Java apps with Docker Model Runner</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-vElKeA3qx4jrEyNtUID3g.png" /></figure><p>You love Java? You’re already using Docker? And you’re exploring AI? Then this tutorial is for you!</p><p>Docker recently launched Docker Model Runner: a model runtime engine that ships with Docker Desktop and model weights that you can pull straight from Docker Hub, just like common images. Think DeepSeek, Gemma3, Phi4, …</p><p>As LangChain4j collaborator and Docker Captain, I had to make the match 🔥 Running language models locally has never been easier, GPU-accelerated, OpenAI-compatible, and fully wired into your existing developer setup. And yes, <strong>it works great with LangChain4j</strong>.</p><h3>🚀 Why is this cool</h3><p>Docker just turned running LLMs into something as easy as docker model run. You don’t need to understand model architecture or inference backends, you just:</p><pre>docker model pull ai/gemma3<br>docker model run ai/gemma3</pre><p>That’s it. It’s the kind of developer experience we’ve been missing in the AI tooling world. Fast, familiar, and frictionless. Especially for teams who already use Docker in their stack. They’re curating and hosting popular models like Gemma, LLaMA, and DeepSeek directly on <a href="http://hub.docker.com/u/ai">Docker Hub</a>, so you don’t have to source weights or worry about sketchy downloads.</p><p>It’s fast. It’s GPU-accelerated. And it plugs directly into your <strong>Java app</strong>.</p><h3>🧪 Try it in under 5 minutes</h3><p><em>Note: At the moment only available on Mac (Windows will follow within a month or so).</em></p><p>First, let’s get your Docker Desktop ready for the Model Runner. Download the latest version (4.40+ required). In settings, scroll down to <strong>Features in Development</strong>, and check</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j6rWk7n4VzIoZOlE" /></figure><p>The first check enables the Model Runner in CLI, the second one makes sure we can access it with LangChain4j. Click <strong>Apply and Restart</strong>.</p><p>List the available commands and check if the installation was successful by opening a terminal</p><pre>docker model command</pre><p>Now let’s pull and run our first model:</p><pre>docker model pull ai/gemma3:latest<br>docker model run ai/gemma3</pre><p>To use this model directly from Java, I put together a minimal, no-frameworks chatbot with memory using <strong>LangChain4j</strong>: 👉<a href="https://github.com/LizeRaes/docker-model-runner-langchain4j"> github.com/LizeRaes/docker-model-runner-langchain4j</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/1*c4tGsQZONXg_GA-BcqSxTg.png" /></figure><p>Follow the instructions in the readme, and there you go, you’re now chatting with a local model via Java.</p><p>The magic happens in ChatServer.java:</p><pre>ChatLanguageModel model = OpenAiChatModel.builder()<br>  .apiKey(&quot;not needed&quot;)<br>  .baseUrl(&quot;http://localhost:12434/engines/llama.cpp/v1&quot;)<br>  .modelName(&quot;ai/gemma3&quot;)<br>  .build();</pre><h3>🧠 What models are available?</h3><p>Browse<a href="https://hub.docker.com/u/ai"> Docker’s AI Hub </a>to see what’s already there. More models are added regularly.</p><p>To check file size and resources, just open a model page on Docker Hub — you’ll see the size, architecture, and version. For example, for Gemma3</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rvPg5FNJmcOLOQYF" /></figure><p>Once you have the model running you can check your GPU usage via (MacOS)</p><p>Activity Monitor → Window → GPU History</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*z3Lq8X_h2v25G_BN" /></figure><h3>🔭 What’s next?</h3><p>Docker’s R&amp;D team proposed a very ambitious roadmap, with things like Windows support, a GUI and Hub uploads that will become available in the next months 🔥</p><p>They already offer containerized MCP servers and are on their way to make all aspects of AI model inference a breeze.</p><p>Your feedback is super valuable for making everything work smoothly, so if you encounter a glitch or miss an essential feature, let the Docker team know via</p><pre>docker feedback</pre><p>Curious to know what you’re gonna build with it!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b5f720a76c85" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Step-by-Step: Let your LLM access Google Drive in Java Using MCP and LangChain4j]]></title>
            <link>https://medium.com/@lize.raes/step-by-step-let-your-llm-access-google-drive-in-java-using-mcp-and-langchain4j-7c97c30f0854?source=rss-5db85223188d------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c97c30f0854</guid>
            <category><![CDATA[google-drive-api]]></category>
            <category><![CDATA[langchain4j]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[model-context-protocol]]></category>
            <category><![CDATA[llm-agent]]></category>
            <dc:creator><![CDATA[Lize Raes]]></dc:creator>
            <pubDate>Wed, 15 Jan 2025 20:28:07 GMT</pubDate>
            <atom:updated>2025-01-16T15:21:33.183Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’re anything like me, your brain has ultimately degraded over a decade of having electronic devices around that allow you to take notes at any time. You cannot remember anything longer than 1 minute and are now using Google Docs as your mental hard drive.</p><p>But ‘remembering’ anything from Google Docs is a pain: taking out your phone, opening the app, finding a file, and then reading it, just to remember to take an in-laws taser on that next family trip.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LiQvN3eijMPyq2tvnB4r8A.png" /></figure><p>As true lazy engineers, we’ll solve this problem by writing a little program that lets an LLM access our Google Drive so it can search and summarise our notes, travel plans, and half-written blog posts.</p><p>The result will look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YP6jZfVCe4I4cIWl" /><figcaption>Note that I didn’t need to specify the filename, and the file in question contains much more than this list. Great job LLM! (one can tell I’m a back end developer <em>🙈)</em></figcaption></figure><p>Let’s get this set up in under 20 minutes! (side note: at the moment of writing, the used Google Drive MCP server only allows reading operations, no editing… yet! When editing is added, you can add it in under a minute, just pull the latest version to have the newest functions added, that’s the beauty of MCPs 🙂)</p><h3><strong>How does it work?</strong></h3><p><a href="https://github.com/langchain4j/langchain4j/"><strong>LangChain4j</strong></a> is a Java library that makes connecting to LLMs really easy so you don’t need to construct and parse those API messages yourself. It supports all the main players (OpenAI, Gemini, Anthropic, … ) and also local models, see <a href="https://docs.langchain4j.dev/integrations/language-models/">list of supported models</a>.</p><p><strong>Model Context Protocol (MCP)</strong> is an open protocol to make tools (and resources) available in LLM-readable format. Everyone can write MCP servers, in any programming language, and open source them for everyone else to use. Amazing ♥️</p><p>At the time of writing, over 100 open source MCP servers can be found on github, for services like Notion, Google Calendar, Minecraft, git, … I suppose many more will follow in the coming months. If you’re tempted to write your own, do have a look at <a href="https://quarkus.io/blog/mcp-server/">this tutorial</a> to write a weather forecast MCP yourself.</p><p>This schema illustrates why MCPs are so cool:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6_lWrGWreMZ2J6cE" /><figcaption><em>Before, you had to study the API of every separate service, get a tool to invoke it, and parse the answer afterwards. So many tools and syntaxes to struggle with! 🤯</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XkVevrhhebsXN9DB" /><figcaption><em>MCP servers translate the different APIs to one LLM-readable format, and take care of the invocations. They are language-agnostic and truly plug-and-play.</em></figcaption></figure><p>LangChain4j has an integration to use MCPs as ‘Tools’, you can now plug in any MCP server in just 3 lines of code. Yuppie 🥳 Let’s go!</p><h3><strong>Prerequisites</strong></h3><p>You’ll need:</p><ul><li><strong>Java 17 or higher</strong> If your version is insufficient, download java 22 SDK and import it into your IDE.</li><li><strong>npm</strong> Verify if you have it by running npm -v in your terminal/cmd. To install: brew install npm (mac) or choco install npm (windows -&gt; get chocolatey if you don’t have it, it’s the best thing ever)</li><li><strong>An API Key for the model provider you choose</strong> I’m using OpenAI in this example and have set the <strong><em>OPENAI_API_KEY </em></strong>in the environment variables (and restart the IDE!). See <a href="https://www3.ntu.edu.sg/home/ehchua/programming/howto/Environment_Variables.html">here</a> how to set environment variables, <a href="https://platform.openai.com/api-keys">here</a> to get an OpenAI key.</li><li><strong>Google API credentials </strong>This is the longest step in the process 😩 But don’t get discouraged, you can reuse these credentials for all Google Services 👍. You’ll need to:</li><li><a href="https://developers.google.com/workspace/guides/create-project">Create a Google Cloud Project</a></li><li>Enable the Google Drive API here (‘ENABLE APIS AND SERVICES’)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4RK94My5mCT0DTRT" /></figure><ul><li><a href="https://developers.google.com/workspace/guides/create-credentials#oauth-client-id">Get OAuth credentials</a> here (‘CREATE CREDENTIALS’). You can keep user type ‘external’, add yourself as test user, and choose Desktop App.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-tJ6DktFdHiHHPXi" /></figure><ul><li>Alternatively, do ‘+ADD USERS’. You can select the scope for these credentials. For our app we’ll need reading rights for Google Drive.</li><li>Once that’s done, download the credentials file next to your OAuth 2.0 Client ID user (a file ending in .keys.json), save it somewhere and store the absolute path to this file (incl. the filename at the end) as environment variable <strong><em>GOOGLE_APPLICATION_CREDENTIALS</em></strong>.</li><li>Also store an environment variable <strong><em>MCP_GDRIVE_CREDENTIALS</em></strong> with the path where the app should store the authentication file (any folder you like). For me the end result looks like this:</li></ul><pre>export GOOGLE_APPLICATION_CREDENTIALS=/path/to/creds_gdrive/gcp-oauth.keys.json<br>export MCP_GDRIVE_CREDENTIALS=/path/to/creds_gdrive/.gdrive-server-credentials.json</pre><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*Zxsgedw6EvmRy67BhZuZuA.png" /><figcaption>Congrats on your nerves if you made it this far!</figcaption></figure><p>Now let’s get some code! Check out the MCP server repo here: <a href="https://github.com/felores/gdrive-mcp-server">https://github.com/felores/gdrive-mcp-server</a></p><p>And run npm installin the project root folder. This should create a dist folder with index.js inside. Save the absolute path to this index.js file, we’ll need it later.</p><p>Then check out the LangChain4j examples repo <a href="https://github.com/langchain4j/langchain4j-examples">https://github.com/langchain4j/langchain4j-examples</a> and open mcp-examples/src/main/java/dev/langchain4j/example/mcp/McpToolsExampleOverStdio.java</p><p>This is the code we’re going to run. Almost there!</p><p><strong>The code</strong></p><p>In McpToolsExampleOverStudio.java we connect to OpenAI GPT-4o mini. (UncommentlogRequests and logResponses if you want to see what is really sent to theOpenAI endpoint).</p><pre>  ChatLanguageModel model = OpenAiChatModel.builder()<br>                .apiKey(System.getenv(&quot;OPENAI_API_KEY&quot;))<br>                .modelName(&quot;gpt-4o-mini&quot;)<br>//                .logRequests(true)<br>//                .logResponses(true)<br>                .build();</pre><p>You can swap OpenAI with any supported model provider, but make sure to add the correct dependency to pom.xml, as described in the <a href="https://docs.langchain4j.dev/category/language-models">model integration documentation</a>.</p><p>LangChain4j lets you run a published version of the MCP server, via</p><pre>McpTransport transport = new StdioMcpTransport.Builder()<br>                .command(List.of(&quot;/opt/homebrew/bin/npm&quot;, &quot;exec&quot;,<br>                        &quot;@modelcontextprotocol/server-filesystem@0.6.2&quot;,<br>                        new File(&quot;./..&quot;).getAbsolutePath()<br>                ))<br>                .logEvents(true)<br>                .build();</pre><p>But there are a few reasons to build and invocate your local clone (the other repo we loaded before):</p><ul><li>Not all MCPs have a published version</li><li>It’s all quite new, bugfixes are frequent, so for the latest version, you’ll need the source code anyway</li><li>The published versions tend to clash with Google credentials (for now)</li></ul><p>So exchange the McpTransport line with this:</p><pre>McpTransport transport = new StdioMcpTransport.Builder()<br>                // run with &quot;auth&quot; only the first time, log in when asked, ignore reported errors in terminal<br>                //.command(List.of(&quot;/usr/bin/env&quot;, &quot;node&quot;, &quot;/path/to/gdrive-mcp-server/dist/index.js&quot;, &quot;auth&quot;))<br>                .command(List.of(&quot;/usr/bin/env&quot;, &quot;node&quot;, &quot;/path/to/gdrive-mcp-server/dist/index.js&quot;))<br>                .logEvents(true)<br>                .environment(Map.of(<br>                        &quot;MCP_GDRIVE_CREDENTIALS&quot;, System.getenv(&quot;MCP_GDRIVE_CREDENTIALS&quot;),<br>                        &quot;GOOGLE_APPLICATION_CREDENTIALS&quot;, System.getenv(&quot;GOOGLE_APPLICATION_CREDENTIALS&quot;)))<br>                .build();</pre><p>Then change /path/to/gdrive-mcp-server/dist/index.js to the path you obtained in the previous section, after running npm build.</p><p>Last thing: set ​​String response = bot.chat(“question”);to a question that makes sense for your Google Drive documents (don’t be too ambitious just yet…)</p><p>Now the first time you’ll need to go through the Google authentication process, by uncommenting the ”auth” line</p><pre>.command(List.of(&quot;/usr/bin/env&quot;, &quot;node&quot;, &quot;/Users/lize/Documents/Code/gdrive-mcp-server/dist/index.js&quot;, &quot;auth&quot;))</pre><p>and commenting out the next one. When you run the code now, you should be brought to a login page like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MPmNzXtmzljzv8Gr" /><figcaption><em>Entschuldige mein Deutsch 😂</em></figcaption></figure><p>Walk through it, and say yes to all the unverified-app-asking-for-all-permissions requests 😈</p><p>If you make it till this here, you’re good:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*f4lLFn8GeI89TIpH" /></figure><p>You can ignore the errors in your IDE and stop the process.</p><p>Comment the ”auth” line out again, and uncomment the next one. Now you’re ready to ask questions to your Google Drive ^^ Have fun!</p><p>You’ll soon find out that it doesn’t work well at all. So we’ll tweak our prompt a bit…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*yODHxtagu0byp4k1SmS5Pw.png" /></figure><h3><strong>Let’s make this work properly</strong></h3><p>To make this any useful, you’ll want to set proper instructions and give some context to the LLM. You can do so by setting a system message that is passed to the LLM at the beginning of the conversation. Typically it’s used to steer the behaviour of the model, eg. ‘you are talking to a 3-year-old’. For our app, we can add</p><ul><li>Some static info about you / the user</li><li>Some dynamic info for context (current date, current location, …)</li><li>Some extra info on how to use the MCP tools, if their included description is insufficient (as is the case for our example)</li><li>Some Q&amp;A examples that help the LLM follow your (weird) reasoning</li></ul><p>You can set this info on the AiService, in this case the interface Bot.java in the same folder.<br><strong>AiServices</strong> are a super practical concept in LangChain4j, once you’ve wrapped your head around them, they can even return POJOs! If you’re curious, here are <a href="https://github.com/langchain4j/langchain4j-examples/blob/main/tutorials/src/main/java/_08_AIServiceExamples.java">examples</a> so you get the hang of them.</p><p>To make my Google Drive searcher work better, I prompted it as follows (Bot.java):</p><pre>public interface Bot {<br><br>    @SystemMessage(&quot;&quot;&quot;<br>            User context: the user is {{name}} and you can talk to her like a 3-year-old.<br>            More context: today is {{current_date}}<br>            How to use your tools:<br>            You have tools to search through the user&#39;s Google Drive and read file contents.<br>            If you need to search:<br>            - guess the most probable keyword that would be in the filename<br>            - use one keyword only as search query, and use no quotes<br>            - if a probably relevant file pops up, look inside for the answer<br>            - if no relevant file is found, try a second keyword<br>            Some examples of what to query for:<br>            Q: what is my IntelliJ shortcut for searching through the entire project?<br>            Reasoning: the file is probably called IntelliJ shortcuts or similar<br>            Query 1: IntelliJ<br>            Query 2 (if needed): shortcuts<br>            <br>            Q: who did I sent Xmas cards to last year?<br>            Reasoning: the file is probably called Christmas cards address list or similar<br>            Query 1: address<br>            Query 2 (if needed): Christmas<br>            Query 3 (if needed): Xmas<br>            Query 4 (if needed): cards<br>            <br>            &quot;&quot;&quot;)<br>    String chat(@UserMessage String prompt, @V(&quot;name&quot;) String userName);<br>}</pre><p>Note that I pass the userName as a second argument for the chat function, so I’ll need to pass this argument as well when I call bot.chat().</p><p>Now it should be working much better!</p><h3><strong>To go further</strong></h3><p>A side note: if it is really about searching file content only, you may rather want to vectorise your Google Drive, and use that vector database as input for your RAG system. It will be faster, cheaper and more reliable. Here’s more details about <a href="https://docs.langchain4j.dev/tutorials/rag">RAG</a> and <a href="https://github.com/langchain4j/langchain4j-examples/tree/main/rag-examples/src/main/java">examples</a>.</p><p>I’m waiting for editing and file creating capabilities to make this really awesome. Let me know in the comments if you found an MCP repo that does that AND when you get it’s authentication working with Google credentials, thx ☺️</p><p>Further reading:</p><ul><li>Using MCP servers in Quarkus: <a href="https://quarkus.io/blog/quarkus-langchain4j-mcp/">https://quarkus.io/blog/quarkus-langchain4j-mcp/</a></li><li>Plenty more MCP servers: <a href="https://github.com/modelcontextprotocol/servers">https://github.com/modelcontextrotocol/servers</a> (or Google around for what you need)</li><li>Creating your own MCP server in Quarkus: <a href="https://quarkus.io/blog/mcp-server/">https://quarkus.io/blog/mcp-server/</a></li></ul><p><em>Stay tuned for more articles in the pipeline that risk to never see the light:</em></p><ul><li><em>Connect to Github, Gmail, Calendar, filesystem via MCP</em></li><li><em>Writing our own MCP for filling our supermarket basket (like in: shout orders from the kitchen)</em></li><li><em>Give this stuff a nice frontend (chat, display retrieved contents and permission-to-proceed buttons)</em></li><li><em>Building the audio input side of SiriOnSteroids</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c97c30f0854" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using AI to address United Nation’s Sustainable Development Goals]]></title>
            <link>https://medium.com/@lize.raes/using-ai-to-address-united-nations-sustainable-development-goals-a661794f3f40?source=rss-5db85223188d------2</link>
            <guid isPermaLink="false">https://medium.com/p/a661794f3f40</guid>
            <category><![CDATA[un]]></category>
            <category><![CDATA[common-good]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Lize Raes]]></dc:creator>
            <pubDate>Sun, 04 Feb 2024 18:01:07 GMT</pubDate>
            <atom:updated>2024-02-04T18:01:07.236Z</atom:updated>
            <content:encoded><![CDATA[<p>Since ChatGPT went viral in December 2022, the world has started a huge transformation. Generative Artificial Intelligence is making solutions possible that we could not have dreamed of just a year ago.</p><p>Throughout history, people have faced problems for which they engineered solutions.</p><blockquote>With AI, we are offered solutions for which we are only beginning to discover the problems.</blockquote><p>So, why don’t we align this new super power with the world’s most pressing issues?</p><figure><img alt="Genie out of a magical lamp, bringing solution for medicine, poverty, inclusion, etc." src="https://cdn-images-1.medium.com/max/1011/1*htrXW5BLxmxwQDm2LYz0eQ.png" /></figure><p>The Sustainable Development Goals (SDGs) of the United Nations aim to address 17 of humanity’s most crippling problems, including hunger, climate change, inequality, and war.</p><p>In 2015, all 193 member states of the UN signed a treaty in which they committed to addressing these goals by 2030, with the aim of creating a fairer, more prosperous, and more peaceful society. Yes, that’s right, <strong>virtually all countries in the world have agreed to tackle these challenges</strong>. Here they are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bfzrXhHcUfvX6BUw.jpg" /></figure><p>Might Artificial Intelligence be our hidden savior in making this reality?</p><p>Thousands of data, software, and hardware engineers are tirelessly working to improve AI — making it better, faster, more reliable and more affordable. Nevertheless, reports of concrete and accessible solutions that use this new superpower for addressing global issues, are scarce.</p><p>Instead, businesses are racing to gain more customers by having the latest and greatest AI chat features, AI assistants, and AI-enhanced smartphones. Companies are restructuring their workforces to align with strategic goals, often centered around AI expansion. Countries are racing to catch up with their legal frameworks, while monetary funds and governments are grappling with how to mitigate growing inequalities resulting from automation.</p><p>What if we diverted even just a fraction of these engineers’ efforts towards harnessing AI’s potential to achieve the Sustainable Development Goals?</p><blockquote>Not all solutions need to be large, enterprise-sized endeavors consuming countless man-hours and resources. One can envision a multitude of smaller solutions that can be created by a team of a handful of software developers in less than a week, and still have a tremendous positive impact.</blockquote><p>What about a <strong>citizen feedback collection</strong> tool for municipalities? A simple web tool that cities can offer to their residents, allowing them to share grievances, issues, appreciation, and proposed solutions related to their city or neighborhood whenever they wish. In the background, AI can categorize this feedback, identify the most common issues, and propose solutions to the city council, based on urgency, cost-effectiveness, and potential benefits for citizens.</p><p>What about a basic <strong>education tool</strong> that assists students in any language, with any subject, adapting to their level and focusing on their specific difficulties? Imagine this online tool being globally accessible, potentially transforming developing countries by addressing the need for a skilled workforce, reducing poverty, promoting dignified work, and reducing gender and overall inequalities.</p><figure><img alt="A kind robot helping a child learn his subjects" src="https://cdn-images-1.medium.com/max/679/1*F_eTH_WgwizF-dTIYm6i8A.png" /></figure><p>What about a <strong>subsidy chatbot</strong> integrated into government websites, that directs individuals to the subsidies they are entitled to? Today, subsidies often don’t reach the disadvantaged groups they were originally intended for, because they require a fair amount of digital and administrative skills in order to obtain them. This chatbot would make it easier to get the subsidies where they belong, assisting people in their own language and at their comprehension level.</p><p>Or what about a <strong>preliminary medical diagnosis tool</strong> that assesses the urgency of your ailment and offers guidance, either suggesting a home remedy or recommending a visit to a healthcare professional when necessary? This would make first-line medical guidance more available to disadvantaged groups, while at the same time freeing medical resources for patients that are in real need of a doctor.</p><p>These are just a few possibilities that have become really easy to build. So much more can be achieved! Imagine the impact of these solutions when dedicated teams work on them full-time. The potential return on investment for the world could be incredible.</p><p>If you’re wondering if AI today is up to the challenge for these proposed solutions, let me introduce you to AMIE, a diagnostic tool developed by Google Research. AMIE surpasses real doctors in diagnostic accuracy and even empathy. Yes, empathy! AI doesn’t have bad days and consistently responds respectfully, regardless of the person it interacts with or their gender or race.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4GnlRaQglhVhu1GlXMIUUA.png" /></figure><p>So, why aren’t these transformative tools available to all? Perhaps they already exist, in the sea of tools out there, but they lack the necessary exposure to be picked up by the students, patients, and cities that would benefit from their potential.</p><p>Or, maybe these innovations are yet to be born. In that case, the path forward is clear — we need a collective effort. We need organizations with the vision and resources to hire talented people to write these tools and make them freely available. Or we need to inspire developers to invest their free time in this noble cause. Either way, a first step towards making this vision tangible, is to share it. Every great transformation in history started with an idea, that gained momentum because people spread their enthusiasm for it. Together, let’s try to make this dream reality.</p><p><em>If you’re a developer and looking for a place to start, consider joining us in Paris from March 19th to the 22nd for the </em><a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/cloudnativehacks/"><em>KubeCon+CloudNative Hackathon</em></a><em>, featuring challenges brought to you by the United Nations.</em></p><p><em>Applications open until February 19th.</em></p><figure><img alt="Logo of Cloud Native Hacks" src="https://cdn-images-1.medium.com/max/1024/1*gAL7OY3ihG9qXxyZjmwoVg.png" /></figure><p><strong>One thing is certain: Artificial Intelligence will reshape our world. Let’s share this vision of harnessing its power for the common good, so that this transformation brings a better future for everyone.</strong></p><p><em>Lize Raes is AI Transition Specialist at </em><a href="https://open-tide.com/"><em>Open Tide</em></a><em>, and inspirational speaker in the Open Source Software community. She loves using technology to solve societal issues, and is a member of the CloudNative Hackathon program committee, to address UN challenges using AI.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a661794f3f40" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>