<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Taneli Leppä on Medium]]></title>
        <description><![CDATA[Stories by Taneli Leppä on Medium]]></description>
        <link>https://medium.com/@taneli-leppa?source=rss-6c5d20f5d1ce------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*tVsDpXfQZf6L0XRDH5jZDg.jpeg</url>
            <title>Stories by Taneli Leppä on Medium</title>
            <link>https://medium.com/@taneli-leppa?source=rss-6c5d20f5d1ce------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 16:51:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@taneli-leppa/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Controlling Spotify via Home Assistant]]></title>
            <link>https://taneli-leppa.medium.com/controlling-spotify-via-home-assistant-e14fb7abd109?source=rss-6c5d20f5d1ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/e14fb7abd109</guid>
            <category><![CDATA[home-assistant]]></category>
            <category><![CDATA[spotify]]></category>
            <category><![CDATA[zigbee]]></category>
            <category><![CDATA[powershell]]></category>
            <dc:creator><![CDATA[Taneli Leppä]]></dc:creator>
            <pubDate>Sat, 12 Oct 2024 11:23:22 GMT</pubDate>
            <atom:updated>2024-10-12T14:00:57.352Z</atom:updated>
            <content:encoded><![CDATA[<p>You might have seen my previous article on turning <a href="https://taneli-leppa.medium.com/spaceballs-the-switch-zigbeeing-a-device-fd2f5d3ee907">a switch into a Zigbee device (Spaceballs! — The Switch)</a>. Concord Aerospace recently released a new switch called <a href="https://www.concordaerospace.com/products/fortunate-son-activation-panel">Fortunate Son</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A0As4KlnJ2SdhXd_k921sA.jpeg" /><figcaption>The switch in question.</figcaption></figure><p>I wanted to make it also functional and to do, well, what is says on the box. Since I’m operating via Zigbee and paired with Home Assistant, I wanted a Home Assistant automation to be able to trigger actions based on this on my PC and control Spotify (and possibly some other things in the future as well).</p><p>You can achieve this in two different ways, use the <a href="https://www.home-assistant.io/integrations/spotify/">Home Assistant Spotify integration</a> or use <a href="https://github.com/LAB02-Research/HASS.Agent">HASS agent</a> to run arbitrary commands on your PC:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*g0B5WWejUEzudBzbeMNbXg.png" /><figcaption>HASS agent installed on Windows.</figcaption></figure><p>HASS agent doesn’t directly have support for Spotify and Spotify’s client on Windows doesn’t unfortunately support command line parameters in a way that would be useful for us (nor it has support for D-Bus like on Linux).</p><p>So the workflow we can set up is:</p><ol><li>Zigbee switch signals its state towards Home Assistant</li><li>Automation triggers based on switch going from on to off or vice versa</li><li>Home Assistant sends a custom HASS agent command to a MQTT broker (I use the Mosquitto broker installed on Home Assistant)</li><li>HASS agent picks up the command on the PC and executes a PowerShell script</li></ol><p>There are some recipes floating around online on controlling Spotify on Windows via WMI, but it didn’t seem to be a very robust solution. So I turned towards PowerShell as it’s a fairly easy way to add some scripting on Windows. There are a few PowerShell modules for Spotify, but they don’t seem to be very recently updated, except for <a href="https://github.com/lennyomg/Spotify-PowerShell">lennyomg/Spotify-PowerShell</a> .</p><p>To make everything work, you’ll need to do the following:</p><ol><li>Install HASS agent on the PC and configure it to work with Home Assistant,</li><li>Install HASS agent add on to Home Assistant,</li><li>Create your application on <a href="https://developer.spotify.com/">developer.spotify.com</a> (remember to follow instructions from the Spotify-PowerShell module),</li><li>Initialize the connection between your PC and Spotify’s API,</li><li>Configure automations in Home Assistant to trigger the custom commands in HASS agent,</li></ol><p>As there is nothing particular special about steps 1 to 3, we’ll skip straight to step 4. But before, we did install PowerShell 7 via WinGet:</p><pre>winget install --id Microsoft.PowerShell --source winget</pre><p>After installation, we should have the PowerShell binary at C:\Program Files\PowerShell\7\pwsh.exe . We can start a PowerShell session and set up the initial connection. Have your Spotify app client ID handy and run:</p><pre>Import-Module Spotify<br>New-SpotifyAccessToken -ClientId &quot;123456789...abcdef&quot; -Scope &quot;app-remote-control&quot;, &quot;streaming&quot;, &quot;user-read-playback-state&quot;, &quot;user-modify-playback-state&quot;</pre><p>Open the URL and copy the command from the text box you receive afterwards into PowerShell and execute it.</p><p>Now we’ll use the following script (also <a href="https://gist.github.com/rosmo/0c4d83fdd0e2625f6dbefcd880211e32">Gist available here</a>, which has extended features like playlists and shuffle) that we will play in our home directory (eg. C:\Users\yourusername\ ):</p><pre>Param<br>  (<br>     [Parameter(Position=0)]<br>     [ValidateSet(&#39;play&#39;, &#39;stop&#39;)]<br>     [String]<br>     $Operation = $(throw &quot;Operation is required (play or stop).&quot;),<br><br>     [alias(&quot;Device&quot;)] <br>     [String]<br>     $DeviceName = $env:COMPUTERNAME,<br><br>     [String]<br>     $Track = &quot;spotify:track:4BP3uh0hFLFRb5cjsgLqDh&quot;,<br><br>     [String]<br>     $SpotifyExe = &quot;spotify:&quot; # UWP apps are launched through URL scheme<br>  ) <br><br>Import-Module Spotify<br><br># Update the access token<br>Update-SpotifyAccessToken<br><br># Check if Spotify is running, if not, start it<br>if ((Get-Process &quot;Spotify&quot; -ErrorAction SilentlyContinue) -eq $Null) {<br>    Start-Process -FilePath $SpotifyExe<br>}<br><br>$DeviceId = $null<br>$Attempt = 0<br>do {<br>    Get-SpotifyDevices | ForEach-Object {<br>        if ($_.name -eq $DeviceName) {<br>            $DeviceId = $_.id<br>        }<br>    }<br>    if ($DeviceId -eq $null) {<br>        # If the device isn&#39;t here yet, wait for 20 seconds until it appears<br>        Start-Sleep -Seconds 2<br>        $Attempt++<br>    } else {<br>        break<br>    } <br>} while ($Attempt -lt 10)<br><br>if ($DeviceId -eq $null) {<br>    throw &quot;Unable to find Spotify device: $DeviceName&quot;  <br>} <br><br>switch ($Operation) {<br>    &quot;play&quot; {<br>        Start-SpotifyPlayback -TrackUri $Track -DeviceId $DeviceId<br>    }<br>    &quot;stop&quot; {<br>        Suspend-SpotifyPlayback -DeviceId $DeviceId<br>    }<br>}</pre><p>The default use is simple, simply run the script with play or stop argument and it’ll start up Spotify, if it’s not already running and start playing the track. You can set device and track ID via Device and Track parameters.</p><p>Finally, we will add two custom commands to HASS:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/899/1*J1SjvPaSudVPZMprKIx3Dw.png" /><figcaption>Custom commands.</figcaption></figure><p>For the command, we will use these:</p><pre>To play:<br>&quot;C:\Program Files\PowerShell\7\pwsh.exe&quot; &quot;C:\Users\yourusername\fortunate-son.ps1&quot; play<br><br>To stop:<br>&quot;C:\Program Files\PowerShell\7\pwsh.exe&quot; &quot;C:\Users\yourusername\fortunate-son.ps1&quot; stop</pre><p>You should now be able to trigger these commands from Home Assistant and watch the magic at work:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KcwocbnvuyCFkP6O0dOwiA.png" /><figcaption>HASS automation.</figcaption></figure><p>That’s it, hopefully this was useful!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e14fb7abd109" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Spaceballs — The Switch! (Zigbee’ing a device)]]></title>
            <link>https://taneli-leppa.medium.com/spaceballs-the-switch-zigbeeing-a-device-fd2f5d3ee907?source=rss-6c5d20f5d1ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/fd2f5d3ee907</guid>
            <category><![CDATA[zigbee]]></category>
            <category><![CDATA[spaceballs]]></category>
            <category><![CDATA[esp-idf]]></category>
            <dc:creator><![CDATA[Taneli Leppä]]></dc:creator>
            <pubDate>Wed, 22 May 2024 10:44:35 GMT</pubDate>
            <atom:updated>2024-10-29T09:24:33.498Z</atom:updated>
            <content:encoded><![CDATA[<h3>Spaceballs — The Switch! (Zigbee’ing a device)</h3><p>If you grew up in the same period of time that I did, you’re probably a big fan of <a href="https://www.imdb.com/title/tt0094012/"><em>Spaceballs</em></a>. So, when I saw somewhere that a company was making the switch from Spaceballs, I knew I had to have it. But it was just a potentiometer and a flip switch. But I wanted it to be more: I wanted it to control my office setup.</p><p>What happened next is that due to some other projects, I got a 3D printer (<a href="https://store.creality.com/eu/products/k1c-3d-printer">Creality K1C</a>) and did a few different Zigbee projects (eg. <a href="https://github.com/rosmo/aircleaner-control">aircleaner-control</a> for a <a href="https://en.wikipedia.org/wiki/Corsi%E2%80%93Rosenthal_Box">Corsi-Rosenthal box)</a>. So it was a natural progression to make the switch into my office light/monitor/whatnot controller.</p><p>If you decide to buy this device (or any other device with 100K potentiometer and a flip switch), I’ll be sharing full printable 3D models, code and guidance to make it super easy for you to recreate the switch.</p><p>First, let’s see the actual device:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_vnpKQDdLoTdi1n1bPf5Gg.jpeg" /></figure><p>First, I figured out the software part (which was quite easy with ESP-IDF, setting up a potentiometer and high/low GPIO will pull down resistor). So, all you need to do is to model 5 switches in Zigbee.</p><p>Then I started modelling a 3D printable case for the switch and thru quite a few 3D prints, I converged on a design. You can see some of my missteps below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j15MIYme8kHS9UnfgcAbzw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mIFGfhjJUa4o8sUa1iGvDQ.jpeg" /><figcaption>On the left: fitment test for first protoype, on the right: several case prototypes and misprints</figcaption></figure><p>Also the software side looks like this, paired with Home Assistant (I’m using a cheap TI CC2531 USB off Amazon):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hKe6nhRSQUcCncChdQ2DzQ.png" /></figure><p>Picture of the prototype on my desk:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SBUbep_Ekpr1h8oGAPNuMA.jpeg" /></figure><p>Of course, a few iterations were needed (and adding bunch of lead weights into the case). But the device itself works well.</p><p>If you want to build one yourself, you can use my ESP-IDF code. The device acts a Zigbee router so it should be able to extend your Zigbee network as well.</p><p>All you need to do is install ESP-IDF 5.2, compile, flash, print and solder. You’ll also of course need a ESP32-C6 device (any will do most likely).</p><p>The code is available here, alongside with the 3MF and STL files for the case. Please note that it’s one of my first 3D models, so it’s not gonna quite perfect, however, you can use it to build your own, especially with regards to the mounting pin layout. There is a hole for a USB-C socket in the rear, and a possibility for a magnet to further keep the device in place.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0kbeUevkQuXYRkwUiVkRzg.png" /><figcaption>Space3r on iPad.</figcaption></figure><p>The code: <a href="https://github.com/rosmo/spaceballs-switch">https://github.com/rosmo/spaceballs-switch</a></p><p>The 3D case is modeled using <a href="https://www.shapr3d.com/">Shap3r</a> on a iPad (I can’t say enough praise about Shap3r for a new 3D modeler compared to Fusion for example — except that it’s wildly expensive).</p><p>The switch itself presents as a 5 switch Zigbee device, where one is the Go-button and 4 other switches (that are kept turned on, you can change the functionality by defining by uncommenting the INDEPENDENT_SWITCHES <a href="https://github.com/rosmo/spaceballs-switch">here</a>).</p><p>Feel free to contact me if you’re having trouble, for example of <a href="https://twitter.com/rosmo">Twitter</a> or just shoot me an <a href="/service/mailto:rosmo@rosmo.fi">email</a>.</p><p>Finally, a photo of the final product:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G0IfoHDXrCQQyiDwo6C9nA.jpeg" /><figcaption>The final switch, still need to sand down the case though.</figcaption></figure><p><strong><em>PS. Since writing this article, I have upgraded with an additional switch and the code has been refined a bit to support that (and updated to latest ESP-IDF Zigbee library. See also </em></strong><a href="https://taneli-leppa.medium.com/controlling-spotify-via-home-assistant-e14fb7abd109"><strong><em>my other article</em></strong></a><strong><em> on how to control Spotify using the Fortunate Son switch.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q1gMoPpH8X0OXZM6vJ6eFQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fd2f5d3ee907" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gemini Pro 1.5 multi-modal Slack bot]]></title>
            <link>https://medium.com/google-cloud/gemini-pro-1-5-multi-modal-slack-bot-a834d397c10b?source=rss-6c5d20f5d1ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/a834d397c10b</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <category><![CDATA[pub-sub]]></category>
            <category><![CDATA[gemini]]></category>
            <category><![CDATA[gcp-app-dev]]></category>
            <dc:creator><![CDATA[Taneli Leppä]]></dc:creator>
            <pubDate>Wed, 10 Apr 2024 16:24:39 GMT</pubDate>
            <atom:updated>2024-04-12T08:10:08.525Z</atom:updated>
            <content:encoded><![CDATA[<p>In <a href="https://medium.com/google-cloud/multi-modal-gemini-pro-vision-slack-chat-bot-dd4ea46147bc">my previous article</a>, I showed how to create a Slack bot that uses Gemini 1.0 Pro Vision LLM to provide users with interactive multi-modal discussion capabilities on Slack.</p><p>Since Google Cloud Next 2024 just kicked off, we got a nice surprise by having public access to the Gemini 1.5 Pro model. It has a bunch of really cool new capabilities (such as audio) and improved performance. I wanted to update the bot to the new version, which turned out to be relatively straightforward.</p><p>And as always, you can deploy the bot into your own Slack workspace (see the instructions <a href="https://medium.com/google-cloud/multi-modal-gemini-pro-vision-slack-chat-bot-dd4ea46147bc">in the previous article</a> and make sure you have the latest code from the <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/tree/main/examples/gemini-pro-slack-bot">repository</a>).</p><p>The bot can be also integrated with <a href="https://cloud.google.com/enterprise-search">Vertex AI Search</a> (and other REST APIs) to give the model direct access into topics specific to your business.</p><p>So let’s explore some of new capabilities that Gemini 1.5 can give to your Slack bot:</p><h3>Basic context-aware answers</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/1*9Ft9FCzMnsKUM5gQmdp6Lg.png" /><figcaption>The questions that bother us.</figcaption></figure><h3>Integration with Vertex Search</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/918/1*4OO9fPTDF7PWsolxIanP_g.png" /><figcaption>This question calls an API function that calls Vertex Search, see next image.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r69m3yJ_map7enSIq4p7_Q.png" /><figcaption>The results returned back to Gemini API.</figcaption></figure><h3>Working with PDFs</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/902/1*q_xvIGtizfwslgyzCOyP7w.png" /><figcaption>Providing a PDF (in this case, a monitor manual) and asking a question from it.</figcaption></figure><h3>Answers using images</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*FQhc1WptQgUvdRvGFOKXQA.png" /><figcaption>Had to use this image.</figcaption></figure><h3>Answers using voice</h3><p>Last but not the least (and for the coolest Gemini 1.5 capability!), we can also just ask Gemini directly using a voice message on Slack:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/958/1*oLQTqOMaLXqzfBqTUv6LhQ.png" /><figcaption>Nice!</figcaption></figure><p>That’s it. Let me know if you deploy a Slack bot using my code or if you take inspiration from it. You can reach me for example of X: <a href="https://twitter.com/rosmo">@rosmo</a> or just drop a comment.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a834d397c10b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/gemini-pro-1-5-multi-modal-slack-bot-a834d397c10b">Gemini Pro 1.5 multi-modal Slack bot</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multi-modal Gemini Pro Vision Slack chat bot]]></title>
            <link>https://medium.com/google-cloud/multi-modal-gemini-pro-vision-slack-chat-bot-dd4ea46147bc?source=rss-6c5d20f5d1ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd4ea46147bc</guid>
            <category><![CDATA[gcp-app-dev]]></category>
            <category><![CDATA[vertex-ai]]></category>
            <category><![CDATA[geminipro]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[cloud-functions]]></category>
            <dc:creator><![CDATA[Taneli Leppä]]></dc:creator>
            <pubDate>Wed, 27 Mar 2024 14:09:23 GMT</pubDate>
            <atom:updated>2024-04-04T12:03:56.039Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lZ6GwlVPZp2wDCTvxAibQw.png" /><figcaption>Slack bot architecture.</figcaption></figure><p>In <a href="https://taneli-leppa.medium.com/building-an-ai-slack-bot-using-vertex-generative-ai-d5f2c9e5e0b0?source=user_profile---------1----------------------------">my last article</a>, we built a PaLM-based chat bot. But since then, the state of art has moved onwards and it’s time to make some improvements to our chat bot and move it over to Gemini.</p><p>We also want to some super powers to the bot, namely multi-modal dialogue support. As always, a fully deployable example is provided.</p><h3>Deploying the bot</h3><p>To deploy the bot, check out the code from <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/tree/main/examples/gemini-pro-slack-bot">GoogleCloudPlatform/pubsub2inbox</a> and follow the <strong>README</strong> instructions.</p><p>This time deploying the bot is even easier, as there is now a Slack manifest for getting the permissions correct the first time.</p><p>Once deployed and added to your Slack workspace, you can discuss with the app directly or add it to a channel and tag it. Some examples of what it can do:</p><h3>Normal text conversation</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*J5nvbA1R4Xn-0JWVTwgeXA.png" /><figcaption>Asking for advice on Terraform.</figcaption></figure><h3>Explaining an image</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*EiUj9kppvsQTxSZQhRqKZg.png" /><figcaption>Well, it isn’t fully automatic, but close.</figcaption></figure><h3>Explaining a PDF</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*4_-mCRi36R5KEzfVEwDpgg.png" /><figcaption>Summarizing PDF for recommendations.</figcaption></figure><blockquote>You can also provide a prompt to be added to a discussion (prompt parameter in Slack processors processMessages part).</blockquote><p><strong><em>Update: Now the bot can also integrate with </em></strong><a href="https://cloud.google.com/generative-ai-app-builder/docs/try-enterprise-search"><strong><em>Vertex Search</em></strong></a><strong><em> (or any other REST service) directly using Gemini function calling! </em></strong><em>See example here: </em><a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/tree/main/examples/gemini-pro-slack-bot#integrating-vertex-ai-search"><em>Integrating Vertex AI Search</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/1*9JGLbWMt0LS9CCCu2lVnPQ.png" /><figcaption>Information sourced from Vertex AI Search via Gemini Pro function call.</figcaption></figure><p>That’s it — you can customize the bot as you wish. Interested to hear about your use cases! You can reach out to me on X for example: <a href="https://twitter.com/rosmo">twitter.com/rosmo</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd4ea46147bc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/multi-modal-gemini-pro-vision-slack-chat-bot-dd4ea46147bc">Multi-modal Gemini Pro Vision Slack chat bot</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Synchronizing containers from JFrog Artifactory to Google Cloud Artifact Registry]]></title>
            <link>https://taneli-leppa.medium.com/synchronizing-containers-from-jfrog-artifactory-to-google-cloud-artifact-registry-5346a13c5808?source=rss-6c5d20f5d1ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/5346a13c5808</guid>
            <category><![CDATA[jfrog-artifactory]]></category>
            <category><![CDATA[artifactory]]></category>
            <category><![CDATA[google-artifact-registry]]></category>
            <category><![CDATA[artifact-registry]]></category>
            <category><![CDATA[pub-sub]]></category>
            <dc:creator><![CDATA[Taneli Leppä]]></dc:creator>
            <pubDate>Wed, 14 Feb 2024 13:16:12 GMT</pubDate>
            <atom:updated>2024-02-14T13:18:46.028Z</atom:updated>
            <content:encoded><![CDATA[<p>If you are leveraging JFrog Artifactory on-premise or other clouds (or even in GCP), you may want to also make your containers available via Artifact Registry. This could be for several purposes:</p><ul><li>No access from GKE or other Compute-based services to Artifactory</li><li>Use your containers in Cloud Functions or Cloud Run (as they can only pull from GCR/AR)</li><li>Scalability or reliability issues</li></ul><p>Having containers both in Artifactory and Artifact Registry can be of course solved with pushing the images from your build pipelines to both registries. But this is extra effort for every pipeline.</p><p>In this article, I’ll present an alternative, based on Artifactory’s support for webhooks and a Cloud Function. We will be basing this on the trusty <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox">Pubsub2Inbox</a> tool (and its sidekick, <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/tree/main/cmd/json2pubsub">Json2Pubsub</a>). Let’s start with a quick overview on the architecture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R9PNOp9yMNUS1uHUSzhAFg.png" /><figcaption>Architecture of the solution.</figcaption></figure><p>The architecture is fairly straightforward. The following flow happens when everything is set up correctly and you push a container to Artifactory:</p><ol><li>A container is pushed to Artifactory with a tag.</li><li>The push triggers a webhook in Artifactory which targets the Json2Pubsub function (the function can be public accessible as it has basically a single IAM permission, or you can also opt for other means, like fronting it with a load balancer or using INTERNAL_ONLY and accessing it via Interconnect directly).</li><li>Json2Pubsub receives the webhook payload and verifies that the shared secret is correctly set in the request header. If so, it writes the payload into a Pub/Sub topic.</li><li>The Pub/Sub message triggers the Pubsub2Inbox Cloud Function.</li><li>The Cloud Function processes the JSON payload and based on the event type, performs either a Docker pull from the source in payload or a delete in Artifact Registry. The Serverless VPC connector provides private access path to Artifactory.</li><li>In case of a new container, a pure Python implementation of Docker downloads the container manifest and layers and then pushes it to Artifact Registry.</li></ol><p>Seems simple. Let’s go through the setup process.</p><h3>Setting up Artifactory</h3><p>For testing purposes, let’s create a new Docker repository in Artifactory:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kTA5w87K1xTjPxQZmEWJrg.png" /><figcaption>Adding a new Docker repository in Artifactory.</figcaption></figure><p>Once the repository is set up, we can create a webhook:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W4y1b02NKx8NvP0BuamfDA.png" /><figcaption>Creating a webhook.</figcaption></figure><p>When creating the webhook, just put a temporary valid URL as the webhook URL. We will update this later with the proper URL of the Cloud Function as it’s not deployed yet. For events, select “Docker tag pushed”, “Docker tag was deleted” and “Docker tag was promoted” event types. Then click Select Repositories and select the repository you just created in the previous step:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xugd_wLOmvGR0W7fI0B2UA.png" /><figcaption>Selecting the repositories.</figcaption></figure><p>For the webhook secret, we can generate a cryptographically secure shared secret using for example OpenSSL:</p><pre># openssl rand -hex 32<br>62ed8ed8d0f2dcc4159ae003f342c5a2a14add86214fc283017865b74f150d9d</pre><p>Take note of the secret as we will need later when setting up Cloud Function. Remember <strong>NOT</strong> to select “Use secret for payload signing” (as how it works is completely undocumented).</p><p>While we could set up the synchronization using our user and password, we’ll set it up properly by creating a separate integration user and granting it Read permissions only to select repositories.</p><p>First, create the user in Artifactory:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KKdcPsV86ZaTh9ABJijjtA.png" /><figcaption>Creating the user</figcaption></figure><p>Remember to remove the user from the default groups, such as readers. Next, we’ll create a new Permission for the just the synchronization user:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aSU88o4ulqUqPijxmPNXnw.png" /><figcaption>Creating a permission.</figcaption></figure><p>Select the repositories you want to be able to synchronize, then select the synchronization user:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U2c83Bzui4DdN5QNeOsGkw.png" /><figcaption>Selecting the user for the permission.</figcaption></figure><p>Finally, let’s select the correct Read permission:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HsZpgzLvv3MA5k3xL7TtDg.png" /><figcaption>Just read.</figcaption></figure><p>Next, we will create an access token for the user to be used by the Cloud Function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iEu3DvdebyWUc3L_kedqKQ.png" /><figcaption>Creating the reference access token.</figcaption></figure><p>After generate, the UI will show you the tokens once:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RrwZ0N9jRKgw79bqnVC1bw.png" /></figure><p>Make a note of the reference token (shorter one) as this will be one “password” that the Cloud Function will be using to access Artifactory. Also please note that there is a delay before the token starts working.</p><p>The setup from Artifactory side is now almost complete. Next we will deploy the Cloud Function.</p><h3>Setting up the Cloud Function</h3><p>You can start by cloning the example code (the whole repository) from <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/tree/main/examples/artifactory-to-artifact-registry">pubsub2inbox/examples/artifactory-to-artifact-registry/</a>. In the directory, lets create a terraform.tfvars file to contain our configuration. The contents should be along these lines (check variables.tf for full customization options):</p><pre>project_id                 = &quot;&lt;your project id&gt;&quot;<br>region                     = &quot;europe-west4&quot;<br>artifactory_webhook_secret = &quot;&lt;the secret you created with openssl&gt;&quot;<br>artifactory_username       = &quot;gcp-sync&quot;<br>artifactory_password       = &quot;&lt;the reference token you created&gt;&quot;<br>vpc_config = {<br>  network         = &quot;&lt;an-existing-vpc&gt;&quot;<br>  cidr            = &quot;10.8.0.0/28&quot; # CIDR range for serverless connector<br>  # network_project = &quot;&lt;shared-vpc-project&gt;&quot; to use shared VPC<br>  # create_connector = false to use existing serverless connector<br>  # connector        = &quot;connector id&quot; for an existing serverless connector<br>}</pre><p>After this, we can deploy the function using Terraform by running the following commands:</p><pre># terraform init<br>...<br># terraform apply<br><br>Plan: 40 to add, 0 to change, 0 to destroy.<br><br>Changes to Outputs:<br>  + instructions = (known after apply)<br><br>Do you want to perform these actions?<br>  Terraform will perform the actions described above.<br>  Only &#39;yes&#39; will be accepted to approve.<br><br>  Enter a value: yes<br><br>...</pre><p>Once the deployment is complete (a few minutes), you should see this:</p><pre>Apply complete! Resources: 40 added, 0 changed, 0 destroyed.<br><br>Outputs:<br><br>instructions = &lt;&lt;EOT<br>Please use the following webhook URL for Artifactory webhook:<br>  https://artifactory-to-artifact-registry-json2pubsub-ma123bc44f-ez.a.run.app<br><br>EOT</pre><p>Capture the Cloud Function URL and update it in Artifactory webhook settings.</p><p>That’s it. We can now build and push a container image into the Artifactory repository we configured, and if all goes according to plan, the container will shortly appear in the Artifact Registry that the Terraform code created:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/947/1*PU1kOaTJE7VqVz2vQSqWiw.gif" /><figcaption>Demonstration of the sync.</figcaption></figure><p>That’s it. The code also supports deleting tags (remove a tag in Artifactory with curl -u &quot;user:pw&quot; -X DELETE &quot;https://artifactory.url/artifactory/repository-name/package-name/tag-name ).</p><p>Thanks for reading another installation of “the Pubsub2Inbox series”!</p><p><em>PS. We might revisit the solution for two additional use cases: replication of containers between Artifact Registries in different regions, and multi-cloud.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5346a13c5808" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[‘Can I get that in writing?’ Email logs from Cloud Run jobs and other services]]></title>
            <link>https://taneli-leppa.medium.com/can-i-get-that-in-writing-email-logs-from-cloud-run-jobs-and-other-services-8de93751ba0b?source=rss-6c5d20f5d1ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/8de93751ba0b</guid>
            <category><![CDATA[cloud-run-job]]></category>
            <category><![CDATA[pub-sub]]></category>
            <category><![CDATA[cloudrun]]></category>
            <dc:creator><![CDATA[Taneli Leppä]]></dc:creator>
            <pubDate>Fri, 24 Nov 2023 14:34:16 GMT</pubDate>
            <atom:updated>2023-11-24T14:34:16.666Z</atom:updated>
            <content:encoded><![CDATA[<h3>The problem</h3><p>While we all wish we had time to keep an eye on dashboards, nothing beats a well timed email to catch one’s attention. Not many services in Google Cloud have direct email integration and for that reason I originally built <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/">Pubsub2Inbox</a> (<em>and yes, it’s going to be another Pubsub2Inbox article</em>).</p><p>This time I was looking for a job-agnostic way of getting logs from Cloud Run jobs (a very neat solution for recurrent automation tasks) to my email so I can keep up on what they are doing. If you haven’t tried Cloud Run jobs yet, they’re great — very flexible and no infrastructure required.</p><p>Cloud Run jobs log their output (ie. stdout and stderr) into Cloud Logging, but it’s not trivial to catch this output from different kinds of containers. Also, I really didn’t want each script to have an email solution of its own.</p><h3>The solution</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SHEfquZfuTQ2WJfpZ3DL6Q.png" /><figcaption>Architecture of the solution.</figcaption></figure><p>Pubsub2Inbox already has <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/blob/main/output/mail.py">an email output</a>. It can send email via:</p><ul><li><a href="https://sendgrid.com/">Sendgrid</a> API</li><li>Microsoft Graph API</li><li>Custom SMTP server (now easier than ever with <a href="https://cloud.google.com/run/docs/configuring/vpc-direct-vpc">Direct VPC egress</a>)</li></ul><p>But it doesn’t have to be email only! You could also send the logs via <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/blob/main/output/slack.py">Slack</a> or <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/blob/main/output/chat.py">Google Chat</a> (and we can even <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/blob/main/output/twilio.py">send SMS</a>!).</p><p>I also wanted to get some information from the Cloud Run job execution and the actual logs from Cloud Logging (so talking to run.googleapis.com and logging.googleapis.com), so I wrote two new processors: <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/blob/main/processors/cloudrun.py">cloudrun</a> and <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/blob/main/processors/logging.py">logging</a> .</p><p>Finally we need a way to trigger Pub/Sub. While it’s trivial to do with gcloud pubsub topics publish , I wanted a more lightweight and easy way as all we need to do is push a simple message to a Pub/Sub queue. This is done just with a few curl commands:</p><pre># Grab a token from the metadata endpoint<br>TOKEN=$(curl -sH &quot;Metadata-Flavor: Google&quot; &quot;http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token&quot; | jq -r &quot;.access_token&quot;)<br># Encode the task information in JSON and wrap it in base64<br>MESSAGE=$(echo -n &quot;{\&quot;job\&quot;:\&quot;${CLOUD_RUN_JOB}\&quot;,\&quot;execution\&quot;:\&quot;${CLOUD_RUN_EXECUTION}\&quot;,\&quot;index\&quot;:${CLOUD_RUN_TASK_INDEX},\&quot;attempt\&quot;:${CLOUD_RUN_TASK_ATTEMPT}}&quot; )<br>MESSAGE_B64=$(echo $MESSAGE | base64 | tr -d \\n)<br># Post a message to the public topic<br>curl -sH &quot;Authorization: Bearer ${TOKEN}&quot; \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -X POST \<br>  -d &quot;{&#39;messages&#39;:[{&#39;data&#39;:&#39;${MESSAGE_B64}&#39;}]}&quot; \<br>  &quot;https://pubsub.googleapis.com/v1/${PUBSUB_TOPIC}:publish&quot;</pre><p>So that’s pretty easy. For the demo, we’ll be just using <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/blob/main/examples/cloud-run-jobs/Dockerfile">a simple container</a> based on Google’s base images where we add the good old Unix fortune command, a tool that’s two years older than me.</p><p>For the <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/blob/main/examples/cloud-run-jobs/cloud-run-jobs.yaml.tpl">Pubsub2Inbox configuration</a>, we fetch the job details with cloudrun processor to get information such as when the job was completed and then fetch the logs using an entries.list request to Cloud Logging with the appropriate filter. Finally, we put the log entries in the email and send it either via SendGrid or a custom SMTP server (the demo can use smtp-relay.gmail.com but setting it up is cumbersome and requires “Less Secure Apps” support which is deprecated and will be removed by the end of 2024).</p><p>As always, full Terraform code is provided. Just clone the repository from <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/tree/main">https://github.com/GoogleCloudPlatform/pubsub2inbox/tree/main</a> and navigate to examples/cloud-run-jobs .</p><p>You’ll need to build the container. I suggest setting up an Artifact Registry in the project and pushing it there:</p><pre>docker build \<br>  -t europe-west4-docker.pkg.dev/your-project-id/your-repository/cloud-run-example:latest \<br>  .<br>docker push \<br>  europe-west4-docker.pkg.dev/your-project-id/your-repository/cloud-run-example:latest</pre><p>There you should create a terraform.tfvars file with content similar to:</p><pre>project_id         = &quot;your-project-id-here&quot;<br>region             = &quot;europe-west4&quot;<br>job_container      = &quot;europe-west4-docker.pkg.dev/your-project-id/your-repository/cloud-run-example:latest&quot;<br>sender             = &quot;the-sender@example.com&quot;<br>recipient          = &quot;your-email@example.com&quot;<br>sendgrid_api_token = &quot;SG.....&quot;</pre><p>You can now run terraform init and then terraform apply . The example will create a Pub/Sub topic, a sample Cloud Run job and deploy Pubsub2Inbox as a Cloud Function v2 (I’ve always been partial to V2s).</p><p>After everything has been created, you can go into Cloud Console and click execute on the sample job that was created:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D1mwbB4Ak9MYCdGgBpTL7w.png" /><figcaption>The Cloud Run job.</figcaption></figure><p>If you check the Cloud Function, you should see that it’s executing and sending the email:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*43KkLAU5l4wH5XR_MQ7n7A.png" /><figcaption>Pubsub2Inbox processing.</figcaption></figure><p>Finally, check your mailbox and receive your fortune:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S73ww30D259S3lDBAVV8rg.png" /><figcaption>The email from Pubsub2Inbox containing the Cloud Run job output.</figcaption></figure><p>You could extend this in multiple ways:</p><ul><li>Pass Cloud Storage links in the Pub/Sub messages and add them as attachments</li><li>Pass recipient information from the job as well</li></ul><p>Did you find this useful? Reach out to me through the usual channels — like my T̶w̶i̶t̶t̶e̶r̶ X account: <a href="https://twitter.com/rosmo">https://twitter.com/rosmo</a></p><p><em>PS. Some of the fortunes can be quite dark, so tread carefully.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8de93751ba0b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Simple and cost effective application high availability using Pubsub2Inbox and regional persistent…]]></title>
            <link>https://taneli-leppa.medium.com/simple-and-cost-effective-application-high-availability-using-pubsub2inbox-and-regional-persistent-37b28810edef?source=rss-6c5d20f5d1ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/37b28810edef</guid>
            <category><![CDATA[compute-engine]]></category>
            <category><![CDATA[google-pubsub]]></category>
            <category><![CDATA[cloud-functions]]></category>
            <category><![CDATA[high-availability]]></category>
            <dc:creator><![CDATA[Taneli Leppä]]></dc:creator>
            <pubDate>Sun, 12 Nov 2023 13:51:13 GMT</pubDate>
            <atom:updated>2023-11-12T14:10:42.384Z</atom:updated>
            <content:encoded><![CDATA[<h3>Simple and cost effective application high availability using Pubsub2Inbox and regional persistent disk</h3><p>You might have read <a href="https://cloud.google.com/compute/docs/disks/high-availability-regional-persistent-disk">Build HA services using regional Persistent Disk</a> which very neatly outlines the different options for providing high availability in case of a zonal outage. While you can always perform the failover using manual operations using gcloud, you may want to automate the failover. At minimum, this involves moving a data disk between two instances by using the force-attach<a href="https://cloud.google.com/compute/docs/reference/rest/v1/instances/attachDisk#query-parameters">method</a>.</p><h3>The architecture</h3><p>In this article, I’ll be providing an example of how to fail over an instance automatically based on a Pub/Sub message (a good candidate for the source are health check logs — since they only log on transition of states, ie. from healthy to unhealthy or vice versa). The setup will look like this:</p><ul><li>Two instances created in two different zones, with two zonal boot disks (the instances will be running Nginx just for demonstration purposes)</li><li>A regional persistent disk that is swapped between two instances (and snapshotted just before the move for extra safety)</li><li>Two unmanaged instance groups (UMIGs) containing the aforementioned instances</li><li>One regional passthrough TCP load balancer (you can also use a HTTP load balancer) with a HTTP health check</li></ul><p>The last piece of the puzzle is using my swiss army knife for Pub/Sub, <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox">Pubsub2Inbox</a>. We’ll be using the computeengineand loadbalancingprocessors to perform the failover and to update the load balancer backend service. The architecture looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UXsS28xWKgsfjax9UKHcgw.png" /><figcaption>Architecture diagram for the high availability setup</figcaption></figure><p>To decide when to do a failover, we’ll be using the health check log messages as the signal to initiate actions. This is easy for one reason: the health check only logs messages during transition from one status to another (eg. from HEALTHY to UNHEALTHY and vice versa), so we don’t need to filter or de-duplicate the Pub/Sub messages.</p><p>The full Terraform sample code is available at: <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/tree/main/examples/repd-failover">https://github.com/GoogleCloudPlatform/pubsub2inbox/tree/main/examples/repd-failover</a></p><h3>Code walkthrough</h3><p>First we create two instances (just using Container OS with an Nginx container and a startup script that initializes and mounts the disk):</p><pre>module &quot;primary-vm&quot; {<br>  source     = &quot;github.com/GoogleCloudPlatform/cloud-foundation-fabric//modules/compute-vm?ref=daily-2023.11.11&quot;<br>  project_id = module.project.project_id<br>  zone       = var.zones.primary<br>  name       = &quot;repd-failover-primary&quot;<br>  boot_disk = {<br>    initialize_params = {<br>      image = &quot;projects/cos-cloud/global/images/family/cos-stable&quot;<br>      type  = &quot;pd-ssd&quot;<br>      size  = 10<br>    }<br>  }<br>  attached_disks = [{<br>    name        = google_compute_region_disk.data-disk.name<br>    size        = google_compute_region_disk.data-disk.size<br>    source_type = &quot;attach&quot;<br>    source      = google_compute_region_disk.data-disk.id<br>  }]<br>  network_interfaces = [{<br>    network    = module.vpc.self_link<br>    subnetwork = module.vpc.subnet_self_links[format(&quot;%s/%s&quot;, var.region, var.vpc_config.subnetwork)]<br>  }]<br>  tags = [&quot;repd-failover&quot;]<br>  metadata = {<br>    user-data              = module.cos-nginx.cloud_config<br>    google-logging-enabled = true<br>  }<br>  service_account = {<br>    auto_create = true<br>  }<br>}<br><br>module &quot;secondary-vm&quot; {<br>  source     = &quot;github.com/GoogleCloudPlatform/cloud-foundation-fabric//modules/compute-vm?ref=daily-2023.11.11&quot;<br>  project_id = module.project.project_id<br>  zone       = var.zones.secondary<br>  name       = &quot;repd-failover-secondary&quot;<br>  boot_disk = {<br>    initialize_params = {<br>      image = &quot;projects/cos-cloud/global/images/family/cos-stable&quot;<br>      type  = &quot;pd-ssd&quot;<br>      size  = 10<br>    }<br>  }<br>  attached_disks = []<br>  network_interfaces = [{<br>    network    = module.vpc.self_link<br>    subnetwork = module.vpc.subnet_self_links[format(&quot;%s/%s&quot;, var.region, var.vpc_config.subnetwork)]<br>  }]<br>  tags = [&quot;repd-failover&quot;]<br>  metadata = {<br>    user-data              = module.cos-nginx.cloud_config<br>    google-logging-enabled = true<br>  }<br>  service_account = {<br>    email = module.primary-vm.service_account_email<br>  }<br>}</pre><p>Then we create two Unmanaged Instance Groups that each hold one of the instances. Now we can plug in these UMIGs into a regional passthrough Network Load Balancer on TCP port 80 and a health check that just probes the root document:</p><pre>module &quot;nlb&quot; {<br>  source     = &quot;github.com/GoogleCloudPlatform/cloud-foundation-fabric//modules/net-lb-ext?ref=daily-2023.11.11&quot;<br>  project_id = module.project.project_id<br>  region     = var.region<br>  name       = &quot;repd-failover&quot;<br><br>  backend_service_config = {<br>    protocol  = &quot;TCP&quot;<br>    port_name = &quot;http&quot;<br>  }<br><br>  forwarding_rules_config = {<br>    &quot;&quot; = {<br>      ports = [80]<br>    }<br>  }<br><br>  group_configs = {<br>    umig-primary = {<br>      zone = module.primary-vm.instance.zone<br>      instances = [<br>        module.primary-vm.self_link<br>      ]<br>      named_ports = { &quot;http&quot; = 80 }<br>    }<br>    umig-secondary = {<br>      zone = module.secondary-vm.instance.zone<br>      instances = [<br>        module.secondary-vm.self_link<br>      ]<br>      named_ports = { &quot;http&quot; = 80 }<br>    }<br>  }<br><br>  backends = [{<br>    group = module.nlb.groups.umig-primary.self_link<br>  }]<br><br>  health_check_config = {<br>    enable_logging      = true<br>    check_interval_sec  = 5<br>    healthy_threshold   = 2<br>    timeout_sec         = 5<br>    unhealthy_threshold = 4<br>    http = {<br>      port = 80<br>    }<br>  }<br>}</pre><p>Then it’s time to put together a log sink in the project, that will capture the health check logs and send them over to a Pub/Sub topic we create:</p><pre>module &quot;project&quot; {<br>  source         = &quot;github.com/GoogleCloudPlatform/cloud-foundation-fabric//modules/project?ref=daily-2023.11.11&quot;<br>  name           = var.project_id<br>  project_create = false<br>  services = [<br>    &quot;compute.googleapis.com&quot;,<br>    &quot;cloudfunctions.googleapis.com&quot;,<br>    &quot;run.googleapis.com&quot;,<br>  ]<br>  logging_sinks = {<br>    repd-failover-healthcheck = {<br>      destination = module.pubsub.id<br>      filter      = &lt;&lt;-EOT<br>        logName=&quot;projects/${var.project_id}/logs/compute.googleapis.com%2Fhealthchecks&quot; AND<br>        jsonPayload.healthCheckProbeResult.healthState=&quot;UNHEALTHY&quot; AND<br>        resource.type=&quot;gce_instance_group&quot; AND<br>        resource.labels.instance_group_name:&quot;repd-failover&quot;<br>      EOT<br>      type        = &quot;pubsub&quot;<br>    }<br>  }<br>}<br><br>module &quot;pubsub&quot; {<br>  source     = &quot;github.com/GoogleCloudPlatform/cloud-foundation-fabric//modules/pubsub?ref=daily-2023.11.11&quot;<br>  project_id = var.project_id # Use var to avoid circular dependency<br>  name       = &quot;repd-failover&quot;<br>  iam = {<br>  }<br>}</pre><p>Finally we’ll create use <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/blob/main/examples/repd-failover/repd-failover.yaml.tpl">a configuration template for Pubsub2Inbox</a> and plug in the necessary values from Terraform into the template and deploy a Cloud Function v2 in the project:</p><pre>module &quot;pubsub2inbox&quot; {<br>  source = &quot;../..&quot;<br><br>  project_id = module.project.project_id<br>  region     = var.region<br><br>  function_name      = &quot;repd-failover&quot;<br>  function_roles     = [&quot;compute-engine&quot;]<br>  cloud_functions_v2 = true<br><br>  service_account = &quot;repd-failover-pubsub2inbox&quot;<br>  pubsub_topic    = module.pubsub.id<br><br>  config = templatefile(&quot;${path.module}/repd-failover.yaml.tpl&quot;, {<br>    concurrency_bucket = module.lock-bucket.name<br>    project            = module.project.project_id<br>    primary = {<br>      instance       = module.primary-vm.instance.name<br>      zone           = module.primary-vm.instance.zone<br>      instance_group = module.nlb.group_self_links[&quot;umig-primary&quot;]<br>    }<br>    secondary = {<br>      instance       = module.secondary-vm.instance.name<br>      zone           = module.secondary-vm.instance.zone<br>      instance_group = module.nlb.group_self_links[&quot;umig-secondary&quot;]<br>    }<br>    regional_disk = {<br>      id          = google_compute_region_disk.data-disk.id<br>      region      = var.region<br>      device_name = google_compute_region_disk.data-disk.name<br>    }<br>    load_balancer = {<br>      backend_service = module.nlb.backend_service.name<br>      region          = var.region<br>    }<br>  })<br>  use_local_files  = true<br>  local_files_path = &quot;../..&quot;<br><br>  bucket_name     = format(&quot;repd-failover-source-%s&quot;, random_id.random.hex)<br>  bucket_location = var.region<br>}</pre><p>If you cloned the example code, you can apply the Terraform into your project (it will create VPC and all the necessary infrastructure components):</p><pre>examples/repd-failover$ terraform apply -var project_id=your-project-id<br>Plan: 44 to add, 0 to change, 0 to destroy.<br><br>Changes to Outputs:<br>  + instructions = (known after apply)<br><br>Do you want to perform these actions?<br>  Terraform will perform the actions described above.<br>  Only &#39;yes&#39; will be accepted to approve.<br><br>  Enter a value: yes<br><br>Apply complete! Resources: 44 added, 0 changed, 0 destroyed.<br><br>Outputs:<br><br>instructions = &lt;&lt;EOT<br>To access the application, open: http://34.123.123.123/<br><br>EOT</pre><p>Now the setup should be activate and serving on the load balancer IP address:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/399/1*TN235KP8JGM-DfKlZLVVvw.png" /><figcaption>Example of the application output.</figcaption></figure><p>We can now test the failover by either stopping the Nginx container or more easily by just terminating the current instance:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*dVFrpM6pQL0XiqVXzCIG5g.png" /><figcaption>Stopping the Nginx container on the primary instance.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tM3_u_4yvMPMfrCgQ9A0Pg.png" /><figcaption>Logging output from the Pubsub2Inbox failover function.</figcaption></figure><p>In a minute or two, the disk should have failed over to the other zone and the instance in the other zone should be running. We can check the load balancer status to see that the backends were updated and that the application is again serving traffic:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4FCEWXAH8qePLGibj1jVUA.png" /><figcaption>Secondary instance is now running and…</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FYYhEInuBsiayC8XWdNlkg.png" /><figcaption>…the backend service backends have been changed over to the secondary UMIG.</figcaption></figure><h3>Conclusion</h3><p>While there are many fine options for doing high availability (and not of all of them cloud specific), this method achieves a relatively application independent way of providing high availability.</p><p>However, it may not be the best option in all cases, especially with more complex application. You could also consider solutions like:</p><ul><li>Pacemaker (open source cluster manager)</li><li>Using managed services (Cloud Functions, GKE, etc.)</li><li>Running in regional managed instance groups and replicating data via Cloud Storage or filer, application-level replication mechanisms or database clustering services</li></ul><p>One other thing to keep in mind is that force-attaching a disk to another instance is a disruptive operation from I/O perspective and requires the filesystem to support journaling correctly (there is always a non-zero chance of filesystem issues due to possibly lurking bugs). In the example, we’ve added automatic snapshotting of the disk to reduce the chance of data loss.</p><p>Also, if you run terraform apply again, Terraform will want to switch the disk back to the primary instance and change the backend to primary UMIG. Before performing further Terraform configuration changes, you should make sure the application is being served from the primary VM (or augment the modules to have ignore_changes for certain fields).</p><p>Finally, zonal outages may not be as clear as a complete outage. You can also fail over the application manually by posting a message containing just {} to the Pub/Sub topic.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37b28810edef" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a wireless plant soil moisture monitoring system using ESP-IDF and ESP32-C3]]></title>
            <link>https://taneli-leppa.medium.com/building-a-wireless-plant-soil-moisture-monitoring-system-using-esp-idf-and-esp32-c3-9786588df286?source=rss-6c5d20f5d1ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/9786588df286</guid>
            <category><![CDATA[esp32]]></category>
            <category><![CDATA[esp-idf]]></category>
            <category><![CDATA[plant-care]]></category>
            <dc:creator><![CDATA[Taneli Leppä]]></dc:creator>
            <pubDate>Fri, 07 Jul 2023 07:46:40 GMT</pubDate>
            <atom:updated>2023-07-07T09:00:02.182Z</atom:updated>
            <content:encoded><![CDATA[<p>If you have have house plants, you probably have some issues with them. For example, ours seem to get brown and perish after a couple months. If you look online, they most common reason for brown leaves is either overwatering or underwatering (!).</p><p>So, to understand what’s the issue, I decided that I want to monitor the soil moisture more scientifically. I looked into commercial soil moisture sensors, but they are either a) unreliable or poorly made, or b) way too expensive (around 25–35 euros per sensor). Since we have about 10 plants, the cost would be quite high.</p><h3>The hardware</h3><p>So what’s the solution? Rolling your own of course! A friend of mine linked me to a new ESP32-C3 from Seeed Studio which seemed to contain all the bits I wanted: WiFi, a few ADC GPIOs and last but not the least, a battery charging controller (I wanted the sensor pods to work on battery power to facilitate easier placement).</p><p>I also scoured AliExpress for the sensors and found a simple soil moisture sensor for 77 cents a piece. I also bought a few 1200 mAh lithium polymer batteries that seemed a reasonable compromise between capacity and size.</p><h3>The firmware</h3><p>Finally, the ESP32 needs software. I had some previous experience with ESP-IDF so I figured out it’s a good start (I wanted something that’s easy to build and compile without extensive IDE setup and Git repository friendly).</p><p>I already had in mind where I wanted to store the moisture information in: InfluxDB. Home Assistant comes with an InfluxDB server add-on, so getting it up and running is extremely simple. All you need is an InfluxDB database, username and password.</p><p>In fact, cobbling together some ESP-IDF example codes resulted in getting something that’s able to read the analog sensors easily and send the data via HTTP to a local InfluxDB:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jE9Hrn51vWMQQh-PTo0Cjw.jpeg" /><figcaption>Moisture sensor with ESP32-C3 and data sent to InfluxDB.</figcaption></figure><p>So, next it was up to build some sensors. The ESP32 has two AD controllers, 1 and 2. The first controller is connected to D0-D2 and the second to D2. In this case, to make it easier, I opted to connect three sensors per sensor pod.</p><h3>Putting it together</h3><p>It was time to start building. I decided to use a bit of prototyping board to mount JST-XH connectors (the same as on the sensor side) so I can disconnect and reconnect the sensors. The sensors take in 3.3V and ground and output 0–3.3V through the third connector. The 3.3V and GND are connected between all the sensors and the third pin is connector to the D0-D2 pins.</p><p>I also used 2 pin JST-XH connectors for the batteries so they can be unplugged.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xIEQ49Rl-Jxc5Ok5p4AEZw.jpeg" /><figcaption>First attempt, didn’t fit.</figcaption></figure><p>I ordered quite small cases (63 x 57 x 35mm) in the beginning, guessing that everything fits in. But unfortunately they turned out to be too small to accommodate the short USB extension cables (for recharging). So slightly larger cases needed to be ordered (115 x 90 x 55mm).</p><p>The sensors themselves are connected via 26 AWG wire and wrapped in a braided cable sheath (4mm).</p><p>While waiting for the new cases, I decided to update the code by adding MAC-based station configuration via a configuration URL. Home Assistant was able to host the configuration text file without much trouble.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jI7G2VNg71lwXzZ4apzb7Q.jpeg" /><figcaption>Testing the unit with cables.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZEmgZYvZWEJlA3EVNDuMbw.jpeg" /><figcaption>The soil moisture sensor.</figcaption></figure><h3>Finishing the build</h3><p>After the new cases, all was needed is to put the hardware inside the box and charge the batteries. I also did some fixes to the firmware, including UDP syslog, improving the NTP time sync and updating ESP-IDF from 5.0 to 5.1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HbQjw3oE_fJ7wteiBp5_5w.png" /><figcaption>Syslog output from ESP32, captured via rsyslog.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6oLp4PApNWQjhzfvDiWPEA.jpeg" /><figcaption>Two of the fully built units.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zPaUbPjNrKGNz7kLP5UDrg.jpeg" /><figcaption>Charging the units via USB.</figcaption></figure><p>Now every unit sleeps for an hour in deep sleep, wakes up, reads the ADCs and submits the information to InfluxDB via a HTTP request. I opted to send raw numbers to InfluxDB, since I can do the dry-moist mapping there. No need to involve the firmware in that. This is how my dashboard looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kL7AnkNGfA0yh3M4HY56Yw.png" /><figcaption>Grafana dashboard in Home Assistant.</figcaption></figure><h3>Bill of Materials</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fairtable.com%2Fembed%2FshrFwuk2bBtCP7pai&amp;display_name=Airtable&amp;url=https%3A%2F%2Fairtable.com%2FshrFwuk2bBtCP7pai&amp;image=https%3A%2F%2Fstatic.airtable.com%2Fimages%2Foembed%2Fairtable.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=airtable" width="800" height="533" frameborder="0" scrolling="no"><a href="https://medium.com/media/d99127122525e82ba7cb87e726615fa7/href">https://medium.com/media/d99127122525e82ba7cb87e726615fa7/href</a></iframe><h3>Build your own</h3><p>I’ve placed the code here alongside with the instructions if you want to build your own: <a href="https://github.com/rosmo/plant-monitoring">https://github.com/rosmo/plant-monitoring</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9786588df286" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Connecting to Google Cloud from OPNsense via IPSec VPN]]></title>
            <link>https://taneli-leppa.medium.com/connecting-to-google-cloud-from-opnsense-via-ipsec-vpn-206f6fbe1833?source=rss-6c5d20f5d1ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/206f6fbe1833</guid>
            <category><![CDATA[google-cloud]]></category>
            <category><![CDATA[cloud-vpn]]></category>
            <category><![CDATA[opnsense]]></category>
            <category><![CDATA[bgp]]></category>
            <dc:creator><![CDATA[Taneli Leppä]]></dc:creator>
            <pubDate>Fri, 26 May 2023 10:15:47 GMT</pubDate>
            <atom:updated>2024-02-14T10:17:07.492Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Updated Feb 14: Added details of IKE reauth/rekeying and proposals.</em></p><p>If you happen to run an OPNsense firewall/router at home and you want to connect to a Google Cloud VPN, well, this is your guide to getting it up and working using VTI-based tunnel and BGP. OPNsense has a new IPSec Connections setup screen, so we will be using that.</p><p>You’ll probably want to make sure that you use a static IP at home, but most ISPs give out the same IP addresses for very long periods. You’ll also want to be sure that there is nothing in front of your OPNsense firewall except the public Internet (no double NAT etc).</p><p>We’ll also be dispensing some of the High Availability features of HA VPN, namely two tunnels (since we only have one ISP at home).</p><p>We’ll also use the following details:</p><ul><li>Your home network: 192.168.1.1/24</li><li>VPC networks on Google Cloud side: 10.142.68.0/23 (and others)</li><li>Google Cloud side ASN: 64512</li><li>OPNsense side ASN: 64513</li><li>We’ll take the default autogenerated BGP IP addresses, which in this case resulted in the following ones:<br>– Google Cloud side: 169.254.52.169<br>– OPNsense side: 169.254.52.170</li><li>The shared secret generated by Google Cloud VPN</li></ul><h3>Creating the Google Cloud VPN</h3><p>We’re not covering this topic in this guide, as there are tons of guides for setting this up existing already. The main point is that we’ll be only using HA VPN with only one interface.</p><h3>Setting up OPNsense</h3><p>On the Firewall &gt; Rules &gt; WAN page, allow ESP and UDP traffic required for IPSec:</p><ul><li>Protocol ESP</li><li>UDP Traffic on port 500 (ISAKMP)</li><li>UDP Traffic on port 4500 (NAT-T)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9-uUbDdPwlLHKTJ6AuaWJw.png" /><figcaption>Firewall rules for inbound IPSec</figcaption></figure><p>Enable IPSec on OPNsense, it’s a tiny checkbox in the lower corner of the IPSec Connections screen:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k0x8YfynLft9liaqqkZceA.png" /><figcaption>Enabling IPSec.</figcaption></figure><p>First, let’s set up the Pre-shared keys, as we’ll be needing those shortly. Go to IPSec &gt; Pre-shared keys and add the keys:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6HHGJRcH0pAbtG7V69GvVw.png" /><figcaption>Creating the shared secret.</figcaption></figure><p>Now we can create the connection in IPSec &gt; Connections [new], starting with the first step:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ny5t0hKRZLDeTYHxVl9Z9Q.png" /><figcaption>Remember to add the description!</figcaption></figure><p>Update Feb 14: To prevent the tunnel from disconnecting occassionally and not reconnecting, I needed to set the following parameters (turn on Advanced mode on the top as you can see in the picture above): Re-auth time (s): 36000, Rekey time (s): 10800m, Proposals: aes256-sha512-modp2048 [DH14] (select only this).</p><p>Now add the Child tunnel by clicking the plus button under Children. We’ll use reqid 10, which corresponds to ipsec10interface. Please remember to uncheck Policies and set the description:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lw1whV9ZgIvkNdPlrrbdgQ.png" /><figcaption>Adding child tunnel. Remember uncheck policies and enter a description.</figcaption></figure><p>Save and then add the previously added pre-shared key to Local and Remote authentication (there is nothing to configure as the pre-shared key already has local and remote identifiers):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-rsRVHTe5tUtE9zLj9hzcA.png" /></figure><p>It should look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lpu9q3YBXxXkl_fhYttr4g.png" /><figcaption>Pre-shared keys.</figcaption></figure><p>Finally save. Next we’ll add a virtual interface under IPSec &gt; Virtual Tunnel Interfaces:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TJBNMBnpClOlhDxz8iZBZw.png" /><figcaption>Adding a virtual tunnel interface.</figcaption></figure><p>Now, if you check the IPSec &gt; Status Overview screen, you should see a connection established:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vTL7CE8Ghe6tIsNF_wRnxg.png" /><figcaption>The connection status is UP!</figcaption></figure><p>You can also try pinging the Google Cloud side IP from either OPNsense console or GUI:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dHUmIP8z5vd-_-MAtUcjmA.png" /></figure><p>At this point, all that is left is setting up BGP routing. First we’ll go to Routing &gt; General and make sure routing is enabled:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ONpiakh1tXpOqvESwYnpiw.png" /><figcaption>Routing settings.</figcaption></figure><p>Then we’ll go to Routing &gt; BGP and set up the basic details:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h7cDwdS3-sTQQjVxGHoF2Q.png" /></figure><p>Then on the neighbors page, add the Google Cloud side:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D42uouMPzCYew4unGh33Jg.png" /><figcaption>Adding BGP neighbor.</figcaption></figure><p>Please note that you don’t have to set prefix lists unless you want/need to (for example you are announcing a 0.0.0.0/0 route from GCP side).</p><p>That’s it, we can now try pinging an arbitrary VM on GCP side from any local network machine:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*SoTBdfAClT-zUNUb8Tcphg.png" /><figcaption>Ping successful!</figcaption></figure><p>That’s it. Now we have a VPN tunnel to GCP through OPNsense.</p><p>If you have set up <a href="https://cloud.google.com/vpc/docs/configure-private-service-connect-apis">Private Service Connect for Google APIs</a> or Private Google Access and are using Unbound on OPNsense, you can also create an override for googleapis.com to use the VPN connection:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rCmj7rzH_9mEWPsA-NtZSg.png" /><figcaption>Redirect *.googleapis.com to Private Service Connect endpoint on GCP.</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=206f6fbe1833" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an AI Slack Bot using Vertex Generative AI]]></title>
            <link>https://medium.com/google-cloud/building-an-ai-slack-bot-using-vertex-generative-ai-d5f2c9e5e0b0?source=rss-6c5d20f5d1ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/d5f2c9e5e0b0</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[vertex-ai]]></category>
            <category><![CDATA[generative-ai]]></category>
            <category><![CDATA[cloud-functions]]></category>
            <dc:creator><![CDATA[Taneli Leppä]]></dc:creator>
            <pubDate>Sat, 13 May 2023 08:37:12 GMT</pubDate>
            <atom:updated>2023-11-29T10:42:39.073Z</atom:updated>
            <content:encoded><![CDATA[<p>Last time we <a href="https://taneli-leppa.medium.com/creating-a-google-cloud-deploy-bot-on-google-chat-17601fdaf4d6">built a Google Cloud Deploy bot</a> using <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox">Pubsub2Inbox</a> and <a href="https://cloud.google.com/deploy">Cloud Deploy</a>. This time we’ll switch gears a little and build a completely serverless <a href="https://api.slack.com/automation/quickstart">Slack bot</a> that uses <a href="https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview">Vertex AI Generative Studio</a> models to add a bit of smarts to the bot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pLClfLoaxOI8744fMWrbmA.png" /></figure><p>To enable Slack bot building, I’ve built three new components for Pubsub2Inbox:</p><ul><li><a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/blob/main/output/slack.py">Slack output</a>, for sending messages,</li><li><a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/blob/main/processors/slack.py">Slack processor</a>, for fetching existing messages and conversations,</li><li><a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/tree/main/cmd/json2pubsub">Json2Pubsub</a>, a tool for taking all kinds of webhooks, verifying them and posting the results into a Pub/Sub topic. It can perform Slack validation using the HMAC SHA256 algorithm and it can be automatically deployed alongside Pubsub2Inbox <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox/blob/main/variables.tf#L186">using Terraform.</a></li></ul><h3>Adding an app (a bot) to Slack</h3><p>To set up a Slack app, we’ll start off in api.slack.com and add a new app there. We’ll need to jot down a few details, such as tokens and signing keys, plus set up the event webhook.</p><p>First, under basic information, we’ll need to capture the signing secret (this is used to verify that incoming requests to our Json2Pubsub webhook are coming from Slack actually):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XaxP5-_0_8ht-u_dkBHyIw.png" /></figure><p>Then we will need the bot token (can be found in OAuth &amp; Permissions section):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0bZ2nnk_184wlfG-MuIYzw.png" /></figure><p>We’ll also make sure that our bot has the correct OAuth scopes for the token, so it can read messages and reply back:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CNgk_exEuYU8aW6DwKLwfQ.png" /></figure><p>We’ll configure the Events webhook after we’ve deployed the code.</p><h3>Deploying via Terraform</h3><p>First, clone the <a href="https://github.com/GoogleCloudPlatform/pubsub2inbox">Pubsub2Inbox repository</a>. You’ll find the example code under examples/vertexgenai-slack-bot/. Change to that directory and create a terraform.tfvars file containing the following:</p><pre>project_id        = &quot;&lt;your-project-id&gt;&quot;<br>region            = &quot;europe-west1&quot;<br>slack_token       = &quot;xoxb-............................MB&quot;<br>slack_signing_key = &quot;8e..........................2f&quot;<br>slack_app_id      = &quot;A0.......L&quot;</pre><p>You also have a few additional settings in variables.tf, such as creation of a custom role with vastly smaller permissions (just one) than the default Vertex AI User IAM role, but that is on by default.</p><p>After this, all you need to do is:</p><pre>terraform init<br>terraform apply<br>....<br>Apply complete! Resources: 3 added, 2 changed, 3 destroyed.<br><br>Outputs:<br><br>instructions = &lt;&lt;EOT<br>Please use the following webhook URL for Slack event webhook:<br>https://vertex-genai-bot-json2pubsub-abcdefghijk-lm.a.run.app</pre><p>It outputs the Json2Pubsub webhook address, which is already made publicly accessible (it will verify that all requests sent to it are done by Slack and it only has Pub/Sub publisher to the specific topic on Google Cloud).</p><p>You can also check that you have two Cloud Functions deployed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oMeS3jl4mZJrcRviq5GvqQ.png" /></figure><p>We can now configure the webhook in api.slack.com under Event subscriptions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4eHbMutnUdo6yq3jQ_XKkA.png" /></figure><p>That’s it, we can now add the Slack bot to our channel:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ja-ST5QjAHYM_OBS1cGaRg.png" /></figure><p>If all went well, you can just mention the bot on the channel and ask it a question:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*39eMsuEsem9QrdfZIfDL-w.png" /></figure><p>You can check the slack-bot.yaml that is in the same directory as the Terraform code to modify any of the Vertex AI settings or do other things.</p><p>If you are not getting a proper reply, look at the Cloud Functions logs for clues.</p><p>We’ll discuss a few technical details next.</p><h3>Receiving Slack events</h3><p>Whenever the bot is mentioned one a public channel, a message is sent to Json2Pubsub, which validates that it’s coming from Slack and submits it to the Pub/Sub topic, allowing Pubsub2Inbox to pick it up and send a reply. (Please note that the bot also gets its own messages as events, but those have an app_id field in the event structure)</p><p>The following is an example of the event:</p><pre>{<br>    &quot;api_app_id&quot;: &quot;A056L1L8RFL&quot;,<br>    &quot;authorizations&quot;: [<br>        {<br>            &quot;enterprise_id&quot;: null,<br>            &quot;is_bot&quot;: true,<br>            &quot;is_enterprise_install&quot;: false,<br>            &quot;team_id&quot;: &quot;T156KL1RBJF&quot;,<br>            &quot;user_id&quot;: &quot;U056HF3L53P&quot;<br>        }<br>    ],<br>    &quot;event&quot;: {<br>        &quot;blocks&quot;: [<br>            {<br>                &quot;block_id&quot;: &quot;cBH&quot;,<br>                &quot;elements&quot;: [<br>                    {<br>                        &quot;elements&quot;: [<br>                            {<br>                                &quot;type&quot;: &quot;user&quot;,<br>                                &quot;user_id&quot;: &quot;U056HE3L13P&quot;<br>                            },<br>                            {<br>                                &quot;text&quot;: &quot; Hello World!&quot;,<br>                                &quot;type&quot;: &quot;text&quot;<br>                            }<br>                        ],<br>                        &quot;type&quot;: &quot;rich_text_section&quot;<br>                    }<br>                ],<br>                &quot;type&quot;: &quot;rich_text&quot;<br>            }<br>        ],<br>        &quot;channel&quot;: &quot;C056KQ1TLUA&quot;,<br>        &quot;client_msg_id&quot;: &quot;2a2f0dd6-94d9-4bd2-a4e1-2642bb928aa7&quot;,<br>        &quot;event_ts&quot;: &quot;1683890230.468149&quot;,<br>        &quot;team&quot;: &quot;T056KL1RAJF&quot;,<br>        &quot;text&quot;: &quot;\u003c@U056HE3L13P\u003e Hello World!&quot;,<br>        &quot;ts&quot;: &quot;1683890230.468149&quot;,<br>        &quot;type&quot;: &quot;app_mention&quot;,<br>        &quot;user&quot;: &quot;U0579EPJN72&quot;<br>    },<br>    &quot;event_context&quot;: &quot;4-ey...In0&quot;,<br>    &quot;event_id&quot;: &quot;Ev057E3B27FF&quot;,<br>    &quot;event_time&quot;: 1683890230,<br>    &quot;is_ext_shared_channel&quot;: false,<br>    &quot;team_id&quot;: &quot;T056KL1RAJF&quot;,<br>    &quot;token&quot;: &quot;7n...........N&quot;,<br>    &quot;type&quot;: &quot;event_callback&quot;<br>}</pre><p>The bot picks up next messages and looks up the threaded conversation history using conversations.replies Slack API, then processes the messages in a suitable format for Vertex AI and then performs the prediction request. The reply is posted back in the thread and you can continue the conversation normally (please note some replies can be cut off with max tokens setting):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*FhWqCCeINDhMLxJPp6ANyA.png" /></figure><h3>How does the bot work?</h3><p>The bot pipeline in Pubsub2Inbox comprises of several steps:</p><ul><li>Message parsing: removing nickname and Slack mrkdwn for the message</li><li>Slack conversation history: fetches the history of a threaded conversation from Slack API and converts it to an array suitable for Vertex AI consumption</li><li>Calling Vertex AI chat model prediction</li><li>Posts the reply from Vertex AI back to the conversation via Slack API</li></ul><p>The Vertex requests looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kWcp73TZnHMNkm-GWd61aA.png" /></figure><p>Context contains the “prompt” and messages the chat history. You can customize the prompt by editing the slack-bot.yaml file.</p><p>Did it work? Do you have an issue or want to send feedback? Reach out to me on Twitter for example: <a href="https://twitter.com/rosmo">https://twitter.com/rosmo</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d5f2c9e5e0b0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/building-an-ai-slack-bot-using-vertex-generative-ai-d5f2c9e5e0b0">Building an AI Slack Bot using Vertex Generative AI</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>