<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Cody Zuschlag on Medium]]></title>
        <description><![CDATA[Stories by Cody Zuschlag on Medium]]></description>
        <link>https://medium.com/@codyzus?source=rss-4b8a203df491------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*fwcZ0OAofbOQ0LwwZDPXvw.jpeg</url>
            <title>Stories by Cody Zuschlag on Medium</title>
            <link>https://medium.com/@codyzus?source=rss-4b8a203df491------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 11:03:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@codyzus/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Cloud Firestore geo queries]]></title>
            <link>https://medium.com/@codyzus/cloud-firestore-geo-queries-a0e095b781c6?source=rss-4b8a203df491------2</link>
            <guid isPermaLink="false">https://medium.com/p/a0e095b781c6</guid>
            <category><![CDATA[firebase]]></category>
            <category><![CDATA[firestore]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[geospatial]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Cody Zuschlag]]></dc:creator>
            <pubDate>Tue, 12 May 2020 16:20:35 GMT</pubDate>
            <atom:updated>2020-05-31T13:44:10.402Z</atom:updated>
            <content:encoded><![CDATA[<h3>Cloud Firestore Geo Queries</h3><p>A voyage through the options for querying Cloud Firestore by geographic proximity or map bounds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WurogVO44LAUnqT3" /><figcaption>Photo by <a href="https://unsplash.com/@dead____artist?utm_source=medium&amp;utm_medium=referral">Capturing the human heart.</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>I teach a web development university course. During the course, students build an Airbnb like web application with Firebase. Every year there is one technical challenge that gets these new devs hung up when using Cloud Firestore.</p><p>How to query for nearby geo-coordinates in Cloud Firestore? The use case is always the same, display a map, and show all of the rooms for rent either around the center of the map or inside the entire bounds of the visible map.</p><p>This article presents the same solutions that I present to my students:</p><ol><li><strong>KISS and forget:</strong> just get the entire collection and filter in memory</li><li><strong>1-D query:</strong> limit the query by the longitude and then filter by latitude in memory</li><li>Use <a href="https://en.wikipedia.org/wiki/Geohash"><strong>geohashes</strong></a>!</li><li>Use a 3rd party search tool that supports geo-searching, like <a href="https://www.algolia.com/"><strong>Algolia</strong></a></li></ol><p>There is no perfect solution. Like most programming problems, the perfect solution depends on the specifics of your project, your users, and your business constraints.</p><p>Let’s take them 1 by 1…</p><h3>KISS and forget (about scaling)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KmW-4nVn0NSTkycv" /><figcaption>Photo by <a href="https://unsplash.com/@joanna_nix?utm_source=medium&amp;utm_medium=referral">Joanna Nix</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Being a developer for over 13 years, 8 of them using Agile methodologies, you learn to balance quality, complexity, performance, and functionality. This means weighing every feature or line of code against the value it will bring to your users and the time it will take to implement and maintain.</p><p>The <a href="https://en.wikipedia.org/wiki/KISS_principle">KISS principle</a> tells us the simplest solution is always the best solution. Why not just download the entire collection and filter by longitude and latitude in memory? Until you have thousands of documents, the performance costs will be negligible.</p><h4>Bonus 🚀</h4><p>There is an easy performance improvement too; if the user enters a city or postal code we can optimize by adding a city or postal code constraint to the Cloud Firestore queries. This optimization certainly isn’t ideal for an application with a map that can pan anywhere in the world, but it is a viable solution that should be considered because of its simplicity.</p><p>This is what I tell my students: until you have thousands of documents to query from Cloud Firestore, even a slow portable phone can work through an array of a thousand documents in memory. Never forget the simple solution. The <a href="https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it">YAGNI principle</a> tells us to add functionality when you actually need it. I consider this solution the perfect starting point.</p><h3>1-D query</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pIZQ6IPrbLWkmjL0" /><figcaption>Photo by <a href="https://unsplash.com/@ilariopiatti?utm_source=medium&amp;utm_medium=referral">Ilario Piatti</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>We can take our KISS solution and make an easy performance enhancement. Cloud Firestore will let us use less-than and greater-than operators on a single document field in a complex query. Therefore, we can include the minimum and maximum longitude visible on the map in the Cloud Firestore query. Once the results are fetched from Cloud Firestore, we can then filter the latitude in memory.</p><p>Depending on your data, this could give you approximately a 50% performance and memory consumption improvement over the KISS solution. Boom 💥! This represents a great compromise solution before exploring more complex solutions.</p><p>Once again, if we follow <a href="https://en.wikipedia.org/wiki/KISS_principle">KISS</a> and <a href="https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it">YAGNI</a> principles, this solution can be a step along the way to a more complex and scalable solution.</p><h3>Geohashes</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qMGb7cENy66kxkSP" /><figcaption>Photo by <a href="https://unsplash.com/@jontyson?utm_source=medium&amp;utm_medium=referral">Jon Tyson</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This is the most complex solution. The reward for that complexity is a robust and scalable solution without requiring any 3rd party search services.</p><p>First, what is a <a href="https://en.wikipedia.org/wiki/Geohash">geohash</a>?</p><p>It’s a fancy way to encode longitude and latitude (don’t worry there’s an <a href="https://www.npmjs.com/package/ngeohash">excellent npm package</a> that takes care of the heavy lifting). When treating longitude and latitude as a grid, encoding with geohashes lets us control the size of the grid rectangles with the length of the hash (the longer the hash, the smaller the rectangle). Furthermore, it is encoded in a way that the closer two rectangles are on the earth, the more likely they will have similar geohashes.</p><p>OK, so how does that help?</p><p>Using some math and approximation, we can draw big rectangles around a geographic point. Then we can query Cloud Firestore for documents that have hashes inside those rectangles. This means that in the worst case, you will need to do 9 queries to Cloud Firestore to fetch all of the documents to populate the map. But there is good news since the hashes are likely similar. If we get lucky, some of them are consecutive and we can query Cloud Firestore with a range to reduce the number of required queries. Often we can optimize down to 4 to 6 range querries.</p><p>Geohashes come with baggage 🛅: being a grid that is based on longitude and latitude, the rectangles are neither square nor regular in their geometric area. The area of rectangles gets smaller the farther they are from the equator. All of that means that we generally do a lot of approximation (coding for the worst case). The result is that in most cases you will query for a substantially larger area than will be shown to the user.</p><p>The good news is that geohashes is a proven technique and is natively supported by several databases (see <a href="https://docs.mongodb.com/manual/core/geospatial-indexes/">MongoDB spatial indexes</a>, <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-point.html">ElasticSearch geo_point</a>, and <a href="https://redis.io/commands/geohash">Redis GeoHash</a>). While we need to write some code to use geohashes, the result is perfectly compatible with Cloud Firestore.</p><h3>Algolia and other 3rd party search services</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Yjlw_fK2rYMK4_8-" /><figcaption>Photo by <a href="https://unsplash.com/@taylorvanriper925?utm_source=medium&amp;utm_medium=referral">Taylor Van Riper</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>I call this the most “cloud-first” solution. Why roll our own solution when we can pay someone else to use their service? <a href="https://www.algolia.com/">Algolia</a> lets us do <a href="https://www.algolia.com/doc/guides/managing-results/refine-results/geolocation/">just that</a>. Plus, just like Firebase, the generous monthly free usage is most likely enough to get your app off the ground and in the hands of users without any out of pocket costs 💲!</p><p>It works like this… you add a Cloud Function that will keep Algolia synchronized with the data in Cloud Firestore that you want to be able to geo search. The <a href="https://firebase.google.com/docs/functions/firestore-events">Cloud Function gets triggered</a> whenever a document containing geo coordinates is written (or deleted). In turn, it ensures that Algolia gets synched with the change.</p><p>In your web application, you can directly <a href="https://www.algolia.com/doc/guides/managing-results/refine-results/geolocation/">query Algolia</a> using either a radius around the center point of the map or a with the bounding box of the map. Algolia takes care of all of the hard work! Plus you can add enough data into the search results that you don’t need to query Cloud Firestore at all while the map is panning and zooming 🔍.</p><p><strong>This solution is fast, scales, and is much simpler than geo-hashes.</strong></p><p>So what are the downsides of Algolia?</p><p>The first point is subjective: you will be relying on yet another service provider for your application. Sure the free plan is more than generous enough to get you started, but it is one more provider to monitor and potentially pay for if your app starts to have success (there are worse problems to have 😉).</p><p>The second point is that to use a Cloud Function to keep Algolia synchronized with Cloud Firestore, you will need to pull out your credit card 💳 and sign up for the <a href="https://firebase.google.com/pricing">Firebase Blaze plan</a> so that your Cloud Function can call APIs outside of Google Cloud. For that reason alone, I don’t encourage my students to go down this path. However, it is important to note that even after entering your credit card, you still get a generous amount of free outgoing network traffic (5GB at the time of writing). That means this solution probably won’t cost you a penny while you get your app off the ground.</p><h3>Conclusion</h3><p>The point that I try to make to the university students is that, as with most things in software development, (nearly) anything is possible. However, we all work with constraints (time, money, management, etc…).</p><p>While at first glance, many students are disappointed in the lack of built-in geo-searching capabilities of Cloud Firestore. However, upon deeper investigation, we find that all of the tools necessary to build a first-rate geo-based application are at our fingertips. If you want your Firebase application to mimic the functionality of AirBnB or Google Maps, you can do it.</p><p>Just don’t jump into the most complex solution without considering the simpler solutions first.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GCuiDTiem57LHXQc" /><figcaption>Photo by <a href="https://unsplash.com/@patrick_schneider?utm_source=medium&amp;utm_medium=referral">Patrick Schneider</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a0e095b781c6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lock down with Cloud Firestore]]></title>
            <link>https://medium.com/@codyzus/lock-down-with-cloud-firestore-61a98d1f4647?source=rss-4b8a203df491------2</link>
            <guid isPermaLink="false">https://medium.com/p/61a98d1f4647</guid>
            <category><![CDATA[firestore]]></category>
            <category><![CDATA[firebase-cloud-functions]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[firebase]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Cody Zuschlag]]></dc:creator>
            <pubDate>Wed, 06 May 2020 16:10:31 GMT</pubDate>
            <atom:updated>2020-05-06T19:23:28.262Z</atom:updated>
            <content:encoded><![CDATA[<p>Using Cloud Firestore transactions to create an atomic lock for Firebase Cloud Functions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5Xw8W2m9VMOkmblq" /><figcaption>Photo by <a href="https://unsplash.com/@art_maltsev?utm_source=medium&amp;utm_medium=referral">Artem Maltsev</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Deploying a hello world serverless function is easy. However, making your deployments stable and production worthy is hard. Serverless functions should be<a href="https://cloud.google.com/functions/docs/bestpractices/tips#write_idempotent_functions"> stateless and idempotent</a>: they should be written so that they can be triggered multiple times without exploding. We can create a lock with Cloud Firestore to ensure that even if our Firebase Cloud Functions are triggered multiple times, only one invocation will be allowed to execute.</p><p>In this article, I will show you how to use Cloud Firestore to create a lock that ensures only one invocation of your Firebase Cloud Function is allowed to execute for a given input or event. For this example, I will assume you are using the <a href="https://firebase.google.com/docs/functions/schedule-functions">schedule functionality</a> of Firebase Cloud Functions. However, the example can be adapted to functions with different types of triggers.</p><p><strong>Using this technique, we ensure our function executes exactly once for a scheduled event.</strong></p><h3>The code.</h3><p>First, the code for the Firebase Cloud Function:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/57ea8b5b09ca4e70f84b98860292353c/href">https://medium.com/media/57ea8b5b09ca4e70f84b98860292353c/href</a></iframe><p>👆 This function executes every hour and uses Cloud Firestore to ensure only a single invocation will ever be allowed to execute. Enjoy the idempotent serverless bliss…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jgDe4c7f3aQFh1uk" /><figcaption>Photo by <a href="https://unsplash.com/@dariusbashar?utm_source=medium&amp;utm_medium=referral">Darius Bashar</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>How does it work?</h3><p>The code uses a Cloud Firestore document to create a lock 🔒. Locks, especially <a href="https://en.wikipedia.org/wiki/File_locking#Lock_files">lock files</a>, have been used by applications for ages. Technically, Cloud Functions do have limited access to the file system. However, since Cloud Functions are auto-magically scaled and possibly deleted and re-initialized, we can’t assume two executions cycles have access to the same file system.</p><p>Fortunately, we can use the database to create a lock. A lock requires a mechanism called an atomic <a href="https://en.wikipedia.org/wiki/Read%E2%80%93modify%E2%80%93write">read-modify-write</a> operation. In short, the operating system or database guarantees that no other process can access the lock between the read and the write operations. In SQL this can be accomplished with an <a href="https://en.wikipedia.org/wiki/Merge_(SQL)">upsert or merge</a> operation that either inserts or updates a row in a table.</p><p><strong>Cloud Firestore does not have upserts. Fortunately, we <em>can</em> use a </strong><a href="https://firebase.google.com/docs/firestore/manage-data/transactions#transactions"><strong>transaction</strong></a><strong> to achieve the same results.</strong></p><h4>Unique IDs 🛂</h4><p>The algorithm needs two unique ids:</p><ol><li><strong>event id:</strong> a unique identifier for the event or input that we want to service exactly once</li><li><strong>execution id:</strong> a unique identifier for the current invocation of the Cloud function</li></ol><p>The <strong>event id</strong> is a way to identify the unique event that is being serviced by the Cloud Function. If we are using Firebase Cloud Functions <a href="https://firebase.google.com/docs/functions/schedule-functions">schedule functionality</a>, the eventId field of the <a href="https://firebase.google.com/docs/functions/pubsub-events#access_message_attributes">PubSub context</a> is the perfect identifier 💥. Other solutions might include a hash of the input parameters.</p><p>Since the Cloud Function may get triggered multiple times for the same external event or input (the problem we are solving), each invocation needs to be identified with the <strong>execution id</strong>. There are several libraries available on npm for generating unique ids; I’m using one of my favorites: <a href="https://www.npmjs.com/package/nanoid">nanoid</a>.</p><h4>The algorithm</h4><ol><li>Generate a unique execution id for the current invocation of the function</li><li>Start a Cloud Firestore <a href="https://firebase.google.com/docs/firestore/manage-data/transactions#transactions">transaction</a> for the steps that follow…</li><li>Check if a lock document for the event id already exists</li><li>If it exists, return true if it was created with same execution id otherwise false (indicating it was created by another invocation of the function)</li><li>If it does not exist, attempt to create the document and set the execution id</li><li>If the document was created successfully, return true</li></ol><p><strong>The magic happens because </strong><a href="https://firebase.google.com/docs/firestore/manage-data/transactions#transactions"><strong>Cloud Firestore guarantees</strong></a><strong> that if a concurrent process modifies the read document before the write completes, the transaction is retried.</strong></p><h3>When to use it?</h3><p>I use this pattern whenever I use <a href="https://firebase.google.com/docs/functions/schedule-functions">Firestore’s incredible schedule functionality</a> for Cloud Functions. The schedule abstracts over the complexities of using a Google Cloud Scheduler and Google Cloud Pub/Sub. However, the function may be called multiple times for the same scheduled event! For example, this will happen if your Cloud Function takes more time to complete and return success than the default Pub/Sub ack timeout. Since Firestore does not (currently) allow for customizing the Pub/Sub ack timeout, this is the best solution that I have found. It ensures that possibly long-running scheduled functions don’t explode 💥.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-oO_P7m-lsQQMiMT" /><figcaption>Photo by <a href="https://unsplash.com/@spacex?utm_source=medium&amp;utm_medium=referral">SpaceX</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Going further.</h3><p>This simple example will leave the old lock documents indefinitely in Cloud Firestore. Add a comment below if you are interested in a future article about how to sanely cleanup left-over lock documents.</p><p>So much can be done with the schedule functionality of Firebase Cloud Function: database maintenance, user cleanup, file conversations, etc. Leave a comment below if you want to see an article exploring some of the possibilities!</p><p>I hope you found this useful. Let me know in the comments if you’ve used this or another method to keep your serverless cloud functions idempotent!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=61a98d1f4647" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Continuous Deployment for Serverless Google Cloud Functions]]></title>
            <link>https://medium.com/chatbots-blog/continuous-deployment-for-serverless-google-cloud-functions-175b0756be28?source=rss-4b8a203df491------2</link>
            <guid isPermaLink="false">https://medium.com/p/175b0756be28</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[continuous-delivery]]></category>
            <category><![CDATA[continuous-integration]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[nodejs]]></category>
            <dc:creator><![CDATA[Cody Zuschlag]]></dc:creator>
            <pubDate>Tue, 27 Mar 2018 07:44:39 GMT</pubDate>
            <atom:updated>2018-03-27T17:14:17.780Z</atom:updated>
            <content:encoded><![CDATA[<p>If you are not doing continuous deployment, then you are doing software wrong (more on that later). Building a continuous deployment pipeline for Google’s serverless Cloud Functions is easy and will help you deploy small, incremental changes often. This can help avoid bugs, reduce surprise delays, and keep all stakeholders aligned with realistic expectations.</p><h3>tl;dr</h3><p>The diagram below illustrates the continuous deployment pipeline for Google Cloud Functions. It consists of 4 steps:</p><ol><li>Changes are committed, pushed (and possibly tagged) to your hosted git repository.</li><li>The continuous integration service runs the test suite on every commit.</li><li>Depending on your workflow, the CI service redeploys the Cloud Function using the GCloud SDK. This could be on every commit on the master branch or on every semver tag.</li><li>Since the Cloud Function is deployed from a Google Cloud Source Repository (configured to mirror a Github or Bitbucket repository), a single, simple gcloud command redeploys the function.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ztOP9Ch4v3_Phh3E32cgjA.png" /><figcaption>Cloud Function continuous deployment pipeline</figcaption></figure><h3>Step by Step</h3><p>Here is a step by step guide to building your pipeline.</p><h4>Step 1: Tests</h4><p>The key to the success of the pipeline lies in the tests. Unit tests, integration tests, etc… whatever gives developers the confidence that they can make changes to the code and be sure that when the tests pass there will be no catastrophic failures. Bugs are a fact of developing software. The goal of the tests is to ensure that there are no major regressions, all new functionality works as intended, and that no catastrophic bugs were introduced.</p><p><a href="https://blog.hubwa.re/dont-nock-our-integration-tests-fa9682ba7155">Don’t nock our integration tests</a></p><p>For higher level integration tests, don’t forget that Google provides an emulator for running the function locally. This could be easily configured to run in a continuous integration step.</p><p><a href="https://cloud.google.com/functions/docs/emulator">Cloud Functions Node.js Emulator | Cloud Functions Documentation | Google Cloud</a></p><h4>Step 2: Deploy your function (the right way)</h4><p>When you deploy your cloud function, deploy it from a <a href="https://cloud.google.com/source-repositories/">cloud source repository</a> that mirrors your Github (or Bitbucket) repository. Google Cloud Functions offer several ways to deploy functions, including packaging a zip and uploading it to Cloud Storage. However, this will result in the need to pay for and delete unused Cloud Storage. By deploying from a mirrored source repository, the Cloud Function will pull the source directly from a private git repository (that mirrors Github or Bitbucket).</p><p>Deploy the first time manually in the Google Cloud console UI. That way you can be sure your function starts and that the source repository is correctly configured. The CI tool will then redeploy the same Cloud Function.</p><h4>Step 3: Configure your Continuous Integration service</h4><p>At Hubware we use Codeship. However, whatever service you use (Jenkins, Codeship, CircleCI, etc.) you should be able to trigger builds based on commits or tags. We run the test suite on all commits. A separate build step redeploys the Cloud Function <em>only</em> when there is a new tag that resembles a <a href="https://semver.org/">semver</a> string, such as v1.0.7. This provides a nice mechanism to control when the Cloud Function is redeployed. Alternatively, you could configure the redeploy step to be triggered on commits to the master branch.</p><p>To accomplish the redeployment, we can use the <a href="https://cloud.google.com/sdk/gcloud/reference/alpha/functions/deploy">GCloud SDK</a> to issue the deploy command. It is a good idea to test your command locally after setting or replacing the environment variable:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c3dd71fd5b300391a3c84c3b287898f5/href">https://medium.com/media/c3dd71fd5b300391a3c84c3b287898f5/href</a></iframe><p>Running the command from your CI service will depend on your service. Most services offer dockerized builds and that is generally the easiest route. Codeship offers a <a href="https://documentation.codeship.com/pro/continuous-deployment/google-cloud/">customized docker image</a> for deploying to Google Cloud when using their pro offering. Simply set some encrypted environment variables and you can call the gcloud command directly from a build step.</p><h4>Step 4: Enjoy Continuous Deployment</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5izCoteagROfYODvbD39KQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/UErWoQEoMrc?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Wil Stewart</a> on <a href="https://unsplash.com/search/photos/cheers?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h3>Discussion</h3><p>Now that we have shared our quick continuous deployment pipeline for Cloud Functions we can discuss why this was important and what is still missing.</p><h4>Why serverless?</h4><p>Going serverless has been a temptation that has been flirting with our dev team for a while. Imagine not having to maintain any servers or even the Kubernetes cluster? Imagine if scaling occurred dynamically. That’s what serverless promises to do. One more step towards the ultimate goal of just putting your code into production without any intermediate steps. One more step towards development utopia.</p><blockquote>[Serverless is] one more step towards the ultimate goal of just putting your code into production without any intermediate steps.</blockquote><blockquote>At the end of the day, as a developer, you really just want to just kind of check your code in and expect that code to land in front of your customers somehow, someway. — <strong>Kelsey Hightower, </strong><a href="https://www.redhat.com/en/command-line-heroes/the-containers-derby"><strong>Command Line Heroes episode 5</strong></a></blockquote><p>Let’s be honest, <strong>Google Cloud Functions is in Beta</strong>. Decide for yourself if you are OK with that. We accepted that fact because the development utopia that Cloud Functions promise is simply irresistible.</p><blockquote>This API might be changed in backward-incompatible ways and is not subject to any SLA or deprecation policy. — <a href="https://cloud.google.com/functions/docs/"><strong>Google Cloud Functions Documentation</strong></a></blockquote><p>What was missing for us was a clear path to creating production worthy cloud functions. Specifically, we wanted the same confidence in our serverless development pipeline that we have with our core node.js + Docker + Kubernetes development.</p><h4>Continuous deployment now</h4><p>If you are not building your software using continuous deployment, then you are doing it wrong. The train has left the station, the ship has sailed.</p><blockquote>Today, if you’re waiting longer than 10 minutes to have something provisioned, then you’re doing something wrong. —<strong> </strong><a href="https://www.redhat.com/en/command-line-heroes/devops-tear-down-that-wall"><strong>Command Line Heroes Podcast episode 4</strong></a></blockquote><blockquote>Programmers don’t burn out on hard work, they burn out on change-with-the-wind directives and not ‘shipping’. <strong>— </strong><a href="https://twitter.com/markab"><strong>Mark Berry</strong></a></blockquote><p>Of course, adopting a continuous deployment pipeline in an organization can be difficult. In my experience, the biggest barrier to adopting continuous deployment is psychological. Developers, product owners, and managers all share the same fear of releasing bugs. However, that fear can be mitigated by strong test suites that give developers (and everyone else) confidence that catastrophic bugs won’t slip through. Also, having a process to quickly “undo” or “rollback” a change that has gone wrong is an important safety hatch to ease everyone&#39;s fears and start doing continuous deployment now.</p><blockquote>In my experience, the biggest barrier to adopting continuous deployment is psychological.</blockquote><h4>Goodbye server, hello serverless</h4><p>Back to serverless… at Hubware, the serverless temptation was irresistible. However, rather than throw away our servers and Kubernetes clusters, we decided to incrementally add serverless functionality. Little by little, we have been adding serverless functionality using Google’s offering: <a href="https://cloud.google.com/functions/">Cloud Functions</a>. It started with non-critical functionality, like synchronizing data between services and regular maintenance tasks performed with cloud cron jobs.</p><p><a href="https://blog.hubwa.re/google-cloud-platform-cron-jobs-sourced-from-kubernetes-33a4f6046e3f">Google Cloud Platform cron jobs sourced from Kubernetes</a></p><p>As long as we were quickly notified by our monitoring services or error reporting that a cron job failed, we could quickly fix the problem without any negative impact on our customers. But once you taste the serverless utopia, you want more! And that meant deploying some of our mission-critical functionality in serverless. However, there was a problem…</p><h4>Unfulfilled serverless promises</h4><p>How can we have the same development and business process bliss, that continuous deployment brought to our primary development, in the serverless domain?</p><p>The tools and processes to bring continuous deployment to serverless are much less mature. Furthermore, since there is no obvious path to continuous deployment with serverless, there is so much temptation to just skip creating the pipeline completely. We gave into this temptation at first. However, the first production bug and wild-west refactor (<em>I hope I didn’t just break something else while fixing this bug</em>) quickly reminded us of how our continuous deployment pipeline has squashed everyone&#39;s fears while developing our primary services.</p><blockquote>...the first production bug and wild-west refactor (<em>I hope I didn’t just break something else while fixing this bug</em>) quickly reminded us of how our continuous deployment pipeline has squashed everyone’s fears while developing our primary services</blockquote><p>That is when we decided we needed a continuous deployment pipeline for our serverless services. The key to the success of the pipeline lies in the tests. Unit tests, integration tests, etc… whatever gives developers the confidence that they can make changes to the code and be sure that if the tests pass and there will be no catastrophic failures.</p><p>The tests only give us continuous integration. To have a continuous deployment pipeline, we need to trigger the deployment of the cloud function after all tests have passed. That was the motivation for building our continuous deployment pipeline.</p><h3>What’s missing?</h3><h4>Dependency lockdown</h4><p>There is another big problem in having production-grade cloud functions: repeatable installs. Google does not document how dependencies are installed. However, after <a href="https://stackoverflow.com/a/48928831/880509">some discussion</a> with the Google Cloud Functions Beta Testers group, I was able to confirm that they do not respect npm-shrinkwrap.json, package-lock.json, or yarn.lock files 😢.</p><p>This could be a deal breaker for some. However, I filed a bug report and it sounds like Google will be fixing this deficiency soon (fingers crossed!). Please <a href="https://issuetracker.google.com/issues/73717432">go and star the bug</a> to show your support.</p><p>There are some alternatives that have been discussed, mostly involving packaging dependencies with the deployment and deploying to a Google Cloud Storage bucket with all the dependencies included. However, this seems heavy and adds some new complexity to the build pipeline. Deploying from source eliminates the need to manage Cloud Storage buckets and makes it simpler to deploy. And simpler is better.</p><h4>Canary testing</h4><p>Something else that is missing from this pipeline is canary testing. We plan to explore this subject in the near future. You can look forward to a future blog post on this topic.</p><p>Even though serverless Google Cloud Functions are only in beta, they bring some huge advantages to creating software: no server or cluster maintenance and automatic scaling. Now that serverless is a serious development platform, it’s time that we start developing seriously for serverless. It’s time for continuous deployment.</p><p>Leave your questions and comments below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=175b0756be28" width="1" height="1" alt=""><hr><p><a href="https://medium.com/chatbots-blog/continuous-deployment-for-serverless-google-cloud-functions-175b0756be28">Continuous Deployment for Serverless Google Cloud Functions</a> was originally published in <a href="https://medium.com/chatbots-blog">Blog d&#39;Hubware sur l&#39;Assistance Professionnelle</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Google Cloud Platform cron jobs sourced from Kubernetes]]></title>
            <link>https://medium.com/chatbots-blog/google-cloud-platform-cron-jobs-sourced-from-kubernetes-33a4f6046e3f?source=rss-4b8a203df491------2</link>
            <guid isPermaLink="false">https://medium.com/p/33a4f6046e3f</guid>
            <category><![CDATA[google-cloud-functions]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Cody Zuschlag]]></dc:creator>
            <pubDate>Tue, 06 Mar 2018 10:23:54 GMT</pubDate>
            <atom:updated>2018-03-22T09:00:04.514Z</atom:updated>
            <content:encoded><![CDATA[<p>It’s not a new question. In fact, it has been asked, and answered, several times. As you build services on top of the Google Cloud Platform, you will inevitably need some sort of “cron” equivalent. Whether for database maintenance (like a garbage collector) or for servicing an <a href="https://blog.hubwa.re/google-cloud-pubsub-background-job-queue-with-cloud-function-workers-78092ce5a8cb">asynchronous job queue</a>. As features get added and complexity grows, cloud services will need tasks that are serviced at regular intervals.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ij4OThN8DISD0zwH-7UlWQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/yD5rv8_WzxA?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Nick Hillier</a> on <a href="https://unsplash.com/search/photos/clock?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h3>Recommended Architecture</h3><p>Google seems unanimous about the recommended architecture. The same architecture is documented in the Google Cloud Platform documentation, on the Firebase blog, and mentioned in the Google Cloud Platform Podcast: deploy a small web server to the Google App Engine that listens for requests triggered by the <a href="https://cloud.google.com/appengine/docs/flexible/nodejs/scheduling-jobs-with-cron-yaml">GAE cron service</a> and in turn publishes PubSub topics. They even provide a <a href="https://github.com/GoogleCloudPlatform/reliable-task-scheduling-compute-engine-sample">nice sample application</a> written in python that is ready to be deployed (<a href="https://github.com/firebase/functions-cron">and another for Firebase</a>).</p><ul><li><a href="https://cloud.google.com/solutions/reliable-task-scheduling-compute-engine">Reliable task scheduling on Compute Engine with Cloud Scheduler</a></li><li><a href="https://firebase.googleblog.com/2017/03/how-to-schedule-cron-jobs-with-cloud.html">How to Schedule (Cron) Jobs with Cloud Functions for Firebase</a></li><li><a href="https://www.gcppodcast.com/post/episode-111-google-cloud-platform-with-sam-ramji/">Google Cloud Platform with Sam Ramji</a></li></ul><p>And why shouldn’t they be unanimous? The proposed architecture has a lot of benefits when running in the cloud. The benefits stem from the decoupling of events (scheduled moments in time) and actions (stuff that gets done) with PubSub. <strong>By using PubSub to decouple events from actions we create scalable cloud cron jobs. Multiple actions can be triggered by a single event and, inversely, a single action can be triggered by multiple events. In addition, changes to scheduling have no impacts on actions and, inversely, changes to actions have no impacts on scheduling.</strong></p><blockquote>The benefits stem from the decoupling of of events (scheduled moments in time) and actions (stuff that gets done) with PubSub.</blockquote><h3>So what’s wrong with that?</h3><p>At Hubware, our customer facing API’s are all written in node.js and <a href="https://stackoverflow.com/a/11715598">scaled horizontally</a> by our Kubernetes cluster hosted in the Google Cloud Kubernetes Engine. However, we have recently been adding new functionality using serverless Cloud Functions. We then combine Cloud Functions with PubSub to distribute messages between our various services. Working with Cloud Functions allows us to scale (or not) certain functionality independent from our API’s deployed in the Kubernetes cluster.</p><p><a href="https://blog.hubwa.re/google-cloud-pubsub-background-job-queue-with-cloud-function-workers-78092ce5a8cb">Google Cloud PubSub background job queue with Cloud Function workers</a></p><p>The common thread linking all of this together is node.js. Working in node has allowed us come to market extremely fast and facilitates constantly adding new functionality that our customers want, whether deployed in Kubernetes or Cloud Functions. This has been a magic recipe: happy customers, happy sales team, happy product team, happy development team… ahhhhh.</p><blockquote>Working in node has allowed us come to market extremely fast and facilitates constantly adding new functionality that our customers want, whether deployed in Kubernetes or Cloud Functions.</blockquote><p>Given the above, when we needed to add cloud cron jobs, deploying a python application to the Google App Engine made the entire team cringe. We did not want any new maintenance overhead (applications running in the GAE) nor did we want to have to maintain a new python application.</p><h3>A better way…</h3><p>Fortunately for us, we found a better way. At least, better for us! The secret sauce was to replace the GAE deployment and <a href="https://cloud.google.com/appengine/docs/flexible/nodejs/scheduling-jobs-with-cron-yaml">cron scheduling</a> with <a href="https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/">Kubernetes CronJobs</a> (available since version 1.8, on <a href="https://cloud.google.com/kubernetes-engine/release-notes">GCP since October 2017</a>). This gave us all the benefits of using PubSub to decouple events from actions, but didn’t add any new infrastructure to maintain in our production environment.</p><blockquote>This gave us all the benefits of using PubSub to decouple events from actions, but didn’t add any new infrastructure to maintain in our production environment.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*n2Y7cnso6zVfGrlWqHWJiw.png" /></figure><p>The design is further simplified because a Kubernetes cron job that publishes to a PubSub topic can be defined in a single config file! No python web-server required to listen for events and then publish them to PubSub. Instead, by taking advantage of the Google’s <a href="https://hub.docker.com/r/google/cloud-sdk/">gcloud-sdk docker image</a> we can publish directly to the PubSub topic without any intermediary.</p><p>The file below illustrates how to publish to a PubSub topic directly in a Kubernetes cron job config file. Easy, cheesy!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e1a022cb1fc69568d422739fee08cfb4/href">https://medium.com/media/e1a022cb1fc69568d422739fee08cfb4/href</a></iframe><h3>Leveraging node.js for cron job actions</h3><p>Bringing node back into the story, Google Cloud Functions can be triggered by messages published to PubSub topics. The great thing about Cloud Functions: they run on node! This allows us to leverage the power of node, its ecosystem, and our existing expertise to quickly create cron job actions for almost anything. Need to do some DB maintenance, send a Slack notification, or just order a pizza… it’s easy with node!</p><blockquote>This allows us to leverage the power of node, its ecosystem, and our existing expertise to quickly create cron job actions for almost anything.</blockquote><h3>Living the good life on the Google Cloud Platform</h3><p>Combining Kubernetes cron jobs with PubSub allowed us to create scalable, distributed, cloud worthy cron jobs without adding any significant maintenance burden. Making a good thing better, we were able to leverage our existing node expertise!</p><p>I hope this magic recipe can help you out when you decide you need cron jobs on the Google Cloud Platform.</p><p>Leave your questions and comments below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33a4f6046e3f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/chatbots-blog/google-cloud-platform-cron-jobs-sourced-from-kubernetes-33a4f6046e3f">Google Cloud Platform cron jobs sourced from Kubernetes</a> was originally published in <a href="https://medium.com/chatbots-blog">Blog d&#39;Hubware sur l&#39;Assistance Professionnelle</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Google Cloud PubSub background job queue with Cloud Function workers]]></title>
            <link>https://medium.com/chatbots-blog/google-cloud-pubsub-background-job-queue-with-cloud-function-workers-78092ce5a8cb?source=rss-4b8a203df491------2</link>
            <guid isPermaLink="false">https://medium.com/p/78092ce5a8cb</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[message-queue]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Cody Zuschlag]]></dc:creator>
            <pubDate>Thu, 22 Feb 2018 08:25:22 GMT</pubDate>
            <atom:updated>2018-02-22T08:25:22.646Z</atom:updated>
            <content:encoded><![CDATA[<p>At Hubware we needed to schedule and queue background jobs and service them in bulk at arbitrary times in the future. We chose the Google Cloud PubSub framework as a message queue and Google Cloud Functions as workers to service the queued jobs. This unique combination created a reliable, scalable, and distributed cloud service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0n_3a-1sRIAEaLQiOhUyDQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/_uAVHAMjGYA?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">eberhard grossgasteiger</a> on <a href="https://unsplash.com/search/photos/cloud?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h3>Requirements</h3><p>The functional description of the feature at first seemed simple: queue background jobs and service the queue at regular intervals. However, there were some key requirements that quickly revealed the feature was going to have some unique technical hurtles:</p><ol><li><strong>Jobs are queued to be serviced at a later time.</strong> This is straight forward, when requesting a job to be serviced, it must be queued now and serviced asynchronously in the future, leaving the requester free for other work.</li><li><strong>Each job has a unique time range when it can be serviced (i.e. Monday-Friday 9:00 am to 5:00 pm).</strong> This adds significant technical complexity. Functionally, the jobs need to be serviced during the “working” hours of our customers. However, each customer has unique working hours. Further complicating the requirements, given that most of our customers are in the e-commerce sector, they have unique holiday schedules (extended hours leading up to and following Christmas). A generic 9 to 5 schedule is not good enough. Instead, the schedule must be customizable for each customer.</li><li><strong>Jobs scheduled “later” should not block jobs scheduled “sooner”.</strong> This is a key reason not to use a traditional message queue and why order is not important. When workers service jobs, the entire queue should be searched for jobs that can be serviced now.</li><li><strong>Jobs should be serviced in bulk when possible.</strong> Further adding to the technical complexity, we work with rate-limited API’s. Fortunately, the API’s support “bulk” operations. Any opportunity to combine API requests into bulk operations saves us money and reduces the chance that we exceed the rate limit when the feature starts to scale. It also saves in computing cost… working with a server-less architecture means each execution cycle has an associated cost. Servicing multiple jobs per execution is more efficient than servicing a single job with multiple executions.</li></ol><h3>Architecture</h3><p>Our architecture is inspired by Google’s article describing <a href="https://cloud.google.com/solutions/using-cloud-pub-sub-long-running-tasks">how to use PubSub for long running tasks</a>. We added a couple of unique twists: let the messages be queued in an existing PubSub subscription and service the jobs in bulk with Cloud Function workers (triggered periodically by <a href="https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/">Kubernetes CronJobs</a>).</p><p>Our background jobs are neither long running nor resource intensive (in contrast to the example presented in Google’s article). Instead, they access rate limited API’s and therefore allowing them to queue to be processed in bulk is desirable. In turn, by triggering the workers periodically at a reasonable interval (i.e. every 5 minutes), the jobs are serviced relatively quickly making it feel responsive.</p><p>There are several benefits to using PubSub with Cloud Functions:</p><ul><li>Reliable, distributed, and secure (thanks Google Cloud!)</li><li>Zero server maintenance (go serverless!)</li><li>Scalable (<em>see below</em>)</li><li>Monitoring (<em>see below</em>)</li></ul><p>The following diagram illustrates the architecture. The numbered arrows illustrate the flow of jobs through the system:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*_fgNU4rBzHlLqFGEJXB_Yg.png" /><figcaption>Clock icon by <a href="https://www.flaticon.com/authors/situ-herrera">Situ Herrera</a> from <a href="http://www.flaticon.com/">www.flaticon.com</a></figcaption></figure><ul><li>A service publishes a message (job) to the PubSub topic TopicEnqueue (<strong>arrow 1</strong>) whenever a new job needs to be serviced.</li><li>The jobs are queued by the PubSub subscription SubDequeue (<strong>arrow 2</strong>).</li><li>A cronjob, regularly publishes messages to the topic TopicCron. In our case the cronjob is configured in Kubernetes. However, it could be <a href="https://cloud.google.com/solutions/reliable-task-scheduling-compute-engine">scheduled by Google App Engine task scheduling</a> or another cron service (such as <a href="https://cron-job.org/en/">cron-job.org</a>).</li><li>A worker Cloud Function is triggered by the PubSub topic TopicCron. The message contents (and attributes) are ignored. The PubSub message is simply the trigger for the function. The worker is effectively triggered at regular periodic intervals.</li><li>The worker Cloud Function <a href="https://cloud.google.com/pubsub/docs/subscriber#push_pull">pulls</a> messages from the PubSub subscription SubDequeue (<strong>arrow 3</strong>).</li><li>The worker services the jobs without acknowledging the pulled messages. Upon successfully servicing a job (<strong>arrow 4</strong>), the worker acknowledges the messages. Jobs not ready to be serviced are left unacknowledged to be serviced during a future execution cycle.</li></ul><h3>Scaling</h3><p>As more jobs are requested, it is possible that jobs are queued faster than they are serviced. There are multiple ways to increase the throughput of the service:</p><ul><li>The simplest solution is to increase the frequency that the CronJob is triggered, resulting in the worker executing more often. The trade-off being that this type of scaling is not dynamicly based on the load. The function will execute more often even when there is low load.</li><li>The Cloud Function can also be deployed several times, resulting in parallel executions every time the CronJob is triggered. This will also result in static scaling.</li><li>A dynamic scaling solution could be implemented by monitoring the SubDequeue subscription (via the <a href="https://cloud.google.com/pubsub/docs/monitoring">monitoring API</a>) by an external process (possibly another Cloud Function) and scaling the number of deployed instances dynamically based on the number of messages (jobs) in the queue.</li></ul><h3>Monitoring</h3><p>The PubSub framework and Cloud Functions can be <a href="https://cloud.google.com/pubsub/docs/monitoring">monitored directly in Google’s Stackdriver tools</a>. No new tooling or thought required. With 1 hour of work a dashboard showing the rate of messages being enqueued and dequeued and the number of messages in the queue can be created.</p><blockquote>With 1 hour of work a dashboard showing the rate of messages being enqueued and dequeued and the number of messages in the queue can be created.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*530yKicZMr09bitsiffOFw.png" /><figcaption>Stackdriver Dashboard monitoring incoming and waiting jobs</figcaption></figure><p>Furthermore, the <a href="https://cloud.google.com/monitoring/alerts/">Stackdriver Alerting Polices</a> provide great ways to send emails and/or slack notifications when the system enters an unhealthy state. The Cloud Function has not executed for 10 minutes? The SubDequeue subscription has too many messages? The oldest message is approaching the hard deadline of 7 days? Stackdriver lets you easily send emails and slack notifications before these problems become catastrophic.</p><h3>Caveats</h3><p>There are few caveats when working with PubSub:</p><ol><li>Jobs not serviced are <a href="https://cloud.google.com/pubsub/docs/subscriber#create">deleted after 7 days from being published</a>. Period. While this is configurable (to be less), 7 days is the max lifetime of messages in PubSub.</li><li>The message acknowledgement timeout is by default 10 seconds and <a href="https://cloud.google.com/pubsub/docs/subscriber#create">configurable</a> up to 10 minutes. Turns out you can extend this limit at runtime for messages with some extra effort on the client side via <a href="https://cloud.google.com/pubsub/docs/reference/rest/v1/projects.subscriptions/modifyAckDeadline">an API call</a> (not exported in the node.js pubsub client).</li><li>Message order is not guaranteed. <a href="https://cloud.google.com/pubsub/docs/faq#order">According to Google</a>, “messages are delivered as fast as possible, with preference given to delivering older messages first, but this is not guaranteed.” Google has a <a href="https://cloud.google.com/pubsub/docs/ordering">long article</a> that explains that in most use cases, the order of message delivery is in fact not important and presents a solution to add order in the client.</li><li>Messages are <a href="https://cloud.google.com/pubsub/docs/subscriber#at-least-once-delivery">guaranteed to be delivered <em>at least</em> once</a>. They state that under rare circumstances, message could be duplicated. Better to write idempotent functions on the client. Not really a problem because idempotent functions are already a principle in functional programming, we can add that principle at the architectural level also.</li></ol><h4>The node.js pubsub client…</h4><p>The design requires the worker Cloud Function to “pull” queued messages from the SubDeqeueue subscription. We found, and later <a href="https://github.com/googleapis/nodejs-pubsub/issues/60#issuecomment-364532641">confirmed by Google’s developers</a>, that the official <a href="https://github.com/googleapis/nodejs-pubsub">node.js PubSub client</a> from Google is not compatible with Cloud Functions. This seems mostly due to the fact that the client does a lot of it’s work in the background. Google’s own documentation <a href="https://cloud.google.com/functions/docs/bestpractices/tips#do_not_start_background_activities">explicitly states to not start background activities</a> in Cloud Functions, but the official node.js PubSub client does just that!</p><blockquote>Google’s own documentation <a href="https://cloud.google.com/functions/docs/bestpractices/tips#do_not_start_background_activities">explicitly states to not start background activities</a> in Cloud Functions, but the official node.js PubSub client does just that!</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UwfRCH-UYh2raOXmNkQFaQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/8zdEgWg5JAA?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Rob Schreckhise</a> on <a href="https://unsplash.com/search/photos/problem?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>The solution is to use Google’s <a href="https://cloud.google.com/pubsub/docs/reference/rest/v1/projects.subscriptions/pull">service REST API to pull</a> messages from the subscription. This works, but as discussed <a href="https://stackoverflow.com/a/48706961/880509">here</a> and <a href="https://github.com/googleapis/nodejs-pubsub/issues/60#issuecomment-364972333">here</a>, the API is not intended to fetch <em>all</em> queued messages. A “hacked” undocumented algorithm is required to work-around how PubSub delivers the messages. This is because the API does not guarantee to deliver <em>all</em> waiting messages in a single request.</p><h3>Conclusion</h3><p>Our experience revealed that PubSub is intended for either immediate (push) processing of messages (i.e. triggering a Cloud Function) or for long running process that pull messages asynchronously and process them immediately. There are some unique challenges involved to read (pull) the entire queue and determine which messages should be serviced in a short lived process (like Cloud Functions). However, we were able to work-around these challenges to create a new innovative cloud service built on a combination of <a href="https://cloud.google.com/">Google Cloud Platform</a> services.</p><p>The result of our innovation using PubSub in combination with Cloud Functions allowed us to quickly add an MVP (minimal viable product) feature with some great advantages: reliability, scalability, and monitoring.</p><p>Some improvements to the architecture are already foreseen. On the technical side, our challenges pulling PubSub messages in a Cloud Function seems theoretically stable, but an unproven use of PubSub. Functionally, the 7 day max lifetime of messages and no easy way to modify/delete messages in the queue may force us to move to a more traditional storage method (like a DB).</p><p>Leave your questions and comments below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=78092ce5a8cb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/chatbots-blog/google-cloud-pubsub-background-job-queue-with-cloud-function-workers-78092ce5a8cb">Google Cloud PubSub background job queue with Cloud Function workers</a> was originally published in <a href="https://medium.com/chatbots-blog">Blog d&#39;Hubware sur l&#39;Assistance Professionnelle</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Don’t nock our integration tests]]></title>
            <link>https://medium.com/chatbots-blog/dont-nock-our-integration-tests-fa9682ba7155?source=rss-4b8a203df491------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa9682ba7155</guid>
            <category><![CDATA[continuous-delivery]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[continuous-integration]]></category>
            <category><![CDATA[integration-testing]]></category>
            <category><![CDATA[nodejs]]></category>
            <dc:creator><![CDATA[Cody Zuschlag]]></dc:creator>
            <pubDate>Thu, 14 Dec 2017 08:00:48 GMT</pubDate>
            <atom:updated>2017-12-14T16:37:35.782Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D7GDnWGWABsbgDaYHaG4Iw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/ck3HFWw2OiM?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">LUM3N</a> on <a href="https://unsplash.com/?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>We’ve worked hard at <a href="http://www.hubwa.re/">Hubware</a> to have an extensive test suite. This allows us to deploy frequently and always with very high level of confidence that there will be no impact for our customers. This has been unarguably a key factor in our success in delivering the features our customers want, fast.</p><blockquote>[Our test suite is] a key factor in our success in delivering the features our customers want, fast.</blockquote><p>We had a fairly “classic” test strategy for a node.js based REST API. Unit tests and integration tests. Our unit tests test individual modules and use a lot of mocking (either stubs or <a href="https://facebook.github.io/jest/docs/en/mock-functions.html">jest’s mocking features</a>). Our integration tests worked directly with the REST APIs (requiring an instance of the back-end service to be running).</p><h3>Trouble in Paradise</h3><p>However, there was trouble in paradise. Time and time again, we were forced to fix failing integration tests. The tests were simply too fragile.</p><p>Every few days (or weeks when we were lucky) the tests started failing due to changes in the data that we collect from external APIs; data that changed the responses tested by our integration tests. These were almost always false positives and not only cost us time but also generated frustration on the team. The fragile tests were slowly destroying our confidence in the integration tests… was a failing test a bug (rarely the case) or simply another false positive? Without confidence in the tests, there is no hope for continuous deployment.</p><blockquote>Without confidence in the tests, there is no hope for continuous deployment.</blockquote><p>This led to numerous discussions about our testing strategy and several ideas to make our tests more durable. One solution that we adopted to make the integration tests less fragile was to make them less strict. We added some regex matching when making assertions with the API responses. This is arguably an anti-pattern (I think there is a contradiction is making tests <em>less</em> strict), but if used selectively, it is a useful tool. There are dates and ids in the API responses that we really can just ignore (or at least assert they match a pattern rather than a fixed value).</p><p>Even though I initially was resistant, after reading the great article <a href="https://medium.com/javascript-scene/mocking-is-a-code-smell-944a70c90a6a">Mocking is a Code Smell</a> by <a href="https://medium.com/u/c359511de780">Eric Elliott</a>, I decided to revisit our test strategy. The problem that I found was that we didn’t have true “integration” tests… our tests either tested individual modules (unit tests) or tested the entire back-end via the REST APIs (we called these “integration” tests). After examining our growing “integration” test suite I realized what was happening, we were testing how modules were working together at a very high level: the API exposed by the back-end.</p><blockquote>I realized what was happening, we were testing how modules were working together at a very high level: the API exposed by the back-end.</blockquote><p>What was missing was a true integration test suite that would allow developers to test how some modules (rather than all) interacted together.</p><h3>nock on Wood</h3><p>I also decided I would revisit <a href="https://github.com/node-nock/nock">nock</a>. I was specifically interested in nock’s <a href="https://github.com/node-nock/nock#nock-back">nock back</a> feature for testing what we call “connectors”: our modular plugins that abstract 3rd party APIs to a common interface. With nock back and a suite true integration tests, we could test our “connectors” isolated from the rest of the back-end. The goal was to have fewer of our old, slow, and heavy integration tests. Plus our new integration tests would be less fragile because they would rely on “recorded” API responses rather than live calls to external APIs.</p><p>However, I had reservations… My reservations with nock come from my previous experiences building microservices…</p><h4>First it was good</h4><p>I worked on a project that used a node + <a href="https://www.docker.com/">docker</a> + <a href="https://github.com/docker/compose">docker compose</a> microservice architecture. Each dev team was responsible for 1 or 2 microservices (depending on the service complexity). In the beginning we decided to write integration tests that would isolate a given microservice from all of its dependencies. Here, nock was a bright shinning star that enabled us to rapidly write integration tests. Everything was great!</p><h4>Then it was bad</h4><p>One day it was decided (for better or worse) that our integration tests should also test all of the services that were dependencies of the service under test.</p><p>So, my service A depended on B and C. The decision was made that the integration tests for A should include (and therefore test) B and C. However there were still other external services that needed to be mocked, but this can’t work in nock. Nock overrides the HTTP stuff in the node core to intercept requests. The magic and simplicity of nock means it can’t intercept HTTP requests from other processes (let alone other docker containers).</p><p>Nock can only intercept HTTP requests being made in the same node process where the tests are running. In the end we stripped nock from the tests and wrote manual stubbed web services that were included in the application during tests.</p><blockquote>Nock can only intercept HTTP requests being made in the same node process where the tests are running.</blockquote><h3>Test Paradise</h3><p>Back to Hubware… The problem was in the details. What we were calling “integration” tests were actually back-end acceptance or validation tests because they were testing via the REST APIs exposed by the back-end. I decided it was time to rename that test suite and add a new, true integration test suite.</p><p>We are now working to re-classify and migrate tests to the appropriate test suite. The goal is to that our old “integration” test suite will now be a light “happy path” <em>acceptance</em> test suite. We have added a new true integration test suite has been introduced to allow testing the interactions between modules, at the module level. This test suite, while separate from the unit tests, will run just after the unit tests in our CI.</p><blockquote>A new true integration test suite has been introduced to allow testing the interactions between modules, at the module level.</blockquote><p>The new integration tests can optionally use nock and nock back to mock external APIs. However in the CI, nock will disable any external HTTP requests, to ensure fast and reliable tests. However, we have chosen to allow using nock consciously, accepting that the tests will be limited to testing the how modules in our application work together and not the entire application. Therefore there is no risk that one day we will try to mock the HTTP requests for the entire application (a problem that nock is not well suited for solving).</p><p>Here is the presentation I made to the team to convince them that this new layer of testing was necessary:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.slideshare.net%2Fslideshow%2Fembed_code%2Fkey%2F3IzcyeoptwEh2V&amp;url=https%3A%2F%2Fwww.slideshare.net%2FCodyZuschlag%2Fhubware-test-strategy-improvements&amp;image=https%3A%2F%2Fcdn.slidesharecdn.com%2Fss_thumbnails%2Fhubwareteststrategypublic-171124145133-thumbnail-4.jpg%3Fcb%3D1511535568&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=slideshare" width="600" height="500" frameborder="0" scrolling="no"><a href="https://medium.com/media/f414528c9ee854e83d188cdf996e6054/href">https://medium.com/media/f414528c9ee854e83d188cdf996e6054/href</a></iframe><p>Leave your question and/or comments below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa9682ba7155" width="1" height="1" alt=""><hr><p><a href="https://medium.com/chatbots-blog/dont-nock-our-integration-tests-fa9682ba7155">Don’t nock our integration tests</a> was originally published in <a href="https://medium.com/chatbots-blog">Blog d&#39;Hubware sur l&#39;Assistance Professionnelle</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[NodeConf EU: An incredible conference for an incredible platform]]></title>
            <link>https://medium.com/chatbots-blog/nodeconf-eu-an-incredible-conference-for-an-incredible-platform-c04fcc9be856?source=rss-4b8a203df491------2</link>
            <guid isPermaLink="false">https://medium.com/p/c04fcc9be856</guid>
            <category><![CDATA[conference]]></category>
            <category><![CDATA[nodejs]]></category>
            <dc:creator><![CDATA[Cody Zuschlag]]></dc:creator>
            <pubDate>Thu, 30 Nov 2017 08:10:58 GMT</pubDate>
            <atom:updated>2017-11-30T20:16:01.556Z</atom:updated>
            <content:encoded><![CDATA[<p>As the lead developer of a team creating highly performant web services with node.js, when my employer, <a href="http://www.hubwa.re/">Hubware</a>, asked me if there were any conferences I wanted to attend this year, the answer was simple: <a href="http://www.nodeconf.eu/">NodeConfEU</a>. The conference was an incredible experience. I was able to see not only what great things are coming to node and its ecosystem, but also how some genius people are finding new and creative ways to interact with technology and how JavaScript (running with node) is now the “go to” language for developing and powering these projects.</p><blockquote>JavaScript (running with node) is now the “go to” language for developing and powering these projects</blockquote><p>In this post I will highlight a few of the talks and workshops that I attended, with a focus on how they relate to my daily development at Hubware and the general adoption of node. At the end of the article you can find a list of links to projects and resources I mentioned in the post.</p><h4>Arrival</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PdMs6f4wis_GdPH_Y83rQg.jpeg" /></figure><p>Being the premier European conference dedicated to node made this a no brainier. Of course, it didn’t hurt that it was held in Kilkenny, Ireland at the beautiful 5-star <a href="http://www.lyrath.com/">Lyrath Hotel and Conference Center</a>. Upon my arrival at the hotel and conference center, I was initially shocked how beautiful the venue was and it quickly became clear that this was going to be an exceptional conference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HqPhGkRGT4f-9RZycm41JQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SpLSeK8mg9DVCO5ECGdssQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wyHgV3PXzonWTdI_2Xq33Q.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_UFObcXxjSoNuNxPNiZvfA.jpeg" /></figure><h4>The Badges</h4><p>I took advantage of the early registration opportunity the first night. At which time I was met with the pleasant surprise of the extraordinary conference badges: a compact embedded microprocessor coupled with a LCD (and other I/O). At the <a href="https://youtu.be/y9vySsTnp8M">first talk of the conference</a>, <a href="https://twitter.com/Espruino">Gordon Williams</a>, Director at Pur3 Ltd, explained that badges were built using <a href="https://www.espruino.com/">Espruino</a> hardware meaning they run (what else other than) JavaScript. The boards were pre flashed with a <a href="https://nodejs.org/api/repl.html">REPL</a> allowing the possibility to enter JavaScript directly on the board using the the game-pad like button layout. This proved to be a painful and mind blowing experience. We were later shown how to connect via bluetooth to the boards and program them in JavaScript directly through a browser based IDE… Coming from a firmware engineering background, having professionally written C++ firmware for ARM7 based microprocessors for several years, I can’t fully express how amazing and accessible this kind of technology has become. Major shout out to Gordon Williams for designing and presenting these badges!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*60MpMigjbutqAQvy4DTAog.jpeg" /></figure><h3>Highlights of the Talks</h3><p>There were so many great talks at the conference. You should really find the time to <a href="https://www.youtube.com/playlist?list=PL0CdgOSSGlBaxNkrUIHrhd1f3ch45f66_">watch them all</a>. Here are a few highlights for me and how they relate to the cutting edge development that we do at Hubware.</p><h4>npm</h4><p><a href="https://medium.com/u/e3e94abc919d">Re: Becca</a> from NPM <a href="https://youtu.be/j0S4M4HWFZs">presented</a> some of the great new and upcoming features in npm. The standouts for me are the npm commands <a href="https://www.npmjs.com/package/npx">npx</a> and npm ci. npx takes a bunch of the hassle out of running npm packages. No more messy global installs or constantly typing node_modules/.bin to run those commands from the console. I’m looking forward to getting this into my workflow. In full disclosure, we switched to yarn ages ago and haven’t looked back. However, now that npm has <a href="https://docs.npmjs.com/files/package-lock.json">package-lock</a> and that the coming npm ci command will install the node_modules based on the package-lock and the claimed negligible performance difference with “other” package manager, I won’t be opposed to revisiting npm in our CI environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t32N9TbGj5S0p1s_pJkKEw.jpeg" /></figure><h4>Node Core</h4><p>I have to mention the <a href="https://youtu.be/hVfQ8KtRptM">talk</a> from <a href="https://medium.com/u/a1f67ed6052d">James M Snell</a>, Open Source Architect at <a href="https://www.nearform.com/">nearForm</a> and member of the node Technical Steering Committee, gave about what is coming to the node core. Sure the changes in core lack some wow factor since we’ve known about them for a long time, but it’s nice to see that node is (relatively quickly) improving and adapting to the needs of the community. This includes making async_hooks official and the coming of async iterators. Async_hooks is a bit of a low level feature for me and my daily development tasks. However, the tooling that async_hooks<a href="https://blog.sqreen.io/diy-node-apm/"> allows in the performance monitoring arena</a> is, in my opinion, incredibly important for the future of node as a first class, enterprise grade platform. Also, James said promises are coming to the core api, as in the fs module, as first class citizens (not wrapped versions taking a performance hit). It’s been a long time coming, maybe too long, but it is nice to see the debate is over: async/await has helped make promises first class citizens.</p><blockquote>incredibly important for the future of node as a first class, enterprise grade platform</blockquote><h4>N-API</h4><p>Another place that I saw some great advances being made was in the <a href="https://youtu.be/E848bgHfoxE">talk presented</a> by <a href="https://twitter.com/mhdawson1">Michael Dawson</a> and <a href="https://medium.com/u/a12d093786f1">Arunesh Chandra</a>, Microsoft, about <a href="https://nodejs.org/api/n-api.html">N-API</a>. This currently experimental API allows compiled modules to work on all versions of node (nearly) and all run-times (V8 and Microsoft’s chakra core) without the need to recompile. It seems to provide a version independent abstraction over the V8 APIs. This will be useful for strict build environments where it isn’t always possible to recompile packages when changing the version of node. I’ve worked on projects where were the CI builders were locked down (for security reasons) and we didn’t have access to gcc. In the end, we had to find “creative” solutions to use compiled packages and, needless to say, there was a lot headache that resulted whenever we changed the version of node and needed to recompile all of those packages. I’m happy to see some progress in this area because the ability to have binary packages is an important part of node.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mKcbdSJ9jeiJfp5VGMSXxA.jpeg" /></figure><h4>Client Side ML and NLP</h4><p><a href="https://medium.com/u/4c6621f68451">Nikhila Ravi</a>, Software Developer at DWYL, gave a <a href="https://youtu.be/vkgnSeGO_W0">spectacular talk</a> about how she was able to run machine learning in the browser… yes, as in the client. Sure, we can imagine the CPU power necessary for this will destroy the batteries of everyone’s smartphones, but the potential benefits are tempting. For example, not needing to upload all of that sensitive data to the server. This is very relevant to the machine learning and artificial intelligence that we use at Hubware (especially considering our users are typically using desktop machines). Having the option to do some of that the AI directly in the user’s browser will give us a lot of flexibility, especially for sensitive data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uxNstHegbXu22EIlOHExaw.jpeg" /></figure><h4>Hardware</h4><p>Having a background as a firmware engineer, even if it’s not my current passion, I have to at least mention all of the incredible hardware demos that used node either to power the project or in the development tool-chain. I already mentioned the amazing espruino badges created by Gordon Williams. <a href="https://medium.com/u/4f84cae4fff3">rachel white</a>, Technical Evangelist at Microsoft, <a href="https://youtu.be/NB2gcjZoQJ0">demoed</a> her tiny art gallery that used a raspberry pi running node to render images on an led matrix. Not explicitly hardware, but <a href="https://medium.com/u/91ccd9244324">Rachel Simone Weil</a>, Technical Evangelist at Microsoft, <a href="https://youtu.be/f61di5B8cWQ">demoed</a> her node powered REST API that modified Nintendo ROMs to build customized screensavers (demoed with an emulator, but that still counts as hardware, right?). <a href="https://medium.com/u/1c9f9321358f">Charlie Gerard</a>, Creative Developer at The New York Times, <a href="https://youtu.be/xbkYjjPndWE">demoed</a> a “brain sensor” controlling a drone where node was the technology used to link the two devices together. The demo gods didn’t let this one happen, but her videos proved it did happen.</p><h3>Workshops</h3><p>While the mornings were filled with great presentations, the afternoons offered various top notch workshops. Here are a few highlights of the workshops I attended.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hR3u7RrXkUhJ3Uha3N3X-Q.jpeg" /></figure><h4>Node Core PRs</h4><p><a href="https://twitter.com/addaleax">Anna Henningsen</a> and <a href="https://medium.com/u/94ab368d62fd">Rich Trott</a> facilitated a Code &amp; Learn workshop helping people make their first PR to node core. While I’ve already made a PR to node core, it was fun to see a room full of people building the node core (really, this is not as hard as it might sound)!</p><h4>Cryptography and Streams</h4><p><a href="https://medium.com/u/58d9bf22e2db">David Mark Clements</a> and <a href="https://medium.com/u/bc6e8b811bba">Matteo Collina</a> facilitated a streams workshop where used streams to do to some serious data processing (gigabytes in a matter of seconds). <a href="https://medium.com/u/fffba16d1372">Mathias Buus</a> and <a href="https://twitter.com/emilbayes">Emil Bay</a> did a crypto workshop where we used <a href="https://github.com/sodium-friends/sodium-native">sodium-native</a> to do some real world, efficient crypto.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pn1JwiIACSikmKEjGihCVg.jpeg" /></figure><h4>Project Management</h4><p>I attended the delightfully named workshop Taming Unicorns with Technology animated by <a href="https://twitter.com/moogster31">Katie Roberts</a> and <a href="https://twitter.com/neil_vass">Neil Vass</a>. They did a wonderful job playing the role of the customer who wants everything but can’t give a single precise detail (we’ve all seen that before, right?). They presented some tools to help with this including the <a href="https://en.wikipedia.org/wiki/MoSCoW_method">MoSCoW method</a> and what they called the “lines of hope and despair”, where you force the stakeholders to prioritize the features and agree on what <em>must</em> be in the MVP and what the <em>hope</em> will be in the MVP. The biggest takeaway was to agree upon a specification and include that specification in all communications to avoid the surprise “but I thought we agreed on something different”.</p><blockquote>agree upon a specification and include that specification in all communications</blockquote><h4>Kubernetes</h4><p>The last day I attended a Kubernetes workshop hosted by <a href="https://medium.com/u/57387f5314d0">Robert Kubis</a>, Cloud Developer Advocate at Google UK. He did a step-by-step walk-through of creating and deploying a project with Kubernetes on Google Cloud. At Hubware, we’ve been using Kubernetes and Google Cloud from nearly the beginning. However, this was a nice overview and gave me the opportunity to ask some very detailed questions relating to our workflow regarding rolling updates and canary testing.</p><h3>Final Thoughts</h3><p>NodeConfEU 2017 was the best conference I have ever attended in terms of the subjects, speakers, workshops, venue, food, and entertainment. I forgot to mention the buses that took us to local pubs to enjoy live music and typical Irish cuisine… a much appreciated moment to unwind and have some fun after intense days of learning!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NBge6tXdmHXq5RnQ5zJonA.jpeg" /></figure><p>What made the biggest impression on me was that node is the go-to platform for almost everything. I think that is what has made, and continues to make, node so successful: it’s both enterprise grade for professionals and accessible and flexible enough for hobbyists.</p><blockquote>[node] …is both enterprise grade for professionals and accessible and flexible enough for hobbyists</blockquote><p>For Hubware, the conference reinforced that we have built our Professional Digital Assistance back-end on the right technology stack. We will be able to continue to improve our offering in terms of security, performance, and functionality. All while being agile and rapidly adjusting to our customers needs. I’m looking forward to integrating some of the new ideas I learned at the conference into our workflow!</p><blockquote>All while being agile and rapidly adjusting to our customers needs.</blockquote><p>Thanks again to <a href="https://medium.com/u/4d2ed76cb5eb">nearForm</a> and all the organizers of NodeConfEU!</p><h3>Useful Links</h3><p>Here is a list of some of links mentioned in this post.</p><ul><li><a href="http://www.nodeconf.eu/">NodeConfEU</a></li><li><a href="https://www.youtube.com/playlist?list=PL0CdgOSSGlBaxNkrUIHrhd1f3ch45f66_">YouTube Videos of All Talks</a></li><li><a href="https://www.espruino.com/">Espruino</a></li><li><a href="https://www.npmjs.com/package/npx">npx</a></li><li><a href="https://github.com/sodium-friends/sodium-native">sodium-native</a></li><li><a href="https://en.wikipedia.org/wiki/MoSCoW_method">MoSCoW method</a></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mQ8mmb9J-05XrIcn6OHjzA.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c04fcc9be856" width="1" height="1" alt=""><hr><p><a href="https://medium.com/chatbots-blog/nodeconf-eu-an-incredible-conference-for-an-incredible-platform-c04fcc9be856">NodeConf EU: An incredible conference for an incredible platform</a> was originally published in <a href="https://medium.com/chatbots-blog">Blog d&#39;Hubware sur l&#39;Assistance Professionnelle</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>