<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pritesh Tailor on Medium]]></title>
        <description><![CDATA[Stories by Pritesh Tailor on Medium]]></description>
        <link>https://medium.com/@drtailor?source=rss-102450e29029------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*4vvHK8Vvieg7N7vN1KSdVg.jpeg</url>
            <title>Stories by Pritesh Tailor on Medium</title>
            <link>https://medium.com/@drtailor?source=rss-102450e29029------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 16:34:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@drtailor/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A Visual Guide to Archimedean Spirals and its Special Cases]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/math-simplified/a-visual-guide-to-archimedean-spirals-and-its-special-cases-5173966800bf?source=rss-102450e29029------2"><img src="https://cdn-images-1.medium.com/max/600/1*2tRuo7ZzoIMJdKIIcrpktw.gif" width="600"></a></p><p class="medium-feed-snippet">An intuitive and animated (but condensed) guide to the Archimedean Spiral, and the related Fermat, Doppler and Hyperbolic Spirals.</p><p class="medium-feed-link"><a href="https://medium.com/math-simplified/a-visual-guide-to-archimedean-spirals-and-its-special-cases-5173966800bf?source=rss-102450e29029------2">Continue reading on Math Simplified »</a></p></div>]]></description>
            <link>https://medium.com/math-simplified/a-visual-guide-to-archimedean-spirals-and-its-special-cases-5173966800bf?source=rss-102450e29029------2</link>
            <guid isPermaLink="false">https://medium.com/p/5173966800bf</guid>
            <category><![CDATA[math]]></category>
            <category><![CDATA[archimedes]]></category>
            <category><![CDATA[animation]]></category>
            <category><![CDATA[spiral]]></category>
            <category><![CDATA[guides-and-tutorials]]></category>
            <dc:creator><![CDATA[Pritesh Tailor]]></dc:creator>
            <pubDate>Sat, 21 May 2022 20:33:32 GMT</pubDate>
            <atom:updated>2022-05-21T21:03:01.628Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[A Visual Guide to Trigonometric Functions and their Reciprocals]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/math-simplified/a-visual-guide-to-trigonometric-functions-and-their-reciprocals-92e94dea95ca?source=rss-102450e29029------2"><img src="https://cdn-images-1.medium.com/max/2422/1*kTA4sKs_xRJIZv-ir_YaDw.png" width="2422"></a></p><p class="medium-feed-snippet">An intuitive and animated guide to the trig functions sin, cos, tan, cosec, sec and cot using the unit circle.</p><p class="medium-feed-link"><a href="https://medium.com/math-simplified/a-visual-guide-to-trigonometric-functions-and-their-reciprocals-92e94dea95ca?source=rss-102450e29029------2">Continue reading on Math Simplified »</a></p></div>]]></description>
            <link>https://medium.com/math-simplified/a-visual-guide-to-trigonometric-functions-and-their-reciprocals-92e94dea95ca?source=rss-102450e29029------2</link>
            <guid isPermaLink="false">https://medium.com/p/92e94dea95ca</guid>
            <category><![CDATA[circle]]></category>
            <category><![CDATA[animation]]></category>
            <category><![CDATA[math]]></category>
            <category><![CDATA[visualization]]></category>
            <category><![CDATA[trigonometry]]></category>
            <dc:creator><![CDATA[Pritesh Tailor]]></dc:creator>
            <pubDate>Sat, 23 Apr 2022 15:41:47 GMT</pubDate>
            <atom:updated>2022-04-23T15:41:47.332Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Gaussians as Exponential Imposters and its Application in Quantum Chemistry]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://www.cantorsparadise.com/gaussians-as-exponential-imposters-and-its-application-in-quantum-chemistry-16f5ffca098d?source=rss-102450e29029------2"><img src="https://cdn-images-1.medium.com/max/1092/1*T-D3xWMFXSBa3dDluak_JQ.png" width="1092"></a></p><p class="medium-feed-snippet">A look at how Gaussians can be used in place of an exponential and its successful use in Quantum Chemistry.</p><p class="medium-feed-link"><a href="https://www.cantorsparadise.com/gaussians-as-exponential-imposters-and-its-application-in-quantum-chemistry-16f5ffca098d?source=rss-102450e29029------2">Continue reading on Cantor’s Paradise »</a></p></div>]]></description>
            <link>https://www.cantorsparadise.com/gaussians-as-exponential-imposters-and-its-application-in-quantum-chemistry-16f5ffca098d?source=rss-102450e29029------2</link>
            <guid isPermaLink="false">https://medium.com/p/16f5ffca098d</guid>
            <category><![CDATA[math]]></category>
            <category><![CDATA[imposter]]></category>
            <category><![CDATA[quantum-chemistry]]></category>
            <category><![CDATA[gaussian]]></category>
            <category><![CDATA[exponential]]></category>
            <dc:creator><![CDATA[Pritesh Tailor]]></dc:creator>
            <pubDate>Wed, 20 Apr 2022 23:10:26 GMT</pubDate>
            <atom:updated>2022-04-21T11:33:19.378Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How Do QR Codes Actually Work?]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://drtailor.medium.com/how-do-qr-codes-actually-work-cf878f6296b4?source=rss-102450e29029------2"><img src="https://cdn-images-1.medium.com/max/800/1*P075YNtszNmCmz75JkKmPA.jpeg" width="800"></a></p><p class="medium-feed-snippet">A basic guide to the workings of QR codes.</p><p class="medium-feed-link"><a href="https://drtailor.medium.com/how-do-qr-codes-actually-work-cf878f6296b4?source=rss-102450e29029------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://drtailor.medium.com/how-do-qr-codes-actually-work-cf878f6296b4?source=rss-102450e29029------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf878f6296b4</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[qr-code]]></category>
            <category><![CDATA[scan]]></category>
            <category><![CDATA[how]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[show]]></category>
            <dc:creator><![CDATA[Pritesh Tailor]]></dc:creator>
            <pubDate>Tue, 05 Apr 2022 14:32:02 GMT</pubDate>
            <atom:updated>2022-04-06T00:08:52.901Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Caching Miniconda Packages to Speed up Bitbucket Pipelines]]></title>
            <link>https://faun.pub/caching-miniconda-packages-to-speed-up-bitbucket-pipelines-93d0c08ba837?source=rss-102450e29029------2</link>
            <guid isPermaLink="false">https://medium.com/p/93d0c08ba837</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[miniconda]]></category>
            <category><![CDATA[bitbucket-pipelines]]></category>
            <category><![CDATA[cache]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Pritesh Tailor]]></dc:creator>
            <pubDate>Mon, 28 Mar 2022 17:50:43 GMT</pubDate>
            <atom:updated>2022-03-28T17:50:43.230Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cXV5VDFTu_6reU2-" /><figcaption>Photo by <a href="https://unsplash.com/@anagani_saikiran?utm_source=medium&amp;utm_medium=referral">Sai Kiran Anagani</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Bitbuckets pipelines is a great out the box tool for continuous testing and deployment. However, depending on the environment you need for building and testing your software this can take a very long time when using conda.</p><p>There are several general strategies that can be employed to speed up setting up conda environments such as using the continuumio/miniconda3 docker image or using faster C++ miniconda implementation such as mambaorg/micromamba image both available from docker hub. However in both cases setting up the environment will still involve downloading all the dependencies each time you set up the environment every time the pipeline is triggered, making the time for it to run even longer and the wait tedious!</p><p>To save on runtime in your pipelines you can use the bitbuckets caching feature. It offers out the box caching support for packages downloaded from pip, node…etc however not for miniconda. So how can we implement ourselves it to cut down our pipeline run time?</p><p>Below is a bitbucket-pipelines.yml where the pkgs directory for miniconda which contains all the zipped up packages from miniconda3 is cached so the <strong>pkgs are not re-downloaded each time the pipeline is triggered</strong>.</p><blockquote>Note: I use mamba instead of conda to get a further speed up. I know adding its installation to the pipeline adds time but it also gets cached as well so no worries! (you can use the mamba image directly instead mambaorg/micromamba)</blockquote><p>In the file, we use the continuumio/miniconda3 image which has miniconda3 already installed to /opt/conda.</p><p>In the definitions section, we specify what the miniconda3 cache path will be, in our case I chose /opt/conda/pkgs to save my pipeline always re-downloading packages each time it runs/sets up an environment and will be referenced to using miniconda3 hence miniconda3: /opt/conda/pkgs/.</p><pre>image: continuumio/miniconda3:latest</pre><pre>definitions:<br>  caches:<br>    miniconda3: /opt/conda/envs/</pre><pre>pipelines:<br>  default:<br>    - step:<br>      name: Lint python code with a cached miniconda3 env<br>      caches:<br>        - miniconda3<br>      script:<br>        - conda install -c conda-forge mamba<br>        - mamba install -c conda-forge gdal python=3.9.10 -y<br>        - mamba env list</pre><p>On the first run of the above, the pipeline will run as normal but at the end in the <em>build teardown</em>, you will see the pipeline upload the contents of /opt/conda/pkgs/ to the pipeline&#39;s cache as shown:</p><pre>Cache &quot;miniconda3&quot;: Compressing</pre><pre>Assembling contents of new cache &#39;miniconda3&#39;<br>Cache &quot;miniconda3&quot;: Compressing<br>Cache &quot;miniconda3&quot;: Compressed in 57 seconds<br>Cache &quot;miniconda3&quot;: Uploading 550.3 MiB<br>Cache &quot;miniconda3&quot;: Uploaded in 34 seconds</pre><p>If the upload is successful and the pipeline is re-run, you should see the following in the <em>build setup</em> section of your pipeline:</p><pre>Cache &quot;miniconda3&quot;: Downloading<br>Cache &quot;miniconda3&quot;: Downloaded 550.3 MiB in 7 seconds<br>Cache &quot;miniconda3&quot;: Extracting<br>Cache &quot;miniconda3&quot;: Extracted in 9 seconds</pre><p>Also in your conda environment installation log, you will see mamba show it&#39;s using cached packages and will not be downloading it from the conda-forge repo.</p><pre>+ gdal                              3.4.1  py39h85832e7_6       conda-forge/linux-64     Cached<br>+ geojson                           2.5.0  py_0                 conda-forge/noarch       Cached<br>+ geos                             3.10.2  h9c3ff4c_0           conda-forge/linux-64     Cached</pre><p>The timing for the above pipeline goes from <strong>1min30secs</strong> to <strong>1min 2secs</strong> for this env with a small number of packages, about a <strong>30% speedup.</strong></p><blockquote>In my own usage of the above caching method for my own miniconda3 environment which had <strong>ALOT</strong> of large packages to download and set up, I managed to get the run time down from <strong>5 mins</strong> to just under <strong>3 mins</strong> per run saving some time.</blockquote><h4><strong>Some Final Remarks:</strong></h4><ul><li>You can clear the cache manually if you need to.</li><li>Bitbucket will hold the cache for 1 week and hence get rebuilt so you must make sure the pipeline is idempotent.</li><li>If for example, you decide to use/add a newer set of dependencies/versions of a package, conda will still download what you need if it&#39;s not found in the current cache.</li><li>If your image uses a different path for conda, just change /opt/conda to whatever you need it to be, in my own custom images I use /opt/miniconda3.</li><li>I recommend using mamba as I have done since it gives an extra boost in speed to install your environment but feel free to remove it to use the standard conda if that&#39;s your preference.</li><li><strong>If you want even more speed up</strong>, I would recommend caching the envs you create as well which is found usually in /opt/conda/envs in addition to the /opt/conda/pkgs this will require a few more tricks to implement and get right.</li></ul><p>Lastly, I hope this article has helped you speed up your bitbucket pipelines when you&#39;re using Conda environments to be a bit more efficient, and hopefully some ideas to go beyond this basic implementation shown here.</p><figure><a href="https://faun.to/bP1m5"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BCiLLad3dvZLwBa-B5cAVQ.png" /></a></figure><p>Join FAUN: <a href="https://faun.to/i9Pt9"><strong>Website</strong></a><strong> </strong>💻<strong>|</strong><a href="https://faun.dev/podcast"><strong>Podcast</strong></a><strong> </strong>🎙️<strong>|</strong><a href="https://twitter.com/joinfaun"><strong>Twitter</strong></a><strong> </strong>🐦<strong>|</strong><a href="https://www.facebook.com/faun.dev/"><strong>Facebook</strong></a><strong> </strong>👥<strong>|</strong><a href="https://instagram.com/fauncommunity/"><strong>Instagram</strong></a><strong> </strong>📷|<a href="https://www.facebook.com/groups/364904580892967/"><strong>Facebook Group</strong></a><strong> </strong>🗣️<strong>|</strong><a href="https://www.linkedin.com/company/faundev"><strong>Linkedin Group</strong></a><strong> </strong>💬<strong>|</strong> <a href="https://faun.dev/chat"><strong>Slack</strong></a> 📱<strong>|</strong><a href="https://thechief.io/"><strong>Cloud Native</strong> <strong>News</strong></a><strong> </strong>📰<strong>|</strong><a href="https://linktr.ee/faun.dev/"><strong>More</strong></a><strong>.</strong></p><p><strong>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=93d0c08ba837" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/caching-miniconda-packages-to-speed-up-bitbucket-pipelines-93d0c08ba837">Caching Miniconda Packages to Speed up Bitbucket Pipelines</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Visual Guide to Taylor Series and its Application to √2]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://www.cantorsparadise.com/a-visual-guide-to-taylor-series-and-its-application-to-2-f7ced7863837?source=rss-102450e29029------2"><img src="https://cdn-images-1.medium.com/max/1200/1*UBWKv36wM-WPIt4mZnlgKw.png" width="1200"></a></p><p class="medium-feed-snippet">A visual walkthrough and guide of the Taylor series and its application to problems with minimal math and maximum graphs.</p><p class="medium-feed-link"><a href="https://www.cantorsparadise.com/a-visual-guide-to-taylor-series-and-its-application-to-2-f7ced7863837?source=rss-102450e29029------2">Continue reading on Cantor’s Paradise »</a></p></div>]]></description>
            <link>https://www.cantorsparadise.com/a-visual-guide-to-taylor-series-and-its-application-to-2-f7ced7863837?source=rss-102450e29029------2</link>
            <guid isPermaLink="false">https://medium.com/p/f7ced7863837</guid>
            <category><![CDATA[visualization]]></category>
            <category><![CDATA[approximation]]></category>
            <category><![CDATA[math]]></category>
            <category><![CDATA[computing]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Pritesh Tailor]]></dc:creator>
            <pubDate>Fri, 04 Mar 2022 20:24:55 GMT</pubDate>
            <atom:updated>2022-03-11T23:16:05.084Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How to set up HTTPS for Jenkins with a self-signed certificate on Ubuntu 20.04.]]></title>
            <link>https://drtailor.medium.com/how-to-set-up-https-for-jenkins-with-a-self-signed-certificate-on-ubuntu-20-04-2813ef2df537?source=rss-102450e29029------2</link>
            <guid isPermaLink="false">https://medium.com/p/2813ef2df537</guid>
            <category><![CDATA[ssl-certificate]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[setup]]></category>
            <category><![CDATA[self-signed-certificate]]></category>
            <category><![CDATA[jenkins]]></category>
            <dc:creator><![CDATA[Pritesh Tailor]]></dc:creator>
            <pubDate>Tue, 01 Mar 2022 09:02:52 GMT</pubDate>
            <atom:updated>2022-03-01T09:02:52.413Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*c5OIt0WZtWWixpVD" /><figcaption>Photo by <a href="https://unsplash.com/@christinhumephoto?utm_source=medium&amp;utm_medium=referral">Christin Hume</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>I’ve struggled to find a simple guide for this, so here is one! I’m going on the assumption you have a standard Jenkins setup already in /var/lib/jenkins and running it as the jenkins user.</p><p>Now let us get down to business on Jenkins:</p><p>In order to get Jenkins to use HTTPS, we need to be able to access your certificate and key from somewhere. Make sure you become the Jenkins user otherwise it will not have permission to read the files you make. Let’s make a folder inside /var/lib/jenkins to put our cert and key in.</p><pre>$ sudo su jenkins<br>$ mkdir /var/lib/jenkins/.ssl<br>$ cd /var/lib/jenkins/.ssl</pre><p>Now we generate our certificate and key in the directory we have created(still as the jenkins user)…we first generate the cert and key then need to convert the key format to PKCS #1:</p><pre>$ cd /var/lib/jenkins/.ssl<br>$ openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 3650 -nodes <br>$ openssl rsa -in key.pem -out private.pk1.key</pre><p>Now we need to edit the Jenkin config to tell it to use HTTPS and where the certificate and key are located. For those who use Ubuntu, this is /etc/default/jenkins. We tell Jenkins to run access via HTTPS on port 8443 (--httpsPort=8443) using the certificate we made earlier(--httpsCertificate=/var/lib/jenkins/.ssl/cert.pem) and the key as well (--httpsPrivateKey=/var/lib/jenkins/.ssl/private.pk1.key). We append this to the config with the command below (as root hence the sudo su ).</p><pre>$ sudo su -<br>$ echo &#39;JENKINS_ARGS=&quot; $JENKINS_ARGS --httpsPort=8443 --httpsPrivateKey=/var/lib/jenkins/.ssl/private.pk1.key --httpsCertificate=/var/lib/jenkins/.ssl/cert.pem&quot;&#39; &gt;&gt; /etc/default/jenkins</pre><pre># Your config should look like this at the bottom<br>$ tail -3 /etc/default/jenkins</pre><pre>JENKINS_ARGS=&quot;--webroot=/var/cache/$NAME/war --httpPort=$HTTP_PORT&quot;<br>JENKINS_ARGS=&quot;$JENKINS_ARGS --httpsPort=8443 --httpsPrivateKey=/var/lib/jenkins/.ssl/private.pk1.key --httpsCertificate=/var/lib/jenkins/.ssl/cert.pem&quot;</pre><p>Now let&#39;s restart Jenkins, so it will re-read its config:</p><pre>$ sudo systemctl restart jenkins</pre><p>Now you should be able to access your Jenkins server via HTTPS. It should be accessible via <a href="https://jenkins.previsico.live:8443/job/ansible-iac/indexing/console">https://YOUR_JENKINS_URL:8443/</a> or for automatic redirecting set HTTP to HTTPS by adding the arg--httpsRedirectHttp=true.</p><p>Note: if you want to disable HTTP, you will need to set HTTP_PORT=-1 in /etc/default/jenkins and restart Jenkins again.</p><p>Those who prefer a video guide, try this one by cloudbees, which formed the basis of this guide: <a href="https://www.youtube.com/watch?v=2uYL4az1BVU">https://www.youtube.com/watch?v=2uYL4az1BVU</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2813ef2df537" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CLI tools for validating and linting YAML files.]]></title>
            <link>https://faun.pub/cli-tools-for-validating-and-linting-yaml-files-5627b66849b1?source=rss-102450e29029------2</link>
            <guid isPermaLink="false">https://medium.com/p/5627b66849b1</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[yaml]]></category>
            <category><![CDATA[linter]]></category>
            <category><![CDATA[continuous-integration]]></category>
            <category><![CDATA[validation]]></category>
            <dc:creator><![CDATA[Pritesh Tailor]]></dc:creator>
            <pubDate>Mon, 21 Feb 2022 10:56:35 GMT</pubDate>
            <atom:updated>2022-02-25T20:31:21.962Z</atom:updated>
            <content:encoded><![CDATA[<h3>CLI tools for validating and linting YAML files</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/694/1*KEGTj2RZLzRhkfoZSIjYIA.png" /><figcaption>yq and yamlint in action, captured by me.</figcaption></figure><p>YAML (<strong>YAML</strong> Ain’t Markup Language) is a frequently used data serialization language that is often used as configs for tools such as Kubernetes, Jenkins and serverless framework, it is supported in some fashion by most popular programming languages.</p><p>More often than not we are keeping these YAML files in version control…so how do we enforce a consistent format/style and ensure we always push up validated YAML files. If we have a strict schema, how would you check 100s of YAMLs in one batch or quickly check a single file on the CLI is valid. Why wait for your app to tell you a YAML file is wrong when deployed when you can do it quickly on the CLI?</p><p>In an IDE there is no problem to solve since many plugins will point out errors straight away, but when we are working in the terminal or with CI/CD pipelines or any automated use of YAML files, this is a luxury we don’t have. So what can we do to make our life easier?</p><h3><strong>Yaml Validation (and formatting) tools.</strong></h3><p>Thankfully many CLI YAML validation tools exist in various forms for formating to set standards:</p><p><strong><em>yq</em></strong>: <a href="https://github.com/mikefarah/yq">https://github.com/mikefarah/yq</a></p><p><strong><em>yq</em></strong> is a great tool for validation as well as auto-configuring the format of your YAML file such as setting the indentation size. <em>It can be installed by popular linux package managers.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/710/1*EnvIN5W3nWIWscGQGV71Mw.png" /><figcaption>yq in action formatting a YAML file to have an indentation of 2 spaces.</figcaption></figure><p><strong><em>yamllint</em></strong>: <a href="https://github.com/adrienverge/yamllint">https://github.com/adrienverge/yamllint</a></p><p><strong>yamllint</strong> sets a much stricter standard and is highly customisable...to quote from the repo “<strong><em>yamllint does not only check for syntax validity, but for weirdnesses like key repetition and cosmetic problems such as lines length, trailing spaces, indentation, etc</em></strong>”. You can configure your style using rules in a config file to fit your needs and standards, perfect for CI/CD pipelines. <em>It can be installed by popular linux package managers and with pip.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/825/1*2IykgJV3gabdnT9A0AcRSg.png" /><figcaption>yamllint in action, all the above rules could be customised via a config file if you&#39;re fussy.</figcaption></figure><p><strong><em>yamale</em></strong>: <a href="https://github.com/23andMe/Yamale">https://github.com/23andMe/Yamale</a></p><p><strong><em>yamale</em></strong> has both a CLI and API for validating YAML with schemas (see below example when a schema is set via a schema.yml file)in a similar spirit to yamllint but much more rigid rules. If you are reading in many files <strong><em>yamale</em></strong> supports parallel(CLI) as well as the ability for it to be imported into Python code which can be incredibly useful depending on your YAML validation task where integration is needed. <strong><em>yamale</em></strong> is particularly powerful when you have a set schema and ingest several thousands of YAML files as a part of a data pipeline. <em>It can be installed via pip or downloaded from its repo.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/984/1*zv1r4BJtrlVTGp0jdVOlqg.png" /><figcaption>yamale in action, we define scheme.yml for our YAML files and we see when running it on file which conforms to the schema a validation success and one which does fail because I defined my age as a string, not an integer.</figcaption></figure><p><strong><em>yaml-validator</em></strong>: <a href="https://github.com/paazmaya/yaml-validator">https://github.com/paazmaya/yaml-validator</a></p><p><strong><em>yaml-validator </em></strong>this command-line tool provides a straight-up validation of a YAML file as well as being backed up with the ability to do validations in node scripts on top. <em>It can be installed via the npm package manager.</em></p><h3>Alternative Validation with “one-liners”</h3><p>In addition to the off the shelf CLI tool, several programming languages can be utilised to do validation as what I like to call ‘one-liners’. *</p><p>For Python3 (with pyyaml installed with a pip3 install pyyaml):</p><p>python3 -c “from yaml import load, Loader; load(open(‘test.yml’), Loader=Loader)”</p><p>For Ruby (with the default ruby on OSs):</p><p>ruby -ryaml -e “p YAML.load(STDIN.read)” &lt; test.yml</p><p>For Perl:</p><p>perl -MYAML -e ‘use YAML;YAML::LoadFile(“./test.yml”)’</p><p><em>*These one-liners were found from my research online. Credit to the following StackOverflow page contributors: </em><a href="https://stackoverflow.com/questions/3971822/how-do-i-validate-my-yaml-file-from-command-line"><em>https://stackoverflow.com/questions/3971822/how-do-i-validate-my-yaml-file-from-command-line</em></a></p><h3>Concluding Remarks</h3><p>There are quite a few ways for the validation, formatting and linting of YAML files, some offer quick and simple checks like the one-liners. Many others provide varying levels of rules and customisation ideal for version control, CI/CD and data pipelines for better quality control and strict schemas.</p><figure><a href="https://faun.to/bP1m5"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BCiLLad3dvZLwBa-B5cAVQ.png" /></a></figure><p>Join FAUN: <a href="https://faun.to/i9Pt9"><strong>Website</strong></a><strong> </strong>💻<strong>|</strong><a href="https://faun.dev/podcast"><strong>Podcast</strong></a><strong> </strong>🎙️<strong>|</strong><a href="https://twitter.com/joinfaun"><strong>Twitter</strong></a><strong> </strong>🐦<strong>|</strong><a href="https://www.facebook.com/faun.dev/"><strong>Facebook</strong></a><strong> </strong>👥<strong>|</strong><a href="https://instagram.com/fauncommunity/"><strong>Instagram</strong></a><strong> </strong>📷|<a href="https://www.facebook.com/groups/364904580892967/"><strong>Facebook Group</strong></a><strong> </strong>🗣️<strong>|</strong><a href="https://www.linkedin.com/company/faundev"><strong>Linkedin Group</strong></a><strong> </strong>💬<strong>|</strong> <a href="https://faun.dev/chat"><strong>Slack</strong></a> 📱<strong>|</strong><a href="https://thechief.io/"><strong>Cloud Native</strong> <strong>News</strong></a><strong> </strong>📰<strong>|</strong><a href="https://linktr.ee/faun.dev/"><strong>More</strong></a><strong>.</strong></p><p><strong>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5627b66849b1" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/cli-tools-for-validating-and-linting-yaml-files-5627b66849b1">CLI tools for validating and linting YAML files.</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Changing the Encryption Type of All Objects in a  S3 Bucket]]></title>
            <link>https://aws.plainenglish.io/change-encryption-type-of-all-objects-in-s3-bucket-4002070a1fb7?source=rss-102450e29029------2</link>
            <guid isPermaLink="false">https://medium.com/p/4002070a1fb7</guid>
            <category><![CDATA[aws-cli]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[storage]]></category>
            <category><![CDATA[encryption]]></category>
            <category><![CDATA[s3]]></category>
            <dc:creator><![CDATA[Pritesh Tailor]]></dc:creator>
            <pubDate>Mon, 14 Feb 2022 22:44:27 GMT</pubDate>
            <atom:updated>2022-02-18T19:09:30.616Z</atom:updated>
            <content:encoded><![CDATA[<h3>Changing the Encryption Type of All Objects in a S3 Bucket</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/677/1*r5_-PbU4fNoSsEzaWN3YNQ.png" /></figure><p><strong>tl;dr</strong>: This is a guide to change the encryption type of all objects in a bucket for the following cases:</p><ol><li><strong>SSE-S3 → SSE-KMS</strong></li><li><strong>SSE-KMS → SSE-S3</strong></li><li><strong>SSE-KMS</strong> → <strong>SSE-KMS</strong> (When the first KMS is different to the second one)</li><li>no encryption → <strong>SSE-S3</strong></li><li>no encryption → <strong>SSE-KMS</strong></li></ol><p><strong>First Point</strong>: AWS S3 provides no direct way to change the encryption type of all objects in a bucket. Changing the default encryption of a bucket only changes the encryption of new objects uploaded, all existing ones remain with the old encryption setting.</p><p><strong>Second Point</strong>: A problem I encountered recently was “<em>oversetting up</em>” unnecessary settings for encryption on a bucket. It was encrypting all objects with KMS by default as <strong>SSE-KMS</strong> and only to realise over time what I really needed was <strong>SSE-S3</strong>. Although <strong>SSE-KMS </strong>is useful this was overkill for my use case and would end up incurring costs overtime unnecessarily and annoying dependencies on KMS when we start getting IAM policies involved.</p><p>I already had thousands of objects in the bucket so how I do solve my issue to change encryption of all current objects from <strong>SSE-KMS </strong>to<strong> SSE-S3 </strong>and ensure all future objects are encrypted with SSE-S3. <em>Note: to re-iterate that just changing the default encryption of objects in a bucket does nothing to the existing object’s encryption type.</em></p><p><strong>Point 3</strong>: How do we do this with the <strong>AWS CLI</strong>?</p><blockquote><em>Note: we assume versioning is switched off and I recommend running first with an additional arg of </em>--dryrun flag at the end of your command so you can see what will happen first before you do a real run.</blockquote><p><strong>First Step</strong>: set the encryption type default of the bucket to what you would like it to be with one of the following commands:</p><p>For <strong>SSE-S3</strong>:</p><pre>aws s3api put-bucket-encryption --bucket <em>bucket-name</em> --server-side-encryption-configuration &#39;{<br>    &quot;Rules&quot;: [<br>        {<br>            &quot;ApplyServerSideEncryptionByDefault&quot;: {<br>                &quot;SSEAlgorithm&quot;: &quot;AES256&quot;<br>            }<br>        }<br>    ]<br>}&#39;</pre><p>For <strong>SSE-KMS</strong>, (For Default KMS Key):</p><pre>aws s3api put-bucket-encryption --bucket <em>bucket-name</em> --server-side-encryption-configuration &#39;{<br>        &quot;Rules&quot;: [<br>            {<br>                &quot;ApplyServerSideEncryptionByDefault&quot;: {<br>                    &quot;SSEAlgorithm&quot;: &quot;aws:kms&quot;<br>                }<br>            }<br>        ]<br>    }&#39;</pre><p>For <strong>SSE-KMS</strong>, (Self managed KMS Key):</p><pre>aws s3api put-bucket-encryption --bucket <em>bucket-name</em> --server-side-encryption-configuration &#39;{<br>        &quot;Rules&quot;: [<br>            {<br>                &quot;ApplyServerSideEncryptionByDefault&quot;: {<br>                    &quot;SSEAlgorithm&quot;: &quot;aws:kms&quot;,<br>                    &quot;KMSMasterKeyID&quot;: &quot;arn:aws:kms:eu-west-2:22222333333333:key/90258e51-2113-8121-jj45-127gdw2d323367&quot;<br>                }<br>            }<br>        ]<br>    }&#39;</pre><p>For removing encryption:</p><pre>aws s3api delete-bucket-encryption --bucket bucket-name</pre><p><strong>Second Step</strong>: Copy all objects in place recursively:</p><p>aws s3 cp s3://bucket-name/ s3://bucket-name/ --recursive</p><p>All done, all objects will now be the encryption type of your choosing.</p><p><strong>Point 4</strong>: How would we do this if I wanted a specific prefix changed?</p><p>Simple, you can do it in a single command here. We do it for the example prefix s3://<em>bucket-name</em>/<em>prefix</em>/.</p><p>For <strong>SSE-S3</strong>:</p><pre>aws s3 cp s3://<em>bucket-name</em>/<em>prefix</em>/ s3://<em>bucket-name</em>/<em>prefix</em>/ --sse AES256 --recursive</pre><p>For <strong>SSE-KMS</strong>, (For Default KMS Key):</p><pre>aws s3 cp s3://<em>bucket-name</em>/<em>prefix</em>/ s3://<em>bucket-name</em>/<em>prefix</em>/ --sse aws:kms --recursive</pre><p>For <strong>SSE-KMS</strong>, (Self managed KMS Key):</p><pre>aws s3 cp s3://<em>bucket-name</em>/<em>prefix</em>/ s3://<em>bucket-name</em>/<em>prefix</em>/ --sse aws:kms --recursive --sse-kms-key-id arn:aws:kms:eu-west-2:22222333333333:key/90258e51-2113-8121-jj45-127gdw2d323367</pre><p><strong>Point 4: </strong>Additionally, if you are intending on using KMS, you may be interested in using bucket keys. These can save you a lot in KMS costs by cutting down on the number of calls to KMS when interacting with objects. To set up your bucket to use bucket keys on top of KMS, modify the put-bucket-encryption to be like the below for the previous commands.</p><pre>aws s3api put-bucket-encryption --bucket <em>bucket-name</em> --server-side-encryption-configuration &#39;{<br>        &quot;Rules&quot;: [<br>            {<br>                &quot;ApplyServerSideEncryptionByDefault&quot;: {<br>                    &quot;SSEAlgorithm&quot;: &quot;aws:kms&quot;,<br>                    &quot;KMSMasterKeyID&quot;: &quot;arn:aws:kms:eu-west-2:22222333333333:key/90258e51-2113-8121-jj45-127gdw2d323367&quot;<br>                },<br>                &quot;BucketKeyEnabled&quot;: true<br>            }<br>        ]<br>    }&#39; </pre><p><strong>Final Point:</strong> There you have it, a quick few commands for changing the encryption of objects in a bucket. If you’re on the scale of millions or billions of objects, you may need to consider using S3 Batch Operations.</p><p><em>More content at </em><a href="http://plainenglish.io/"><strong><em>plainenglish.io</em></strong></a><em>. Sign up for our </em><a href="http://newsletter.plainenglish.io/"><strong><em>free weekly newsletter</em></strong></a><em>. Get exclusive access to writing opportunities and advice in our </em><a href="https://discord.gg/GtDtUAvyhW"><strong><em>community Discord</em></strong></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4002070a1fb7" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/change-encryption-type-of-all-objects-in-s3-bucket-4002070a1fb7">Changing the Encryption Type of All Objects in a  S3 Bucket</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Disposable Virtual Machines in 3 commands with Multipass]]></title>
            <link>https://faun.pub/disposable-virtual-machines-in-3-commands-with-multipass-ab5afc12eba9?source=rss-102450e29029------2</link>
            <guid isPermaLink="false">https://medium.com/p/ab5afc12eba9</guid>
            <category><![CDATA[multipass]]></category>
            <category><![CDATA[development]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[virtual-machine]]></category>
            <dc:creator><![CDATA[Pritesh Tailor]]></dc:creator>
            <pubDate>Sat, 29 Jan 2022 18:11:28 GMT</pubDate>
            <atom:updated>2022-01-29T18:11:28.076Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ote5PGMms_eM7_Oc" /><figcaption>Photo by <a href="https://unsplash.com/@timothycdykes?utm_source=medium&amp;utm_medium=referral">Timothy Dykes</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Virtual environments are invaluable tools for developers be they data scientists, software/cloud/DevOps engineers and system administrators. They provide you with quick disposable environments for experimentation, testing and endless fresh starts.</p><p><strong>Multipass</strong> released by Canonical has presented itself as a fresh and easy to use tool inside of this domain. Which I now use on a daily basis over similar tools for VMs! So what exactly makes it so great? (Beyond VMs for M1 MacBooks!)</p><h4><strong>Simple to install</strong></h4><p>It is a one-liner to install in most cases:</p><ul><li>Linux: sudo snap install multipass</li><li>Mac: brew install multipass</li><li>Windows: there is a link <a href="https://multipass.run/download/windows">Download Multipass for Windows</a></li></ul><h4>Simple to use</h4><p>You can manage the entire VM lifecycle with 3 commands:</p><ul><li>Spin up an ubuntu 20.04 VM multipass launch 20.04</li><li>Enter the VM terminal prompt multipass shell &lt;vm name&gt;</li><li>Delete the VM multipass delete &lt;vm name&gt;</li></ul><h4>Simple VM customisation</h4><ul><li>launch VM with 2 cpus multipass launch 20.04 --cpus 2</li><li>launch VM with 1GB memory multipass launch 20.04 --mem 1G</li><li>launch VM with 5GB disk multipass launch 20.04 --disk 5G</li></ul><h4>Simple file transfers</h4><ul><li>mount a host directory to VM multipass mount /host/path &lt;vm name&gt;:/some/path</li><li>transfer a file from host to VM multipass transfer script.sh &lt;vm name&gt;:/tmp/script.sh</li><li>transfer a file from VM to host multipass transfer &lt;vm name&gt;:/root/script.sh /path/on/host</li></ul><h4>Simple clean up (and recovery)</h4><ul><li>Delete a VM multipass delete &lt;vm name&gt;</li><li>Permanently delete all VMs multipass purge</li><li>Recover a deleted VM multipass recover &lt;vm name&gt;</li></ul><h4>Multipass in Action</h4><p>Here is a demo of multipass in action to show how simple it is.</p><pre>#Find available VM images<br>$ multipass find<br>Image Aliases Version Description<br>18.04 bionic 20220104 Ubuntu 18.04 LTS<br>20.04 focal,lts 20220111 Ubuntu 20.04 LTS<br>21.04 hirsute 20220113 Ubuntu 21.04<br>21.10 impish 20220111 Ubuntu 21.10</pre><pre>#Spin up an ubuntu 20.04 instance<br>$ multipass launch 20.04<br>Launched: glowing-ibex</pre><pre>#Run uname command to check kernel version in VM<br>$ multipass exec glowing-ibex -- uname -rp<br>5.4.0-94-generic aarch64</pre><pre>#Delete VM<br>$ multipass delete glowing-ibex</pre><pre>#Permanently delete VM <br>$ multipass delete glowing-ibex<br>$ multipass ls<br>Name                    State             IPv4             Image<br>glowing-ibex            Deleted           --               Not Available<br>$ multipass purge<br>$ multipass ls<br>No instances found.</pre><h4>To Conclude</h4><p>Multipass is a super simple but powerful tool to spin up VMs and destroy them in very few CLI lines. Its beauty is in its simplicity, as you can imagine it saves you time by letting you spin up VMs for quick experiments locally rather than on cloud instances, it could provide a disposable environment to run a local data pipeline or even form a part of CI/CD pipeline. You could use it to test your DevOps configuration scripts or experiment with cluster technology setups. Of course, your creativity is your limit, just know it&#39;s easy from start to finish with Multipass.</p><figure><a href="https://faun.to/bP1m5"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BCiLLad3dvZLwBa-B5cAVQ.png" /></a></figure><p>Join FAUN: <a href="https://faun.to/i9Pt9"><strong>Website</strong></a><strong> </strong>💻<strong>|</strong><a href="https://faun.dev/podcast"><strong>Podcast</strong></a><strong> </strong>🎙️<strong>|</strong><a href="https://twitter.com/joinfaun"><strong>Twitter</strong></a><strong> </strong>🐦<strong>|</strong><a href="https://www.facebook.com/faun.dev/"><strong>Facebook</strong></a><strong> </strong>👥<strong>|</strong><a href="https://instagram.com/fauncommunity/"><strong>Instagram</strong></a><strong> </strong>📷|<a href="https://www.facebook.com/groups/364904580892967/"><strong>Facebook Group</strong></a><strong> </strong>🗣️<strong>|</strong><a href="https://www.linkedin.com/company/faundev"><strong>Linkedin Group</strong></a><strong> </strong>💬<strong>|</strong> <a href="https://faun.dev/chat"><strong>Slack</strong></a> 📱<strong>|</strong><a href="https://thechief.io/"><strong>Cloud Native</strong> <strong>News</strong></a><strong> </strong>📰<strong>|</strong><a href="https://linktr.ee/faun.dev/"><strong>More</strong></a><strong>.</strong></p><p><strong>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ab5afc12eba9" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/disposable-virtual-machines-in-3-commands-with-multipass-ab5afc12eba9">Disposable Virtual Machines in 3 commands with Multipass</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>