<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ankit Choudhary on Medium]]></title>
        <description><![CDATA[Stories by Ankit Choudhary on Medium]]></description>
        <link>https://medium.com/@ankitblogs?source=rss-f07f96ca111d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*lsPPjzoUZ_QSwuhswtGN0A.jpeg</url>
            <title>Stories by Ankit Choudhary on Medium</title>
            <link>https://medium.com/@ankitblogs?source=rss-f07f96ca111d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 07:00:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ankitblogs/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building a Desktop in 2020 vs 2010]]></title>
            <link>https://ankitblogs.medium.com/building-a-desktop-in-2020-vs-2010-1f4cfd94add0?source=rss-f07f96ca111d------2</link>
            <guid isPermaLink="false">https://medium.com/p/1f4cfd94add0</guid>
            <category><![CDATA[pc]]></category>
            <category><![CDATA[gaming]]></category>
            <category><![CDATA[computers]]></category>
            <category><![CDATA[build]]></category>
            <category><![CDATA[desktop]]></category>
            <dc:creator><![CDATA[Ankit Choudhary]]></dc:creator>
            <pubDate>Thu, 31 Dec 2020 21:11:23 GMT</pubDate>
            <atom:updated>2020-12-31T21:11:23.251Z</atom:updated>
            <content:encoded><![CDATA[<p>The transformation of technology over the last decade…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*7tXkYhe8q-X_X1BknQUfag.png" /></figure><p>The last time I had built and used a personal desktop computer was at least a decade before (somewhere between 2008–2010). Since then, life has been pacing fast enough, fueled by portable devices like laptops, tablets and smart phones. Few weeks earlier, I decided to setup a flight simulator at my home to aid my ongoing private pilot training. I planned on using the Microsoft Flight Simulator 2020 and clearly my personal laptop wasn’t capable to perform as per the recommended configuration specs even per the minimum specs. So, I decided to build a new desktop computer and explore the trending technologies prevailing in the world of desktops.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F2mnonGpPRdc%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D2mnonGpPRdc&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F2mnonGpPRdc%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/1780cc1234862d362f1b4f121e55a09e/href">https://medium.com/media/1780cc1234862d362f1b4f121e55a09e/href</a></iframe><p>Undoubtedly, I was captivated by the evolution of technology. Though the components remain the same, the form factor and the performance has improved exponentially over the past decade. Let’s look at the few components of a desktop of 2020 vs the one of 2010 which underwent serious transformations.</p><h3>Processor:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/479/1*xBggkHd-V-Qoy0sKXWuybg.png" /></figure><p>The processing power of the CPUs have evolved in multiple magnitudes. The earlier CPUs had dual (2) cores whereas the latest processor released (Comet Lake series) have been manufactured with deca (10) cores, eventually multiplying the threads from 4 to 20. Similarly, the processor frequency has risen up from ~2.9GHz to 3.6GHz. Below is a high level comparison of a processor from now and then.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/783/1*bJ_k48UrGZLAzKKsX_AfqQ.png" /></figure><p>I found the below image through a Google Search that shows the various generations of Intel Processors. The 10th generation processors have been termed as “Comet Lake”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/739/1*AMCWOFziFhooUz9NaCXZGg.png" /></figure><h3>RAM:</h3><p>If I remember correctly, the RAM of my 2010 desktop was 4GB. It was a pair of 2GB SDRAMs. The latest ones are the DDR4 SDRAMs. They offer higher data rate, transfer speed as well as working speed. The capacity of each RAM stick now ranges from 8GB — 32GB. In my build, I used 4x 16GB RAM totaling to 64GB,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gRAlF4S7eft0JiEtiZO69g.png" /></figure><p>The RAM sticks available in today’s world are also available with RGB lightings to make the Desktop even more appealing. The RGB lighting on the RAM sticks can be controlled via the Motherboard and can be synced with the other RGB lights on the entire build.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*YIjoH5LPNkfrKzXfEQmqSw.png" /><figcaption>2020- ARGB enabled DDR4 SDRAM</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*HH_Ebzjga_jkLZzAnUGKgA.png" /><figcaption>2010 — SDRAM</figcaption></figure><h3>Storage:</h3><p>This component completely blew off my mind. The bulky 3.5&quot; HDDs with spinning disks have been replaced with thin wafer chips. HDDs were succeeded with SSDs followed by the latest M2-NVME (Non Volatile Memory Express). The data read/write speed has multiplied with each innovation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TM0o_g_j5ZdtdRt3.jpg" /><figcaption>Image Source: Google Search</figcaption></figure><p>While traditional HDD’s speed is limited to 160 MB/s, SATA III SSD’s to 600 MB/s, the NVMe M.2 SSD has a speed of about 3500 MB/s. The NVMe is actually seven times faster than the SATA HDDs and four times faster than the SATA SSDs. For my build, I used 2 x 1TB NVMe chips.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*EJlh6yM7fZRCej3p.png" /></figure><p>And just to give you an idea of the size of the chip, this is how it looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/474/0*BpaAlhRYD0teDsAB" /></figure><h3>Cooling:</h3><p>Lastly, I also wanted to talk about the advancements made towards the CPU cooling. Back in 2010, case and processor fans were the only ways to keep the CPU cool. Today, liquid cooling has taken up the heavy lifting task of cooling the hard working CPU. The AIO (All in One) liquid cooling systems are highly efficient in dissipating the generated heat out of the CPU.</p><p>And definitely, the aesthetics of a liquid cooling system makes it a better choice. However, liquid cooling comes with a cost.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*FGJlUdF-J-TkKP-U.jpg" /></figure><p>This article on PCGuide.com is a good read to help you decide between the either systems :</p><blockquote>If you want good-to-great cooling performance without spending too much money, then air cooling is your best option. If you want good-to-great cooling performance while keeping noise levels low, then closed-loop liquid cooling is your best option. Arguably the most important of all, if you just think liquid cooling looks cool, especially with RGB, then liquid cooling is your best option.</blockquote><p>Similarly there are a lot more advancements, like the Graphics Processing Unit (GPU), the Power Supply Unit(PSU), the motherboard itself, etc. However, I wanted to limit this to a short blog and outline the major evolutions that left me with gratified and thrilled.</p><p>I am leaving below the specs of the desktop that I built along with all the other details if it helps:</p><ol><li>CPU: Intel Core i9-10850K 3.6 GHz 10-Core Processor (<a href="https://www.newegg.com/intel-core-i9-10850k-core-i9-10th-gen/p/N82E16819118175?Item=N82E16819118175">$429.99 @ Newegg</a>)</li><li>CPU Cooler: Corsair-H100i RGB Platinum Liquid CPU Cooler (<a href="https://www.newegg.com/corsair-hydro-series-h100i-rgb-platinum-liquid-cooling-system/p/N82E16835181151?Item=N82E16835181151">$159.99 @ Newegg</a>)</li><li>Motherboard: Gigabyte-Z490 AORUS ELITE AC ATX LGA1200 Motherboard (<a href="https://www.newegg.com/p/N82E16813145193?Item=N82E16813145193">$169.99 @ Newegg</a>)</li><li>Memory: OLOy-WarHawk RGB 32 GB (2 x 16 GB) DDR4–3200 CL16 Memory — $520 (<a href="https://www.newegg.com/oloy-32gb-288-pin-ddr4-sdram/p/N82E16820821199?Item=N82E16820821199">$129.99 x 4 @ Newegg</a>)</li><li>Storage: Intel-660p Series 1.02 TB M.2–2280 NVME Solid State Drive (<a href="https://www.newegg.com/intel-665p-series-1tb/p/N82E16820167469?Item=N82E16820167469">$109.99 x 2 @ Newegg</a>)</li><li>Video Card: MSI GeForce GTX 1650 SUPER 4 GB GAMING X Video Card (<a href="https://www.newegg.com/msi-geforce-gtx-1650-gtx-1650-d6-gaming-x/p/N82E16814137527?Item=N82E16814137527">$230.81 @ Newegg</a>)</li><li>Case: MUSETEX 8 PCS RGB Fans ATX Mid-Tower Case (<a href="https://www.amazon.com/MUSETEX-Mid-Tower-Computer-Tempered-Management/dp/B08HN8YNCV/ref=asc_df_B08HN8YNCV/?tag=bingshoppinga-20&amp;linkCode=df0&amp;hvadid=&amp;hvpos=&amp;hvnetw=o&amp;hvrand=&amp;hvpone=&amp;hvptwo=&amp;hvqmt=e&amp;hvdev=c&amp;hvdvcmdl=&amp;hvlocint=&amp;hvlocphy=&amp;hvtargid=pla-4584482465837266&amp;psc=1&amp;tag=&amp;ref=&amp;adgrpid=1294125542223135&amp;hvadid=80882933901292&amp;hvnetw=o&amp;hvqmt=e&amp;hvbmt=be&amp;hvdev=c&amp;hvlocint=&amp;hvlocphy=&amp;hvtargid=pla-4584482465837266&amp;hydadcr=">$89.99 @ Amazon</a>)</li><li>Power Supply: RAIDMAX Vortex RX-800AE-V 800W Non Modular Power Supply (<a href="https://www.newegg.com/raidmax-vortex-rx-600ae-v-800w/p/1HU-0061-000A4?Item=9SIA9ZNB0R8424">$79.99 @ Newegg</a>)</li><li>Operating System: Microsoft Windows 10 Home OEM 64-bit (<a href="https://www.amazon.com/Windows-10-Home-USB-Lifetime/dp/B08JD187WM/ref=sr_1_4?dchild=1&amp;hvadid=78477708127916&amp;hvbmt=bb&amp;hvdev=c&amp;hvqmt=b&amp;keywords=windows+10+home+64+download&amp;qid=1609448449&amp;sr=8-4&amp;tag=mh0b-20">$119.99 @ Amazon</a>)</li></ol><p>And here’s a quick visual:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F2mnonGpPRdc%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D2mnonGpPRdc&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F2mnonGpPRdc%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/1780cc1234862d362f1b4f121e55a09e/href">https://medium.com/media/1780cc1234862d362f1b4f121e55a09e/href</a></iframe><blockquote>If you can show people how to build castles, make sure you do not neglect building and nurturing your own — Suzy Kassem</blockquote><p>Thanks for stopping by my last blog of 2020. I hope to continue my blogging stride in 2021 and beyond. Let’s build!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1f4cfd94add0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Python Package]]></title>
            <link>https://ankitblogs.medium.com/building-a-python-package-d3b9a6043b27?source=rss-f07f96ca111d------2</link>
            <guid isPermaLink="false">https://medium.com/p/d3b9a6043b27</guid>
            <category><![CDATA[simple]]></category>
            <category><![CDATA[package]]></category>
            <category><![CDATA[pypi]]></category>
            <category><![CDATA[easy]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Ankit Choudhary]]></dc:creator>
            <pubDate>Sun, 20 Dec 2020 04:08:15 GMT</pubDate>
            <atom:updated>2020-12-20T08:55:41.239Z</atom:updated>
            <content:encoded><![CDATA[<h4>A simple and easy to follow recipe!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/506/0*5y-NaAmz6J9v8eqo.png" /><figcaption>Courtesy of Google: Possibly the most suitable image for this blog :)</figcaption></figure><p>Few weeks earlier, I was tasked with building an application that had to be hosted as a python package. After numerous iterations of code reviews and pull requests, we deployed the final version and it was much different than the initial version.</p><p>So, what changed? The answer lies in my favorite quote from <a href="https://martinfowler.com/">Martin Fowler</a>:</p><blockquote>Any fool can write code that a computer can understand. Good programmers write code that humans can understand.</blockquote><p>In this blog, I am going to the share the recipe for creating a python package the right way. So, let’s get started…</p><p>A <strong>Python package</strong> is a collection of modules. Modules that are related to each other are mainly put in the same package. When a module from an external package is required in a program, that package can be imported and its modules can be put to use.</p><h3>Elements:</h3><p>Before we dive deep into the code structuring, let’s look at the basic housekeeping items you would need. Just a few items that I specifically wanted to talk about (of course not the entire list):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/791/1*Po68fvDZZniDPCv0JLWjJQ.png" /><figcaption>The key components in the journey</figcaption></figure><p><strong>Code Editors:</strong> While the <em>coding-nerds</em> might disagree, I highly recommend using a editor (IDE) for all your coding purposes. Code Editors like Pycharm (my fav) were built for a purpose and they have been around for their efficiency. With just the click of a button, you can perform essential activities like auto-indenting the code lines, reformatting the entire code file, optimizing the imports and much more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*1YEoyhqayG3wUTaOmfgaSg.png" /></figure><p>I am a huge fan of the inline warnings that helps you identify the mistakes then and there. If I had to speak for “<em>my why</em>”, the editors do a fabulous job of aligning the code to the Python standards (PEP-8) or atleast act as a catalyst in doing so. There’s just so much that I can write about it.</p><p>And needless to say, the integration of external scripts (say for linting) and version control (git, etc.) provides the muscle to do everything at one place. Now I understand, why editors are often termed as “<strong>Integrated</strong>” Development Environment (IDE).</p><blockquote><a href="https://medium.com/swlh/the-gates-of-code-quality-pylint-b8b38faab0ae?sk=bbc24b9c28cfaabb9dcf1e34a636f40c"><strong>Bonus</strong></a>: Checkout my <a href="https://medium.com/swlh/the-gates-of-code-quality-pylint-b8b38faab0ae?sk=bbc24b9c28cfaabb9dcf1e34a636f40c">blog</a> on the cookie cutter <a href="https://medium.com/swlh/the-gates-of-code-quality-pylint-b8b38faab0ae?sk=bbc24b9c28cfaabb9dcf1e34a636f40c">instructions</a> to integrate Pylint with Pycharm along with the different stages of using pylint efficiently <a href="https://medium.com/swlh/the-gates-of-code-quality-pylint-b8b38faab0ae?sk=bbc24b9c28cfaabb9dcf1e34a636f40c">here</a>.</blockquote><p>Version Control: Apart from safeguarding your development efforts from vanishing away because of a system failure, version control systems (like Github) is a must have for all code changes, especially with feature releases and bug fixes. Even after rigorous test cases and detailed code reviews, stumbling across unpredicted issues and bugs isn’t a surprise. Feasibility of rolling back to the previous versions is a non-negotiable ask.</p><p>Cloud hosting platforms for version control like <a href="https://github.com/">Github</a> lets you and others work together on projects from anywhere. The service <a href="https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks">hooks</a> enable seamless continuous integrations (CI). Integrating Github with a CI/CD platform (like <a href="https://codefresh.io/">Codefresh</a>, <a href="https://azure.microsoft.com/en-us/services/devops/">Azure Devops</a>, etc.) can automate the entire build pipeline. You can set up the pipelines to run the test cases, check for lints, build the package and deploy it to artifactory with every pushed commits.</p><p>In the below flow chart, I have outlined the important steps and how they fit into each components that we are discussing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/997/1*qyJMfeQsVECrJt4Gh-a69w.png" /><figcaption>Python Package Development Lifecycle</figcaption></figure><p>CI-CD Platform: The major chunk of packaging and shipping the package is undertaken by this component. As shown in the the above diagram, the CI cycle is kicked off by the Git service hooks (Push Commits/PR Merges/etc). The CI pipelines should consist of but no limited to the steps as shown, i.e. the unit tests, lint checks and build and publish of the packages.</p><p>The seamless integration of the modern version control hosts with the cloud based CI platforms make the entire lifecycle as continuous as it could be. I will continue to stress more on the Unit Tests and Lint Checks in the below sections.</p><p>Artifactory: Also known as Repository, this is the last component that falls into a the package development lifecycle. Think of it as a library where people can search for books and borrow them for use. Artifactories can be public (<a href="http://pypi.org/">pypi</a>) as well as private (inter-organizations, ex: JFROG platform hosted with a company’s IT infra). Artifactories store the published packages and make them available for use. It also maintains the versioning of the package, which is implicitly set by the CI pipelines.</p><p>Now that we are good with the housekeeping stuff, let’s proceed to look deeper into the package itself…</p><h3>Structure</h3><p>The bare bone structure for a package code should look like the below. Let’s call the package in discussion as “<em>my_simple_package</em>”.</p><pre>my-simple-package-root<br>├── my_simple_package<br>│   ├── __init__.py<br>│   ├── __main__.py<br>│   ├── module_a.py<br>│   ├── module_b.py<br>│   ├── -----------<br>│   ├── module_z.py<br>│   ├── my_simple_sub_package<br>│   |   ├── module_a.py<br>│   |   ├── module_b.py<br>├── tests<br>│   ├── test_module_a.py<br>│   ├── test_module_b.py<br>│   ├── ----------------<br>│   ├── test_module_z.py<br>│   ├── test_my_simple_sub_package<br>│   |   ├── test_module_a.py<br>│   |   ├── test_module_b.py<br>├── CHANGELOG.md<br>├── DEVELOPING.md<br>├── LICENSE<br>├── MANIFEST.in<br>├── README.md<br>├── requirements.txt<br>├── setup.cfg<br>├── setup.py</pre><h4>Root</h4><p>As illustrated above, the root directory should not contain any actual code files that goes into the package. Rather, the content to deployed as a part of the package should be present in a folder named as per the package name. The root folder should mostly contain all that’s needed to develop, support and test the package contained within it.</p><p>Let’s discuss each of the files briefly:</p><h4><strong>my_simple_package:</strong></h4><p>This is the core folder containing all the code lines that goes into the package. It is suggested that you design your modules (ex: module_a, module_b, etc.) in such a way that they can be independently tested. Further, the modules should be logically collated.</p><h4>__init__.py</h4><p>The __init__.py files are required to make Python treat the directories as containing packages; this is done to prevent directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path.</p><p>It is recommended to have the __init__.py empty. You can read more about the various structures <a href="http://It is recommended to have the __init__.py empty">here</a>.</p><p>I liked the below example referenced from <a href="https://www.reddit.com/r/learnpython/comments/fuxv57/can_init_py_actually_be_empty/">Reddit</a>:</p><p>Assuming your package looks like this:</p><pre>package<br>|-- __init__.py<br>|-- subpackage.py</pre><p>An empty __init__.py won&#39;t let you do this:</p><pre>import package<br>instance = package.subpackage.Class()</pre><p>But it <em>will</em> let you do this:</p><pre>import package.subpackage<br>instance = package.subpackage.Class()</pre><h4>__main__.py</h4><p>To execute a single file python program, we write the below if statements. Say you have a file named test_main.py</p><pre>def main():<br>  print(&quot;Hello Main&quot;)</pre><pre>if __name__ == __main__:<br>  main()</pre><p>When you run a program by calling the Python interpreter on it, the magic global variable __name__ gets set to __main__. Thus, when we execute python test_main.py, it executes fine.</p><p>But what if we had multiple modules (python package) and want to define an entry point to make the package ? This is done using the __main__.py. The magic file __main__.py is called when you run your project with the -m module flag. If you code is intended to be used a module first, and command line interface second, this makes perfect sense to use. Think of it as a place we can put whatever would be in our body of our if __name__ == __main__ statement.</p><p>Thus to run our example package, we can invoke it as :</p><pre>$ python -m my_simple_package</pre><h4>Module Files (module_a.py, etc.)</h4><p>Here goes all the core code for your package. Few things to consider while developing this:</p><ol><li>Keep functions independent of each other.</li><li>Ensure functions are logically grouped in modules.</li><li>Name the functions as such to convey what they do.</li><li>For functions, use lowercase with words separated by underscores as necessary to improve readability.</li><li>Use one leading underscore only for non-public methods and instance variables.</li><li>It’s a good practice to have the package split into multiple modules if the application is large enough</li></ol><h4>Tests (test_module_a.py, etc.)</h4><p>Perhaps the most important part of any package or application. Test cases assert the correctness of the code. You must have at least 75% code coverage. Code coverage is basically a percentage of count of lines that the test cases traversed to the overall lines in the code.</p><p>While the <a href="https://docs.python.org/3/library/unittest.html">core unit test module</a> would suffice writing and executing test cases, I highly recommend using pytest for unit tests. It makes writing tests super easy with all the friendly features like fixtures, parameterization and much more.</p><p>Let’s look at an example from the pytest docs:</p><pre><strong>import</strong> <strong>pytest</strong><br><br><br><strong>@pytest</strong>.mark.parametrize(&quot;test_input,expected&quot;, [(&quot;3+5&quot;, 8), (&quot;2+4&quot;, 6), (&quot;6*9&quot;, 42)])<br><strong>def</strong> test_eval(test_input, expected):<br>    <strong>assert</strong> eval(test_input) == expected</pre><p>In the above example, pytest-parameterize runs the test_eval method against the 3 sets of input and expected outputs. With other test frameworks, you would have to write these individually thrice! Read more about pytest <a href="https://docs.pytest.org/en/stable/">here</a>.</p><p>By rule of thumb, every change must be accompanied by test cases. And all test cases must be run to ensure coverage and certain that the code works correctly. This helps to detect and protect against bugs in the future.</p><h4>CHANGELOG.MD</h4><blockquote>In the software industry a changelog, as the name suggests, is a file that logs all the changes made to a specific software program. The reason for creating and keeping a changelog is simple; when a contributor or end-user wants to see if any changes have been made to a software program, they can do that easily and precisely by reading the changelog. All they need to do is go to the changelog and it will show what, and when, any changes were made between the different versions or releases of the particular software. — <a href="https://changelog.md/">Changelog.md</a></blockquote><h4>DEVELOPING.MD</h4><p>This file is intended to set the guidelines for all the contributors to the package. It should cover the below points:</p><ol><li>How to set up the development environment.</li><li>The development lifecycle.</li><li>Any development dependencies.</li><li>Running Examples.</li><li>Running Tests.</li><li>Adding Examples.</li><li>Requirements for merging PR.</li></ol><p>You can refer to sample ones on Github <a href="https://github.com/search?l=Markdown&amp;q=developing.md&amp;type=Code">here</a>.</p><h4>LICENSE</h4><p>It’s important for every package uploaded to the Python Package Index to include a license. This tells users who install your package the terms under which they can use your package. As suggested in the python-packaging, for help picking a license, you can refer to <a href="https://choosealicense.com/">choosealicense</a>.com</p><h4>MANIFEST.in</h4><p>A MANIFEST.in file consists of commands, one per line, instructing <a href="https://pypi.org/project/setuptools/">setuptools</a> to add or remove some set of files from the <a href="https://docs.python.org/3/distutils/sourcedist.html">sdist</a>. By default only a minimal set of files are included in the . You may find yourself wanting to include extra files in the source distribution, such as an authors/contributors file, data files used for testing purposes, screenshots for references. All these extra files must be added in the MANIFEST.in file.</p><p><em>Clarify, if there are some files to add or some files to exclude. If neither is needed, then there is no need for using </em><em>MANIFEST.in.</em></p><h4>README.md</h4><p>A <a href="https://en.wikipedia.org/wiki/README">README</a> is a text file that introduces and explains a project. It contains information that is commonly required to understand what the project is about.</p><p>The contents of the README.md typically includes one or more of the following:</p><ul><li>Configuration instructions</li><li>Installation instructions</li><li>Operating instructions</li><li>A file manifest (list of files included)</li><li>Copyright and licensing information</li><li>Contact information for the distributor or programmer</li><li>Known bugs</li><li>Troubleshooting</li><li>Credits and acknowledgments</li><li>A changelog (usually for programmers)</li><li>A news section (usually for users)</li></ul><p>Do look at <a href="https://www.makeareadme.com/">makeareadme.com</a></p><h4>requirements.txt</h4><p>Python’s package is all about reusability. This requirements.txt file is used for specifying what other python packages is the current project dependent on.</p><p>Example requirements.txt:</p><pre>Flask==0.8<br>Jinja2==2.6<br>Werkzeug==0.8.3<br>certifi==0.0.8<br>chardet==1.0.1<br>distribute==0.6.24<br>gunicorn==0.14.2<br>requests==0.11.1</pre><p><strong>Note:</strong></p><ol><li>It should contain <strong>only </strong>the required packages.</li><li>Specify a minimum version for each dependent package.<br>i.e. use Flask==0.8 instead of just Flask. This binds the codebase to a minimal version of the dependent packages and hence avoiding any code breaks or bugs.</li></ol><h4>setup.cfg &amp; setup.py</h4><p>Python projects are packaged using <a href="https://pypi.org/project/setuptools/">setuptools</a>. setup.py is the build script for <a href="https://packaging.python.org/key_projects/#setuptools">setuptools</a>. It tells setuptools about your package (such as the name and version) as well as which code files to include.Setuptools allows using configuration files (usually setup.cfg) to define a package’s metadata and other options that are normally supplied to the setup() function (declarative config).</p><p>Example setup.cfg :</p><pre><strong>[metadata]</strong><br>name <strong>=</strong> my_simple_package<br>version <strong>=</strong> 1.0.0<br>description <strong>=</strong> My Simple Package&#39;s Description<br>long_description <strong>=</strong> file: README.md, CHANGELOG.MD, DEVELOPING.MD, LICENSE</pre><pre><strong>[options]</strong><br>zip_safe <strong>=</strong> False<br>include_package_data <strong>=</strong> True<br>packages <strong>=</strong> find:<br>install_requires <strong>=</strong><br>    requests==2.24.0</pre><pre><strong>[options.packages.find]</strong><br>where <strong>=</strong><br>    my_simple_package</pre><pre><strong>[options.extras_require]</strong><br>pdf <strong>=</strong> ReportLab&gt;=1.2; RXP<br>rest <strong>=</strong> docutils&gt;=0.3; pack ==1.1, ==1.3</pre><p>Example setup.py :</p><pre>import setuptools<br><br>if __name__ == &quot;__main__&quot;:<br>    setuptools.setup()</pre><p>setuptools.setup() by default, looks for the presence of setup.cfg in the same folder. Configs provided in the setup.cfg are essentially the parameters to be passed to setuptools.setup . Hence, if the project contains a setup.cfg as well as parameters passed to setuptools.setup(), the parameters override the values passed in the setup.cfg.</p><p>An extensive reference sheet for the keywords available to use under setup.cfg can be found <a href="https://setuptools.readthedocs.io/en/latest/userguide/declarative_config.html">here</a>.</p><p>This summarizes the extensive list of files/resources that goes into a standard Python Package. Let’s move to the last section that details on the steps to build and publish your developed package to the artifactory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-XatY9G2-JX_lh_L.jpeg" /></figure><h3>Build &amp; Publish</h3><p>After you have tested and versioned your code, it’s now ready for publishing it to an artifactory. The steps for publishing a package are same irrespective of the artifactory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/691/1*xSTXafK_rIXZEXfWPqVRnQ.png" /><figcaption>The final step; build and deploy.</figcaption></figure><p><strong>Build Package</strong></p><p>You can choose to make a source distribution of your package by running:</p><p>python setup.py sdist</p><p>or</p><p>You can also choose to make a build distribution <a href="https://realpython.com/python-wheels/">wheel</a> for your package by running:</p><p>python setup.py bdist_wheel</p><p><strong>Publish Package</strong></p><p>You can use twine to upload your package to PyPI or any other artifactory. Install twine package first, and then publish the package using the command below:</p><pre>twine upload dist/*</pre><p>You will be prompted to enter your username and password. If the upload is successful, it will spit out the URL for your package.</p><p>And now, your package is available on the artifactory!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1002/0*36_ikIdSVp_f-M99.jpg" /></figure><p>Building a Python Package may seem challenging and horrifying for the first time… or at least I felt it that way when I first built one. However, with the right approach, things are fairly simple and easy to understand.</p><p>I hope this blog was helpful. Looking forward to hear your comments and suggestions.</p><p>Cheers!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d3b9a6043b27" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Private Pilot License on H1B]]></title>
            <link>https://ankitblogs.medium.com/private-pilot-license-on-h1b-558224fba4b2?source=rss-f07f96ca111d------2</link>
            <guid isPermaLink="false">https://medium.com/p/558224fba4b2</guid>
            <category><![CDATA[tsa]]></category>
            <category><![CDATA[india]]></category>
            <category><![CDATA[pilot]]></category>
            <category><![CDATA[afsp]]></category>
            <category><![CDATA[h1b]]></category>
            <dc:creator><![CDATA[Ankit Choudhary]]></dc:creator>
            <pubDate>Sat, 31 Oct 2020 22:50:19 GMT</pubDate>
            <atom:updated>2020-10-31T22:50:19.759Z</atom:updated>
            <content:encoded><![CDATA[<p>(Things you should know before applying!)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*joIz6MI3YLoIKvt3.jpg" /></figure><p><strong>Airplanes &amp; Motorcycles</strong>, the 2 things that drive me crazier. Flying an aircraft has been my dream since I was 8. I have been chasing my pilot dreams for the past 22 years. I still remember the days where I used to fill my diary with the cut outs of the airplane pictures from the daily newspapers. Financial limitations, lack of opportunities and career options were a few key reasons leading to the pushback of my dreams, but my passion for flying had kept the dream alive.</p><p>Good things take time. So true! After spending close to 2.5 years in the United States of America, I finally took the first step towards pursuing my dream. And am not stepping back…. not at all.</p><p>I am on a H1B visa. Getting a Pilot’s License in the US for non-US Citizens has been made very strict and stringent. And that’s totally understood and justified, especially after the tragic incident of 9/11.</p><p>In this blog, I am going to share my experiences on the steps to get all the clearances and approvals that a non-US citizen must obtain prior to enrolling for a flight school in the US.</p><p>0: Introductory Flight Lesson. Take this. You don’t need a license to take this lesson. This is the first time you will get your hands on the yoke and the throttle. This ride makes you confident and sure about your decision of pursuing flying as a hobby/profession. It’s fun.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fzd3iYQjL8u0%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dzd3iYQjL8u0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fzd3iYQjL8u0%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e5f3841e131055419a699006cf4e4aa6/href">https://medium.com/media/e5f3841e131055419a699006cf4e4aa6/href</a></iframe><p>1: The Airport/Flying School — The first step is to find out an airport with a flight school where you want to train for your private pilot license. My recommendation would be to choose a school that is closer to your house. This gives you the freedom to frequently visit the airport and socialize with other students. And needless to say, you can accommodate for weather changes and plan your flight schedules better. And of course, do read the reviews of the school.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*huDvJWl46hWP-N4u" /></figure><p>Once you have decided the school where you want to train, you should let them know that you are planning to take on the Private Pilot’s License training and are considering their school. You should also clearly specify that you are a non-US citizen and that they should be expecting a correspondence from TSA (Transportation Security Administration).</p><p>2: The Aircraft. Choose the aircraft that you want to train on. The decision totally depends on you. However, there are a few things that might help you chose. For instance, if you weigh above 150 pounds, you should be definitely looking for planes like the Cessna 172, else the Cessna 152 should be good enough. The major difference is only in the hourly rental rates. Make sure to decide the aircraft type, before proceeding further.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*4-GupvrdXwmCmA_y.jpg" /></figure><p>Let’s agree to the fact that flying is an expensive hobby. The choice of aircraft would make a huge deal in your finances. Training in a 152 would bring down your hourly cost by at least $30 and that’s significant when your training is expected to last for 60–65 hours (US average).</p><p>Do consult your airport on what’s best for you. Also, try to get a feel of the cockpit before you zero down on the aircraft. Flight schools are very cooperative and won’t mind giving you a tour of the facility and the aircrafts. I tried both the Cessnas (152 &amp; 172) and I could clearly decide that I want to get trained in a 172.</p><p>3: Apply. Once you have all the above sorted out, you should start working towards the necessary paperwork needed prior to start training.</p><p>FAA (Federal Aviation Administration) has come with a program for the non-US citizens who are legally residing in the US to facilitate their flight training background checks a.k.a STA (Security Threat Assessment). For STA, the FAA has coined the program called AFSP (Alien Flight Student Program). One must create their account and provide their basic as well as resident/employment/etc. details. There’s no room for error here. All the details must be 100% accurate without any exceptions. I had made a foolish mistake here and it costed me a delay of 10 days. We will revisit my mistake in the later sections.</p><p>Get Ready with the below Documents. None of the documents should be expired.</p><ul><li>Passport.</li><li>US ID — Driver’s License.</li><li>US Visa</li><li>Latest H1B approval — I-797 (if the stamped US Visa on your passport is expired)</li></ul><p>Visit <a href="http://www.flightschoolcandidates.gov/">www.flightschoolcandidates.gov</a> and create your account, submit the relevant information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kpyZKK7xWrZV-AqxKQiRsQ.png" /></figure><p>Again, make sure to enter the information exactly as it appears on your documents.</p><p>Here comes the most challenging part where I had spent 35 days and exchanged over 20 emails with TSA trying to figure out what needs to be done when your initial visa has expired and you have an H1B extension/transfer approval. So, here’s the secret.</p><p>In the Section: Upload Documents, upload your visa as a separate entry and your H1B approval (I-797) as a separate entry. Upload only the latest H1B approval that you have.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7shIqaJww_OcPr3SHpXQnw.png" /></figure><p>For your expired US Visa, fill in the number as appears on your stamped Visa with the exact start and end dates. This entry is circled in Green in the above picture.</p><p>For the latest H1B approval, you should add a new entry selecting the category “U.S. Visa” and though you don’t already have a Visa number, enter the below:-</p><ul><li><strong>Visa Type </strong>- H1b — Temporary Worker In Speciality Occupation</li><li><strong>Visa Number </strong>- The I-94 Number printed at the bottom of the H1B Approval (I797). If you have an approved I-140, you must enter the A-Number here.</li><li><strong>Date Issued -</strong> The I-94 start date mentioned in I-797</li><li><strong>Expiration Date</strong> - The I-94 start date mentioned in I-797</li><li><strong>Status -</strong> Current</li><li><strong>Country of Residence when Issued -</strong> United States Of America</li></ul><p>Figuring out the above secret sauce was a whacking task. And I had promised myself that I would blog about what needs to be done if I succeed in getting my approvals so that someone else can benefit from my experience with it and save their precious time.</p><p>Once you are done filling all the required details and have uploaded the needed documents, the next step would be to create a training request.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vuMWccSnveDZiHqycsoLsA.png" /></figure><p>Select Category 3 candidates for training on aircraft weighing 12,500 lbs and under.</p><p>And fill in the details. You would be able to fill this without any issues if you have followed the steps outlined above.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eq2D6bY-j-Bb9yG2g3yhzg.png" /></figure><ul><li><strong>Where will you be training</strong>? — The state where your flying school is located.</li><li><strong>Who will you be training with?</strong>- The Flying school that you decided in Step-1.</li><li><strong>Estimated Start Date - </strong>My suggestion would be to enter a date at least 3 weeks later than the day you submit the form.</li><li><strong>Estimated End - </strong>Let’s set the max date possible here to have enough bandwidth. My research says this date can be max 365 days from the Start Date.</li><li><strong>Aircraft Type -</strong> Based on what you decide from Step 2.</li><li><strong>Class Name - </strong>Would be prefilled. Make sure it states “Initial License”</li><li><strong>Course ID -</strong> Leave it blank.</li></ul><p>Once the above is submitted, TSA reached out to the mentioned flying school to confirm their interest in training you. And here’s where you might wish to thank me for my advice on the later part of Step 1 :)</p><p>Once TSA, receives the confirmation from the flying school, you would be notified by an email and will be asked to make a non-refundable payment of $130 for further processing.</p><p>After the payment has been made, TSA-AFSP starts reviewing your documents and am sure if you have followed the steps correctly (Step 3), this would be done in not more than a week’s time.</p><p>The last step, is fingerprinting. Once AFSP reviews your application for accuracy and provides you the green light, you must submit your fingerprints at one of the enlisted centres. Schedule an appointment, be there at least 15 mins early, ensure you carry all the original documents that you had uploaded to the portal, along with a credit/debit card to make the payment.</p><p>If you remember, in the earlier sections I have stressed a lot about making sure that you check the filled in details again and again to ensure no mistakes. After my ever longing correspondence with TSA, when I was finally at the fingerprinting location, I was told that my Gender was showing up as “Female” in their database. Yes, I screwed up! I hadn’t paid attention to the drop down menu while filling up the form. I had to come back home and redo my application. And that’s all due to the silliest mistake I could ever do.</p><p>After you are done with your fingerprinting, expect an email from TSA with the golden subject “<strong>AFSP Training Request XXXXX: Final Approval</strong>”</p><p>That’s it! You are all set to start training from the moment you receive this email.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eocihIW0gUEYlvO-.png" /></figure><p>I am pretty excited about my upcoming training days where I will get to learn and fly an aircraft, all by myself. I wish you, the reader of this blog, all the best for your trainings.</p><p>I hope to write more on my experiences as I start training. Stay Tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=558224fba4b2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Gates of Code Quality — Pylint]]></title>
            <link>https://medium.com/swlh/the-gates-of-code-quality-pylint-b8b38faab0ae?source=rss-f07f96ca111d------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8b38faab0ae</guid>
            <category><![CDATA[coding-style]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[pylint]]></category>
            <category><![CDATA[standards]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Ankit Choudhary]]></dc:creator>
            <pubDate>Sun, 20 Sep 2020 18:07:51 GMT</pubDate>
            <atom:updated>2020-09-21T10:52:03.393Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Gates of Code Quality — Pylint</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UZnRBjdM98TVrL55.jpg" /></figure><p>A <strong>Code linter</strong>, is a static code analysis tool used to flag programming errors, bugs, stylistic errors, and suspicious constructs. The term originates from a Unix utility that examined C language source code. Like Google, “lint” became a verb that meant static checking your source code.</p><p>Everyone knows that programming errors are bad. Some errors cause glitches that frustrate users. Others compromise the safety and security of a critical system.</p><p>Linting is an essential process and code linters are available across programming languages. In this blog, we will talk specifically about the Python Linter aka <a href="https://www.pylint.org/">Pylint</a>. Pylint checks your code against the norms stated in the styling guide for Python as stated in <a href="https://www.python.org/dev/peps/pep-0008/">PEP-8</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/150/0*YpGC_cFUxP-mHwHD.png" /></figure><p><a href="https://www.pylint.org/"><strong>Pylint</strong></a> is a tool that checks for errors in Python code, tries to enforce a coding standard and looks for code smells. It can also look for certain type errors, it can recommend suggestions about how particular blocks can be refactored and can offer you details about the code’s complexity.</p><p>Pylint will display a number of messages as it analyzes the code and it can also be used for displaying some statistics about the number of warnings and errors found in different files. The messages are classified under various categories such as errors and warnings.</p><p>Last but not least, the code is given an overall <strong>mark</strong>, based on the number and severity of the warnings and errors</p><p>Gates of Implementation: A check when performed at multiple stages ensures no leakage. However the rule book (we will talk about it in the later sections) applied to each stage must be the same to avoid conflicts. Performing Pylint Checks in multiple stages also avoids rework and last minute changes/deviations from deployments/releases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zF7LzEHMY6Nv99PA7d4mKQ.png" /></figure><p>1: This stage is the closest to all the groundwork and forms the first layer of straining your code against the quality filter. <a href="https://www.jetbrains.com/pycharm/">Pycharm</a> stays the undebatable and undefeated champion of all the editors out there for Python. Integration of Pylint with Pycharm eases the lint checks with just the click of a button.</p><p>As and when you keep building your code, you should get habituated with the check-as-you-go way of running checks. For that, let’s leverage the “External Tool” feature of the editor. The steps to integrate Pylint in the editor have been beautifully shared in one of the stack overflow <a href="https://stackoverflow.com/questions/38134086/how-to-run-pylint-with-pycharm/46409649#46409649">answers</a>.</p><p>You can set up pylint to work with PyCharm by following the next steps:</p><p><strong>Install pylint:</strong></p><pre>$ pip install pylint</pre><p><strong>Locate your pylint installation folder:</strong></p><pre>$ which pylint # MacOS/Linux<br>/usr/local/bin/pylint # this is just a possible output check yours</pre><pre>$ where pylint # Windows %LocalAppData%\Programs\Python\Python36–32\Scripts\pylint.exe # possible location</pre><p>Open the PyCharm settings window with <strong>File -&gt; Settings</strong>, then navigate to <strong>Tools -&gt; External Tools</strong> in the sidebar. (Or search “external tools”)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BlMty71wMoXFwGTr.png" /></figure><p>Setup an external tool by clicking on the <strong>+</strong> sign and filling the fields accordingly. In Program use the path you got when running <strong>which pylint</strong>, for the other values you can use the same of the image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JOr0SF5GMWjQILRF.png" /></figure><p>To run pylint from Tools -&gt; External Tools -&gt; pylint:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jKWC1dYzJwE-kgbI.png" /></figure><p><strong>Look your output in the PyCharm terminal:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/0*wjFSe0mZy4Mox9tx.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yc699GMMbwth5oMMkr3k9A.png" /></figure><p>2: The second gate enforces the 1st level of governance in the code quality. This gate prevents a developer from contributing any code (having lints) to the GIT repository. Git has various <a href="https://docs.github.com/en/developers/webhooks-and-events/about-webhooks">web-hooks</a> that eases the development efforts. For the code linter, we will be using the pre-commit hook. It’s a simple script that gets triggered prior to any commit event.</p><blockquote>The pre-commit hook is run first, before you even type in a commit message. It’s used to inspect the snapshot that’s about to be committed, to see if you’ve forgotten something, to make sure tests run, or to examine whatever you need to inspect in the code. Exiting non-zero from this hook aborts the commit, although you can bypass it with git commit --no-verify. You can do things like check for code style (run lint or something equivalent), check for trailing whitespace (the default hook does exactly this), or check for appropriate documentation on new methods.</blockquote><p>A sample pre-commit hook for running Pylint has been shared <a href="https://fitzgeraldnick.com/2009/12/03/git-hooks-and-pylint.html">here</a> by <a href="https://fitzgeraldnick.com/">Nick Fitzgerald</a>. I recommend modifying this sample script slightly by implementing Python Argument <a href="https://docs.python.org/3/library/argparse.html">Parser</a> to enable dynamic value fetching from user instead of hard coding it within the code. For ex: you can set a default pass rank for the pylint checks while allowing the user to pass a value which would override the defaults. It also helps to choose the pylintrc file from the user if the need be.</p><p>So far, we saw the gates that help us in implementing and governing the lint checks, however these 2 steps are totally dependent on a user/developer as it becomes their sole responsibility to implement the quality filters.</p><p>Hence, the 3rd gate. This gate forms the final gate which strictly enforces the checks and cannot be bypassed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nhsWO_ymyrdcUN7bEjTGhQ.png" /></figure><p>3: The integration of pylint checks with the CI/CD pipeline forms the bulletproof shield against quality compromises. Run the linter within your CI pipeline (set to trigger by a Pull Request) and pass/fail the Pipeline based on the results of the lint checks, thereby prohibiting the succession of your code from lower to higher branches (feature -&gt; dev, dev -&gt; master, etc).</p><p>For my demonstration, I am using <a href="https://codefresh.io/">CodeFresh</a> to orchestrate the CI pipeline. In the below example, I am building a Python package and publishing it to a artifactory manager, <a href="https://jfrog.com/">JFROG</a>. This pipeline is triggered whenever there is a Pull Request raised from feature branch to dev branch and the results are sent back to Git using a service hook registered between Codefresh and Git.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kNRLzmdMFL41D2onz49DzA.png" /></figure><p>Hence, the reviewer can refer the automated checks before proceeding with the actions to be taken on the Pull Request. You can also set the build of CI pipeline to be triggered upon the merging of a PR, totally depends on you!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/858/1*xM4eJRHF8bLboM3dMCTZGA.png" /></figure><h4><strong>What is Pylintrc?</strong></h4><p>You would have noticed that I mentioned “pylintrc” a few times so far. This is the rule book that I mentioned at the beginning of this blog. It’s a config file for the pylint that states the various checks to be performed/skipped explicitly by deviating from generic PEP-8.</p><p>For ex: PEP-8, states that the max <a href="https://www.python.org/dev/peps/pep-0008/#maximum-line-length">length</a> of a line to be limited to 79 chars:</p><blockquote>Limit all lines to a maximum of 79 characters. For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters.</blockquote><p>However, most of the teams/projects consider this unnecessary and are fine with limiting the line length to 100/120 characters. This override must be specified in the pylintrc file as:</p><pre>[FORMAT]<br>max-line-length=100</pre><p>Similarly, if you want to skip checking for Module doc-strings along with the above specification, the pylintrc would look as:</p><pre>[FORMAT]<br>max-line-length=100</pre><pre>[MASTER]<br>disable=<br>    C0114, # missing-module-docstring</pre><p>You can generate a default rc file by using the pylint flag --generate-rcfile and set the overrides as required.</p><pre>pylint --generate-rcfile &gt; .pylintrc</pre><p>Alternatively, there are a lot of predefined pylintrc file available as open source. Feel free to refer any of those and tweak them according to your use case.</p><p>Invoking Pylint with a specific pylintrc file should be done as below:</p><pre>pylint --rcfile &lt;PATH_TO_RC_FILE&gt; &lt;PATH_TO_PY_MODULE/FILE&gt;</pre><blockquote><em>One of my favorite quotes on computer programming:</em></blockquote><blockquote>Any fool can write code that a computer can understand. Good programmers write code that humans can understand.</blockquote><blockquote>— Martin Fowler, 2008.</blockquote><p>I hope the above blog post helped you in getting started with code linters. I strongly feel that the best way to improve one’s coding habits is to read through the comments raised by the editors as you start developing and try to rectify those then and there. As you start following pylint messages, you would also learn better ways of implementing the same functionality. It definitely helps!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*0FUoRqvksZJN82lU.jpg" /></figure><p>Coding with better standards and improved quality comes with practice and patience. The toughest task is to get started. Once you leave the shore and start sailing, trust the wind and it will propel you smoothly towards the zenith.</p><p>As I always believe, start with one step at a time!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8b38faab0ae" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/the-gates-of-code-quality-pylint-b8b38faab0ae">The Gates of Code Quality — Pylint</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sys Tables in Apache Spark/Databricks]]></title>
            <link>https://medium.com/swlh/sys-tables-in-apache-spark-databricks-b30d3993b7bd?source=rss-f07f96ca111d------2</link>
            <guid isPermaLink="false">https://medium.com/p/b30d3993b7bd</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[tricks]]></category>
            <category><![CDATA[metadata]]></category>
            <category><![CDATA[databricks]]></category>
            <dc:creator><![CDATA[Ankit Choudhary]]></dc:creator>
            <pubDate>Sun, 09 Aug 2020 05:00:15 GMT</pubDate>
            <atom:updated>2020-08-09T05:00:15.577Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/884/0*-p3S1OOsZd7w_w7e.png" /></figure><p><a href="https://medium.com/u/5ae67e7eecef">Databricks</a> does an amazing job of orchestrating Apache Spark. The Databricks Delta Table is a revolutionary invention and brings a set of new possibilities to the Big Data Ecosystem.</p><p>While companies are shifting to Databricks as their new Data Warehouse, there’s one feature of the conventional Database systems that I wished Databricks had. It’s the availability of a dynamic system database (sys db) which stores the metadata about the larger set of spark tables holistically.</p><p>The sys tables (sys.tables, sys.columns, etc.) comes in handy when you want to scan and search your database based on the metadata. One such use can be the search for tables/columns containing <a href="https://www.investopedia.com/terms/p/personally-identifiable-information-pii.asp">PII</a>. Wouldn’t it be great if you can just search for all the tables/columns having the keyword “email/phone/etc”. With the fast moving Data Privacy Acts like <a href="https://gdpr-info.eu/">GDPR</a> &amp; <a href="https://en.wikipedia.org/wiki/California_Consumer_Privacy_Act">CCPA</a>, these things can save a lot of time.</p><p>So, how do we get this sys db created in Databricks? Well, you have landed at the right place. Just follow the instructions below and start exploring.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*ahrvK5tgfI6GsWgp5IpCUA.png" /></figure><h4><a href="https://github.com/ankitkchoudhary/databricks-meta-data-tables/blob/master/databricks-metadata-tables.py">Here</a>’s the Entire Code on: <a href="https://github.com/ankitkchoudhary/databricks-meta-data-tables/blob/master/databricks-metadata-tables.py">Github</a></h4><p>Once the code snippet/notebook has been processed, it stores the results in the below tables:-</p><ul><li><strong>sys.db_tables : </strong>Contains the list of Databases , their respective Tables and the Object’s fully qualified names.</li><li><strong>sys.db_table_columns : </strong>Contains the table vs column mappings along with the data types and comments (if any).</li><li><strong>sys.db_table_properties : </strong>Contains other crucial table properties information like InputFormat, OutputFormat, Storage Properties, numFiles, sizeInBytes, etc.</li><li><strong>sys.db_table_partitions : </strong>Contains the Table vs Partition Columns Mapping.</li></ul><p>Here’s a snapshot from a Test Notebook that I created to show the results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*hJDxzpIf5lBzVLnqc7La2g.png" /><figcaption>Actual Table and Database names have been wiped off for security</figcaption></figure><p>It is recommended that you schedule this notebook as a job within your Databricks workspace and run it a desired frequency/interval (daily/weekly/your_choice), so that the latest state of your workspace is captured and stored into these meta-data tables.</p><p>I wish Databricks incorporates this as one of their packaged offerings in the future releases. Until then, I hope this blog could help you solve some of your use-cases.</p><p>Happy Coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b30d3993b7bd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/sys-tables-in-apache-spark-databricks-b30d3993b7bd">Sys Tables in Apache Spark/Databricks</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tips on Flying Safe during COVID-19]]></title>
            <link>https://ankitblogs.medium.com/tips-on-flying-safe-during-covid-19-d682e088c335?source=rss-f07f96ca111d------2</link>
            <guid isPermaLink="false">https://medium.com/p/d682e088c335</guid>
            <category><![CDATA[air-travel]]></category>
            <category><![CDATA[airports]]></category>
            <category><![CDATA[safe]]></category>
            <category><![CDATA[usa]]></category>
            <category><![CDATA[covid19]]></category>
            <dc:creator><![CDATA[Ankit Choudhary]]></dc:creator>
            <pubDate>Sat, 04 Jul 2020 04:53:21 GMT</pubDate>
            <atom:updated>2020-07-04T19:38:29.101Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*h1sT1HqZcjpYgJ2YlnlwOw.jpeg" /></figure><p><strong>COVID-19</strong> has hit us really bad. As of writing this piece, its been close to 5 months where the daily life has come to a pause. Offices have shifted to the Work From Home Culture and people have been adapting themselves to the new norms of the lifestyle during the pandemic. On the contrary, certain industries still need to operate in person and certain things cannot be virtual at all. I had to fly from Dallas, TX (DFW) to New York City(LGA) for unavoidable reasons and this blog shares a few tips, tricks and experiences that I learned and discovered which might help you to make your travel safe.</p><h3>Buckle Up!</h3><p><strong>Fly Economy : </strong>Have a flexible schedule and at least a 2 days window for your travel. Once your travel date window is decided, shop around for the flight tickets. For my trip, I observed that almost all the American Airlines flights had their First Class Seats filled up, whereas the Economy class was 60–65% occupied. Now, that gives a very good chance of maintaining social distancing. The odds are in your favor that of the 3 seats in a row, you will find at least 1 empty seat as compared to those First Class seats. I was allotted the seat 20D with no one else in 20E and 20F (yes, an empty row), thereby giving me ample space and distance from other passengers. Thus, flying First Class isn’t always necessarily better.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kXcQbWVWp3pT9kryYH0P3Q.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XYf_E5T5WpSwY8SMkfJc7g.jpeg" /><figcaption>A complete row to myself. Only possible in Economy Class.</figcaption></figure><h3>What to Wear?</h3><p><strong>Masks : </strong>I bought a KN95 mask and a regular surgical mask. I had decided to keep my face covered at all the times until I reached my destination.</p><figure><img alt="I would sometimes get paranoid and wear the Surgical one on top of KN95" src="https://cdn-images-1.medium.com/max/768/1*N1Zj37qdWOnz9jMQk2aaCw.jpeg" /></figure><p>Why the double mask? Well, here’s the plan. KN95s are very strong masks and it’s cumbersome to wear them all the time (Hats off to the Healthcare workers who stay suited up all the time, top to bottom). I had the KN95 at all the crowded places and would switch to the surgical ones as soon as I get a socially distant feeling.</p><p>And on being paranoid, I would wear the Surgical one on top of the KN95 :)</p><p>Also, I chose to discard all that I will be wearing during my travel. So, I chose a pair of clothes that I had planned to get rid of from a long time. As soon as I reached my destination, all the clothes went into a trash bag and I went straight for the warm shower.</p><p><strong>Gloves: </strong>Stock up on these precious and noble inventions made by mankind. I had dual layer of gloves on me. I knew I would come in contact with a lot of things. And the plan was to discard the top layer before drinking water or taking a bite of snack. Safety first! Once done refueling, I would put on a fresh pair of gloves as the 2nd layer again. A few of you might be thinking that it’s overdoing, but the very fact that we are on this blog says that we all are concerned about the safety and it’s better safe than to be sorry. So, read on!</p><figure><img alt="Dual Gloves for Easy Travel" src="https://cdn-images-1.medium.com/max/900/1*e1BGhWJAD-jY3y4bYzhmxw.jpeg" /></figure><h3>Checked In Bags</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/692/1*r-uXd7_SzJjFhXNjb-baDQ.jpeg" /></figure><p>I had to check in my bags and thus I decided to have my bags wrapped in the plastic. I got the roll of plastic wrap from Home Depot for $12. I did this so that, I can get keep my stuff safe from any slightest chances of contamination. As soon as I collected my bags from the baggage claim carousel, I took off the wrap and trashed it away.</p><p>Luckily it also helped me keep my things safe and dry as it was raining heavily while we landed. And I could see that most of the bags had got slightly drenched (may be while transporting the bags from the aircraft to the baggage loading section).</p><h3>Get a Water Bottle</h3><p>Make sure to have your own personal water bottle. You can carry an empty bottle to the airport and refill it past the security check. It would come in handy as you won’t have to reach out to any vending machines or shops that might be visited by a lot more travelers. Also, helps to not ring the Call for Assistance Bell in-flight.</p><h3>In-Flight Services</h3><p>Airlines are doing their best to protect their customers. I flew with <a href="https://medium.com/u/9e3bb64945e6">American Airlines</a> and I was awed by their seriousness in handling the COVID situation. They made regular announcements in the airport and in-flight to ensure that every traveler had their face mask on at all the times.</p><p><a href="https://medium.com/u/9e3bb64945e6">American Airlines</a> have also stopped their Onboard catering and have instead started handing over a basic kit to the passengers while on-boarding the aircraft. The kit usually contains a Bottle of Water, a snack and a Hand Sanitizer Sachet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*lqRIsklloEiIeE1wfYGEnw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*JXdzpB6-Jnd7IMbb8DzF1A.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*nYPGARXOHX60HctnB-mYWg.jpeg" /><figcaption>American Airlines — Kit</figcaption></figure><h3>Hitting the Loo?</h3><p>Let’s talk the obvious. A 4-hours flight would most probably make you visit the restroom at least once. Well, no special instructions here. Just make sure you wipe the Toilet Seat Before and After Use without fail. Other than that, if possible, close the lid and flush it before use (and of course after use).</p><h3>Do I need a PPE Gown?</h3><figure><img alt="I was thinking of doing the same" src="https://cdn-images-1.medium.com/max/600/1*qYcu13veDNtoAfHzLwZdHQ.jpeg" /></figure><p><strong>No</strong>, <strong>you don’t!</strong> I have been asking this question to myself until I began my journey. I had also planned on buying one from Amazon for my travel. But, the fact is that you don’t really need one. Airports are scarcely populated these days and I felt very comfortable as soon as I stepped inside. The Security Check Queues were literally empty and so were the Baggage Drop Counters.</p><p>However, please ensure that you are having the basic safety measures like a face shield/mask and hand gloves. DO NOT compromise on these as they are the most important ones that would keep you shielded.</p><p>All that I mentioned above are simple things that everyone knows about. Sometimes, stating the obvious becomes necessary. Be safe, be healthy!</p><p>Let’s do our best in the battle against COVID-19. Travel only if necessary, stay indoors and have patience.</p><p>I hope that the Virus goes away very soon and life gets back to normal. This reminds me of my cute little niece Ishi. She says that the virus will be gone by the time her birthday arrives and she will have a blast! I hope her dream comes true. She’s turning 5 this October…</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d682e088c335" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fastest way to get your Texas Driver’s License — 3 days]]></title>
            <link>https://ankitblogs.medium.com/fastest-way-to-get-your-texas-drivers-license-3-days-2122523ab99?source=rss-f07f96ca111d------2</link>
            <guid isPermaLink="false">https://medium.com/p/2122523ab99</guid>
            <category><![CDATA[indian]]></category>
            <category><![CDATA[texas]]></category>
            <category><![CDATA[license]]></category>
            <category><![CDATA[hacks]]></category>
            <category><![CDATA[fast]]></category>
            <dc:creator><![CDATA[Ankit Choudhary]]></dc:creator>
            <pubDate>Wed, 03 Jun 2020 19:32:04 GMT</pubDate>
            <atom:updated>2020-06-03T19:40:44.876Z</atom:updated>
            <content:encoded><![CDATA[<h3>Fastest way to get your Texas Driver’s License — 3 days</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9yhvUpHDPFcT5P_-rI0w9w.png" /></figure><p>I moved to Dallas, Texas in the month of June 2018 (from India). And the first thing, that I needed for anything and everything was a photo ID document. Without any form of Photo ID, life comes to a pause. I also needed a Driver’s License for mobility as the public transport in Dallas and especially Irving isn’t broad as other cities. I remember using <a href="https://www.dart.org/">DART</a> to commute to my office once and it took 55 minutes as DART has it’s own route, whereas my office is just 6 mins drive from my apartment. When I started researching about the process of getting the license, I was told that it takes somewhere from 1.5 months to 3 months, just to get an appointment at the DMV.</p><blockquote>And yet, I managed to start the process on Tuesday and I had my license on Thursday. Here’s how:</blockquote><p><strong>1Documentation</strong> → (Estimated Time Needed — 0.5 day) The law states that you must present proofs for various sections. You can find the details <a href="https://www.dps.texas.gov/internetforms/Forms/DL-15.pdf">here</a> and <a href="https://www.dps.texas.gov/internetforms/Forms/DL-57.pdf">there</a>. I will keep this post specific and to the minimum requirements (after all we are focussed on the fastest way get behind the wheel).</p><p>For non-immigrants like me (on H1B or any other Visa), you must present the below :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YpOHg2eE8_CGsKChVYvJwQ.png" /></figure><p><strong>Application for the Texas Driver’s License:<br></strong>Fill and Print this <a href="https://www.dps.texas.gov/internetforms/Forms/DL-14A.pdf">from</a> with due diligence.</p><p><strong>Proof (1) of Lawful Presence in USA &amp; Proofs (2) of Residency in Texas:<br></strong>1. I-94<br>2. Visa (H1B, I-20) Approval<br>3. Get a Bank Statement (Savings/Checking/Credit Card).<br>4. Paycheck/Pay Stub.<br>(for #3 and #4 — It must have your residence address on it — (Proof of Residency in Texas.)</p><p><strong>Proof of Identity:<br></strong>Your foreign passport (Passport from the Country of your Citizenship)</p><p><strong>Social Security Card: <br></strong>Your SSN Card</p><p><strong>Vehicle Registration and Insurance:<br></strong>Assuming that you don’t own one (How could you, without a License? Right?). Make sure to Answer “<strong>NO</strong>” to the section titled “<strong>VEHICLE REGISTRATION AND INSURANCE INFORMATION” </strong>while filling the application.<br><em>(If you own a vehicle, please submit the registration document, Simple!)</em></p><p><strong>Course Completion Certificate (ITTD — Impact Texas Drivers):<br></strong>You need to present the ITTD Course Completion Certificate. More about this below.</p><p><strong>2Impact Texas Drivers (ITD) Program</strong>: (Estimated Time → 1–3 hours). Texas Law mandates that you <strong>must</strong> complete the Awareness Video applicable to you (per your age). I highly recommend paying undivided attention to the video as Texas witnesses a large number of crashes/road accidents due to distracted driving. You can complete the video <a href="https://www.dps.texas.gov/DriverLicense/ITD.htm">here</a>.</p><p>Once you complete the awareness video, you will be assigned a completion <strong>certificate</strong> which is valid for 90 days. (We just need 3 days validity 😂 ).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*omk42T1YE2JaRfp41CRUqA.png" /></figure><p><strong>3Online Test</strong>: (Prepartaion Time → 1 day) You must appear the online test that will be taken at the DPS center.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*mbBRw_4ViRPBfJezhSK4Zw.jpeg" /></figure><p>Please thoroughly read the Driver’s Handbook published by the department <a href="https://www.dps.texas.gov/internetforms/Forms/DL-7.pdf">here</a>. Again, I <strong>insist</strong> you to read through the book. I learnt a lof of things that I had earlier taken for granted. Once you are done reading, you might want to look at a few pratice questions that I have shared on my <a href="https://drive.google.com/drive/folders/1FX7ailOxwhqF1TT3rcZSjJY0n2x5VHQS?usp=sharing">Google Drive</a>.</p><p><strong>4Appointment</strong>: You will need to secure an appointment at the DPS office twice. Once for the Learners Driver’s License and then for the Actual Driver’s License. There’s a hack in securing an appointment which I learnt the hard way. Here’s the secret:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/399/1*TogD5l8D2McsQZEpvO-NqQ.jpeg" /></figure><p>The appointment portal refreshes daily at <strong>7 AM CT</strong> for the current day. Wake up before 7 and login to the <a href="https://www.dps.texas.gov/administration/driver_licensing_control/rolodex/getinlineonline.htm">Get-In-Line portal</a>. Secure the appointment for the earliest slot (get ahead of the traffic and the crowd).</p><p>Make sure that you carry all the documents listed above along with the Application Fee. Once you are the DPS center, wait for your turn, submit the documentation and the fees. The officer will collect your biometrics and perform a vision test. You will be then allotted a Computer to appear for the Knowledge Test. I am relying on you to pass the test. All the best!</p><p>Once done, you will be called back and issued a Learner’s License.</p><p><strong>5Behind the Wheel Test</strong>: This is where my blog is centered around. Securing an appointment for the Behind-the-Wheel test takes months. Texas DMV has a flexibility where you can appear the test at a private center (aka 3rd Party). I researched for a few <a href="https://www.dps.texas.gov/DriverLicense/tpstSchools.htm">Certified 3rd Party Driving Schools</a> where you can appear for the Test. Based on my research you can enroll for the test, cheapest at <strong>$55</strong>.</p><p>I would recommend taking an hour of car diving lesson (even though you are a pro) from the school where your test is scheduled. They usually are able to accommodate the lessons and tests back-to-back (Bonus Tip: Search for Schools around the Irving/Carrolton area if you are in DFW metroplex). The driving lesson helps as a mock test for the actual behind-the-wheel test. Once your (1-2 hour) lessons is done, let the school know that you are ready to appear for the Behind-the-Wheel test. That’s it! Appear for the test and clear it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*pj1nEiEho5CtZS5MkHbVVw.jpeg" /></figure><p>Note: They are very serious about the test and allow no negligence. You will be mostly appearing for your test with the same instructor you took lessons from. But that doesn’t mean that the instructor will allot grace points towards your mistake. <strong>Rules are Rules!</strong></p><p>After you clear the test, the school will enclose the test results (signed and stamped) in a sealed envelope. You will need to submit this sealed envelope(test results) to the DMV to convert your Learners License into a Driver’s License. <strong>DONOT OPEN THE ENVELOPE AT ANY TIME BY YOURSELF.</strong></p><p>6 <strong>The Day</strong>: This is the day you are going to be thankful to me forever. This is the day, you will be licensed to drive. As I mentioned earlier, wake up early, secure an appointment.</p><blockquote><strong>Tip:</strong> Carry <strong>all</strong> the documents as before (including the application), even though you might be tempted to skip them as you already submitted it just 2 days back. There isn’t a feeling worst than having to return back or get to the end of the queue just for some missing documents.</blockquote><p>If you have followed my instructions clearly, I can assure you that you will be securing your license at the fastest way possible.</p><h3>My Timeline — Beat this Speed!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eytHzxdJi8GUczsaoYXNug.jpeg" /></figure><p><strong>Tuesday</strong> — Arranged all the Required Documents, Watched the ITTD Video, Practiced the Sample Questions.</p><p><strong>Wednesday Morning </strong>— Secured my Learner’s Licence</p><p><strong>Wednesday Evening</strong> — Cleared Behind-the-Wheel Test</p><p><strong>Thursday Morning</strong> — Secured my Driver’s License</p><p>You would know from my bio that am a motorcycle enthusiast. Soon after I got my drivers license, I applied for the motorcycle license.</p><p>I will try to write another blog on the easiest way to score a motorcycle license.</p><blockquote>Some beautiful paths can’t be discovered without getting lost. <br>― <strong>Erol Ozan</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2122523ab99" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Electricity Usage Dashboard]]></title>
            <link>https://medium.com/swlh/electricity-usage-dashboard-7b0b31f866d1?source=rss-f07f96ca111d------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b0b31f866d1</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[texas]]></category>
            <category><![CDATA[dashboard]]></category>
            <category><![CDATA[electricity]]></category>
            <category><![CDATA[smart-meter]]></category>
            <dc:creator><![CDATA[Ankit Choudhary]]></dc:creator>
            <pubDate>Mon, 11 May 2020 18:45:27 GMT</pubDate>
            <atom:updated>2020-05-15T23:12:01.716Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to reduce your electricity bill → Real Time Usage Dashboard → Python, Raspberry PI , Azure, Docker, Streamlit— Available on Github, Docker Hub</h3><figure><img alt="Snapshot of the Reporting Dashboard" src="https://cdn-images-1.medium.com/max/1024/1*uw8WZOh_wDEuUvPb98ENhw.jpeg" /><figcaption>Snapshot of the Reporting Dashboard</figcaption></figure><p>Project: A utility to continuously monitor the electricity usage of a consumer and plot the various usage patterns on a Dashboard. The entire application has been built on <a href="https://www.python.org/">Python</a> and is hosted on a <a href="https://www.raspberrypi.org/">Raspberry Pi</a>. I have made the code base available on <a href="https://github.com/ankitkchoudhary/electricity-usage-monitoring">Github</a> and the builds can be found on <a href="https://hub.docker.com/u/ankitkchoudhary">Docker-hub</a>.</p><p>The below information is currently displayed on the Dashboard:<br>→Meter Info (Address of Property, ID, etc)<br>→Usage in the Current Cycle.<br>→Time Series for the Usage across the past 24 Hours.<br>→Last Billed Reading.<br>→Time Series for the Usage across the past 45 Days.<br>→Billing Patterns for the past 12 months.<br>→Time Series (with 15 minute Interval) for an entire day (T-2).</p><p><strong>Cost to build</strong>: It took me close to 10 days time for researching the usage, sourcing the data provisioning API and building the Dashboard. I also procured a pre-owned raspberry-pi-4b from <a href="https://www.microcenter.com/">MicroCenter</a> for $40. You can get a brand new one from <a href="https://www.microcenter.com/product/609038/raspberry-pi-4-model-b---4gb-ddr4">here</a> for $55.And finally a monitor. I had an old Dell Monitor lying around and that came in handy. <em>Optionally</em>, you can create a free Azure Cloud Subscription.</p><p><strong>Inspiration: (</strong><em>Getting the best deal on electricity by controlling usage.)</em><br>am on an electricity plan (<strong>Maxx Saver 12) </strong>from <a href="http://4changeenergy.com/">4changeenergy</a> where I get a <strong>bill credit of $90</strong>, if my usage crosses the 1000kWh benchmark.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TKKngrx2y0y8BGxgY0upHw.png" /><figcaption>Electricity Charges — Initial Billing Cycle</figcaption></figure><p>When, I initially started this plan, I ended up using <strong>853kWh</strong> for the first billing cycle and was charged an amount of <strong>$139.</strong></p><p>The usage was lower than I expected because my flatmate was traveling and I was staying alone. I ended up losing the <strong>$90</strong> credit :(</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v-BZO_CVHufbZDHnCWno8g.png" /><figcaption>Electricity Charges — Subsequent Billing Cycle</figcaption></figure><p>In the subsequent month, my flatmate was back and this time we ensured that we are not sacrificing on the comfort of our lives and partied harder than ever before. Unfortunately, we ended up consuming (1343kWh) way beyond our benchmark. We did get the $90 credit, however there was a guilt feeling of overusing the resources.</p><p>Now, that’s when I wanted to build a utility that keeps me posted on the usage status and thereby letting me consume wisely and enabling me with actionable insights.</p><p>And that’s when I learnt about the <a href="http://smartmetertexas.com/">Smart Meter Texas</a> portal. It’s a site that stores daily, monthly and 15-minute interval energy data recorded by digital electric meters (commonly known as “smart meters”), and provides secure access to that data to customers and authorized market participants.</p><p>I have been exploring various ways of extracting the data from the portal and could learn about its internals by scanning through the Developer Mode of my Safari browser. I was able to figure out the details of the calls being made to the back-end server, the various APIs the request limits and much more.</p><p>The details of the API that am polling for my project here can be found <a href="https://github.com/ankitkchoudhary/electricity-usage-monitoring/blob/master/api_endpoints.ini">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B5e1LPSXcusANQTCXNuKtA.png" /><figcaption>A Quick Look at the Smart Meter Texas Portal</figcaption></figure><p><strong>Challenges</strong>: Following the developers mantra, I started with googling around to see for any previous works already done in this field. I did come across a few repositories working with smart-meter-texas, however all of them were <a href="https://en.wikipedia.org/wiki/Web_scraping">web-scrapping</a> projects.</p><p>I am not a big fan of the web-scrapping way of acquiring data points as they are not reliable, scalable and are prone to break with any new UI changes made by the host. I spent a lot of hours in the <a href="https://support.apple.com/guide/safari/use-the-developer-tools-in-the-develop-menu-sfri20948/mac">Developer mode</a> of my safari browser. I was able to trace the authentication calls and subsequently all calls made by the Website to it’s backend servers.</p><p><strong>Challenge 1:</strong> Requests to API were simply ignored as they weren&#39;t made from the browser and from the smartmeter portal.</p><p><strong>Solution 1: </strong>I had to mock the request headers to replicate the browser as below:</p><p><em>{ <br>‘</em><strong><em>User-Agent</em></strong><em>’: ‘Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML,’ ‘ like Gecko) Chrome/77.0.3865.90 Safari/537.36’,<br>‘</em><strong><em>Origin</em></strong><em>’: ‘https://www.smartmetertexas.com</em>’<em>, <br>}</em></p><p><strong>Challenge 2</strong>: I was trying to make similar POST calls to the APIs using the same set of parameters (headers + body + endpoint) using python’s requests module, however I was unable to authenticate myself. This was due to the fact that the website uses cookies to handle its flow. I tried disabling the browser’s permissions for cookies and the smartmetertexas.com wouldn’t load.</p><p><strong>Solution:</strong> I had to make use of Requests → Sessions Module to handle the cookies. Here, I make an initial call to the portal to fetch the cookies making it believe that the incoming request is from a web browser. I would then set the same cookies while making calls to the Authentication API. These steps can be clearly seen <a href="https://github.com/ankitkchoudhary/electricity-usage-monitoring/blob/master/meter_session_manager.py#L33">here</a> on the github repo.</p><p>After solving the authentication challenges, I was able to make calls to various endpoints of the API to fetch the different data points. That covers the data ingestion part. Let’s move on to the visualizations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dYWTh6sfrwCH8Eo94AF9Aw.png" /><figcaption>Overview of the Setup</figcaption></figure><p><strong>Dashboard</strong>: Building the dashboard was another major challenge. Reasons being primarily as my lack of Front End Skills. I tried doing a use case on Tableau Desktop/Public, Power BI, etc. and couldn&#39;t opt for any of these as all of them are tagged with a pricing factor that would no longer make this project budget free.</p><p>The choice of dashboard had to be pretentious as I wanted this utility to be a light weight tool. We have aimed to run this application on a single board computer. Loading a heavy software on the <a href="http://raspbian.org/">raspbian OS</a> would slowdown the system and heat up the motherboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7zYXbvsWyjv2XZnZmokcmA.png" /></figure><p>To the rescue came the purely python based, fast and easy to use framework called <a href="https://www.streamlit.io/">Streamlit</a>.</p><p>The data ingestion framework stores all the data files for the trends in CSV format and that makes it very easy to form Pandas Dataframe on it.</p><p>Streamlit provides a very fast and efficient way of plotting the charts and visualizations. The initial dashboard was up and running in the first few hours of the dashboard development.</p><p>Further, Streamlit open a port on the computer to the network which it is connected to and shared a link accessible by all other devices on the network. Thus, this dashboard is now accessible across mobile devices and computing devices of all my flatmates. Mobility is definitely a cherry on the cake.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zq-jCJT3t6Y8IBP0V2AYyQ.png" /><figcaption>Screenshot of the Dashboard from my iPhone’s browser</figcaption></figure><p>Here’s how the setup at my home looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kHu2s3X69WK2xqAm-NTGVg.jpeg" /><figcaption>Used a spare monitor, removed the stands, turned the screen to Portrait mode</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F_SE1v3LQA6S4DhYpRVkKw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tFXCtCsbUCnKGKtfVcv-3A.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HLYMtFyE3eeq1_TXYIrxMQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NoB5BLAuWs6Ihn-NlhwzBA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RI-eS3qlAmM9m-ZIQxP0lQ.jpeg" /><figcaption>Assembling the Raspberry PI 4. Fascinating!</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3IC5X4qie8KUGv4RVcuEjg.jpeg" /><figcaption>The Final Product. A fully loaded computing platform.</figcaption></figure><p>Here’s a video of the live dashboard.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FruB6oIoASDw%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DruB6oIoASDw&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FruB6oIoASDw%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/da1f2fc6553afa14c0daea016de3b6e8/href">https://medium.com/media/da1f2fc6553afa14c0daea016de3b6e8/href</a></iframe><p><strong>Risks</strong>: One and a Major one! I am relying on the smart meter APIs. Now that am making my code base publicly available, if smart meters decides to change the policies and makes changes to their access methods, this project will be a cripple. I have made sure that the application is built ethically and only looks for the concerned user’s data and respects the API limits imposed. Rest is upto the providers to decide. 🤞</p><p><strong>Enhancements</strong>: A lot can be added to this project. A few major things that I am planning to do constitutes of adding prediction analysis to leverage the available data and compute the expected billing amount.</p><p>We can also incorporate the offers from various energy providers participating in smart meters and help a consumer choose the best plan suited for his/her household. A lot can be done!</p><p><strong>Stay Tuned:</strong> I will be publishing further blogs in silos sharing the setup of Docker, Raspberry Pi, Azure Apps, etc. If there’s anything that you want me to write specifically on, please drop a note. ✌️</p><blockquote>“The best way to cheer yourself is to try to cheer someone else up.”— <a href="https://en.wikipedia.org/wiki/Mark_Twain">Mark Twain</a></blockquote><p><em>Hope you liked my first blog on </em><strong><em>medium.com</em></strong><em>, please let me know your thoughts. Got a question? Leave a comment.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b0b31f866d1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/electricity-usage-dashboard-7b0b31f866d1">Electricity Usage Dashboard</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>