<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Swapnil Kant on Medium]]></title>
        <description><![CDATA[Stories by Swapnil Kant on Medium]]></description>
        <link>https://medium.com/@swapnilkant11?source=rss-6d10c8780163------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*O7DDMchHkfxOqxeG63_Juw.jpeg</url>
            <title>Stories by Swapnil Kant on Medium</title>
            <link>https://medium.com/@swapnilkant11?source=rss-6d10c8780163------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 15:20:40 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@swapnilkant11/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[OpenSearch With Rails]]></title>
            <link>https://medium.com/javarevisited/my-experience-using-opensearch-with-rails-6b29044d131d?source=rss-6d10c8780163------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b29044d131d</guid>
            <category><![CDATA[elasticsearch]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Swapnil Kant]]></dc:creator>
            <pubDate>Mon, 24 Jun 2024 14:02:35 GMT</pubDate>
            <atom:updated>2024-06-25T15:18:22.835Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Kb6M1M9ZQJHGRmmetujiQw.png" /></figure><h4>Introduction</h4><p>Hello folks, while working with <strong>OpenSearch</strong> I learned about it’s architecture and how it works on the backend for managing so many nodes, indexes and indexing documents to provide the most fastest and accurate search results. While I was working on my project where I had to integrate <strong>OpenSearch </strong>using<strong> Rails </strong>I worked on some interesting code configurations to setup <strong>OpenSearch </strong>on my <strong>Rails </strong>application. In this article I would be sharing my experience while using <strong>OpenSearch </strong>with <strong>Rails.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*SpICb7WXF8RPUwmbfjRMMQ.png" /><figcaption>OpenSearch</figcaption></figure><h4>What is <strong>OpenSearch</strong>?</h4><p>The first question which come to your mind after hearing OpenSearch<strong> </strong>is</p><blockquote><strong>“Is OpenSearch a database?”</strong></blockquote><p>well the answer to this question is that OpenSearch<strong> </strong>can be loosely defined as a database but to be more specific OpenSearch<strong> </strong>is a distributed search engine.</p><blockquote><strong>OpenSearch </strong>is a <strong>distributed</strong> search engine which enables you to search almost anything and everything and analyse your data once you ingest your data to <strong>OpenSearch. </strong>The term <strong>distributed </strong>means that it is capable of running into multiple nodes.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*O5Vsy8zm1YgvztWk-TTWlA.png" /><figcaption>History of OpenSearch</figcaption></figure><h4>History of OpenSearch</h4><p>Well, talking about the history of OpenSearch.</p><p>In 2015, Amazon took an advantage of the open-source licence to launch Amazon ElasticSearch Service, a cloud based managed service which would allow the users to launch scalable ElasticSearch services, connect the data sources to cluster endpoints, and load, process, analyse or visualise data in cloud.</p><p>But the developers at NV would eventually object to Amazon’s use of their products and trademarks and filed a suit against the tech giant Amazon later Amazon resolved this suit on two terms</p><ol><li>In January 2021, Elastic N.V. announced that, ElasticSearch would be <a href="https://www.elastic.co/blog/licensing-change">licensed</a> under the Server Side Public license (SSPL) which prevents Amazon and other organisations from providing ElasticSearch as a service without directly collaborating with Elastic.</li><li>In April 2021 Amazon announced that it will be forking the open-source version of ElasticSearch to launch a new open-source distributed search engine which later named as <strong>OpenSearch.</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/743/1*jD-cDiSk8KYYZNfKLcJZXQ.png" /><figcaption>OpenSearch Components</figcaption></figure><h4>OpenSearch Components</h4><p><strong>OpenSearch </strong>has various components which work together to make it speed an cost efficient, Let’s discuss some of it’s main components</p><ol><li><strong>Documents</strong> in OpenSearch<strong> </strong>is defined as the units which are responsible for storing data information, talking about how it stores the data information is in the <strong>JSON </strong>format. Each <strong>document</strong> refers to a single data which comprises of fields and values. In easy words <strong>documents</strong> in OpenSearch<strong> </strong>can be easily referenced with the <strong>documents </strong>present in MongoDB database in terms of it’s structure.</li><li><strong>Indices </strong>in OpenSearch<strong> </strong>is defined as a collection of documents. In easy words <strong>indices </strong>can be understood as a database which comprises of a collection of documents (similar to rows in a table).</li><li>OpenSearch<strong> </strong>is a distributed search engine and from the term <strong>distributed </strong>it means that it is capable of running in multiple nodes. The <strong>Clusters </strong>in OpenSearch is defined as a collection of these nodes which makes OpenSearch a distributed search engine which carries out search and analysis operations on your data.</li><li>Clusters in OpenSearch comprises of a <strong>Cluster Manager Node </strong>which is responsible for managing indices in OpenSearch. It is responsible for <strong>deleting</strong> and <strong>creating</strong> indices which contains all your data information. The <strong>Cluster Manager Node </strong>inside a cluster is responsible for managing the documents within these indices which together combine to form shards present in a node and node is responsible for combining all the data information from multiple nodes and sending it as a combined response to fulfil your request.</li><li><strong>Shards </strong>in OpenSearch is defined as a combination of indices. OpenSearch splits indices into multiple <strong>shards.</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/563/1*gh1ItaMT6bXgQmd3AK42Wg.png" /><figcaption>Cluster In OpneSearch</figcaption></figure><h4>How does Sharding helps?</h4><p>Now, suppose you have 800gbs of index in your cluster in OpenSearch, now while making a request to OpenSearch it is not feasible to make a request to a single node as it decreases the system’s performance and decreases the scalability of the system and makes it very slow resulting in delay response, but dividing these indices into various shards say each of 80gbs will make the system efficient and faster, and hence each shards contribute to the scalability and maintainability of the system.</p><p>Now, when you have multiple shards which has multiple indices within a cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/676/1*D-XfiMiI7hOir3oO-BBGgA.png" /><figcaption>OpenSearch</figcaption></figure><h4>Why OpenSearch?</h4><p>OpenSearch is famous for it’s two main features</p><ol><li><strong>Scalability </strong>is defined as a system’s capability to increase or decrease the system’s efficiency or performance in response to change in application and system’s processing demands.<br>Now, in the previous case where we came across the concept of sharding where we divided a 800gb chunk of data into multiple 80gbs smaller indices chunks into multiple shards which in return increased the system’s performance and hence, made the system scalable and also minimised the cost which would have been more if it was left to be computed from a single node.</li><li>AWS<strong> </strong>allows<strong> </strong>it’s users to pay for only the amount of services they are using be it a specific amount of storage the number of EC2 instances running and the amount of data transfer taking place, it has a <strong>pay as you go</strong> plan which is much cost efficient than other similar service providers and enables a user to manage their <strong>cost and usage.</strong></li><li>AWS services comes up with <strong>IAM (Identity and Access Management) </strong>service which allows the admin to setup certain rules and permissions to handle the AWS services. It might have a group which consists of various users within it having certain rules and permissions assigned or might be a single user contributing to strong <strong>Security.</strong></li></ol><h4><strong>Setting Up OpenSearch</strong></h4><p>For setting up <strong>OpenSearch</strong> you can follow the link <a href="https://docs.aws.amazon.com/opensearch-service/latest/developerguide/gsg.html">here</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/307/1*eVuq5yaDHpyQayRK4gai_A.png" /><figcaption>AWS and Rails</figcaption></figure><h4><strong>Configuration and Connection</strong></h4><p>For testing my <strong>OpenSearch</strong> with rails I have created a basic flight app which is a basic CRUD app on rails and I have tried to show how to perform basic CRUD operations with <strong>OpenSearch.</strong></p><p>Use the gems to integrate <strong>OpenSearch </strong>service in your application</p><pre># Opensearch client for ruby<br>gem &#39;opensearch-ruby&#39;<br>gem &#39;opensearch-aws-sigv4&#39;</pre><p>For connecting OpenSearch to your rails application you can refer to the code which I have used in my application (refer to method <strong>open_search_client</strong>)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b06b80d02eeb489ee15ca1c6baec055f/href">https://medium.com/media/b06b80d02eeb489ee15ca1c6baec055f/href</a></iframe><p>followed by your environment variables in .env file as</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/15989f652b73d72e2c58bd6d2507aac3/href">https://medium.com/media/15989f652b73d72e2c58bd6d2507aac3/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KRV5gTzRLdoUVf69iCeFNQ.png" /><figcaption>OpenSearch Vs ElasticSearch</figcaption></figure><h4>OpenSearch Vs ElasticSearch</h4><ul><li><strong>High Performance</strong>: Elasticsearch engine is 40–140% faster than OpenSearch while consuming fewer compute resources.</li><li><strong>Pricing</strong>: Elastic Cloud pricing starts at $95+/month for a Standard subscription, while AWS customers can start using OpenSearch Service for free if they remain under the AWS Free Tier usage limits.</li><li><strong>Community Support</strong>: GitHub reveals that the Elasticsearch codebase has had a greater number of commits over the past year compared to OpenSearch.</li></ul><p><strong>Keep learning and keep growing and also keep exploring more!</strong></p><p><strong>All the very best!</strong></p><p>For more exciting and informative articles and tips follow me on <a href="https://swapnilkant11.medium.com/"><strong>Medium</strong></a><strong> and </strong><a href="https://www.linkedin.com/in/swapnil-kant-279a3b148/"><strong>Linkedin</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b29044d131d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/javarevisited/my-experience-using-opensearch-with-rails-6b29044d131d">OpenSearch With Rails</a> was originally published in <a href="https://medium.com/javarevisited">Javarevisited</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Atlas-Powered MySQL Migration Planning For GORM In Go]]></title>
            <link>https://javascript.plainenglish.io/atlas-powered-mysql-migration-planning-for-gorm-in-go-bc935d438a93?source=rss-6d10c8780163------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc935d438a93</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[migration]]></category>
            <dc:creator><![CDATA[Swapnil Kant]]></dc:creator>
            <pubDate>Fri, 07 Jun 2024 17:35:30 GMT</pubDate>
            <atom:updated>2024-06-08T13:45:40.160Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rRXZd6pU3_rfri2hQiWEgQ.png" /></figure><h3>Introduction</h3><p>Hello, folks! While working on migrations in <strong>Golang</strong> using <strong>MySQL</strong>, I came across various online resources to figure out how to write migrations that could be modified or upgraded over time according to product needs. That was when I realized that no resources explained how to plan an <strong>Atlas</strong>-powered <strong>MySQL</strong> migration with the help of <strong>GORM</strong> in <strong>Golang</strong>. In this blog, I will explain how to migrate your <strong>MySQL</strong> database using <strong>Atlas</strong> and <strong>GORM</strong> in <strong>Golang</strong>.</p><p><strong>So let’s…..</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/1*awA9o_5gRyiKcQN3jSPiUw.png" /></figure><h3><strong>What is GORM?</strong></h3><blockquote><strong>GORM</strong> is an <strong>ORM</strong> <strong>library</strong> for the <strong>Go</strong> community that provides a high-level abstraction, automating the transfer of data stored in relational database tables into objects commonly used in <strong>Golang</strong>. It allows a developer to manage their database schemas using its <strong>AutoMigrate</strong> feature, which is usually sufficient during development and in many simple cases. You can read more about <strong>GORM </strong><a href="https://gorm.io/docs/index.html">here</a>.</blockquote><h3>What is AutoMigrate?</h3><p>In the previous section, we encountered the term <strong>AutoMigrate</strong>. Now, let’s briefly understand what it means.</p><blockquote><strong>AutoMigrate</strong> automatically updates your schema by creating tables and adding missing foreign keys, constraints, columns, and indexes. It will modify an existing column’s type if there are changes in its size, precision, or nullability. However, it will NOT delete unused columns to safeguard your data. Additionally, it automatically creates database foreign key constraints.</blockquote><p>Sounds cool… right?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/820/1*aQVMwGrCr6P4Dg7zoUk55w.png" /></figure><p>but wait…. it also has certain limitations.</p><h3><strong>Challenges with AutoMigrate</strong></h3><p>Using <strong>GORM’S</strong> <strong>AutoMigrate </strong>function can be very convenient for quickly setting up and modifying your database schema based on your Go struct models. However, there are several challenges and potential pitfalls associated with relying solely on <strong>AutoMigrate </strong>for schema management. Here are some of the key challenges:</p><ol><li><strong>Limited Control Over Schema Changes: </strong>It does not generate down migrations (i.e., steps to reverse changes). This makes it hard to roll back schema changes if something goes wrong.</li><li><strong>Schema Drift: </strong>Over time, differences can arise between the actual database schema and the desired state as defined by your models, especially in complex projects. This drift can lead to unexpected issues and difficulties in maintaining the schema.</li><li><strong>Lack of Versioning: </strong>It does not keep a history of migrations. Without versioned migrations, it is difficult to track changes over time or understand the evolution of the database schema.</li><li><strong>Performance Issues: </strong>Creating and managing indexes through <strong>AutoMigrate</strong> can be inefficient, especially for large tables with existing data.</li><li><strong>Complex Schema Changes: </strong>Complex schema changes often require custom SQL scripts, which are not possible with <strong>AutoMigrate</strong>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*HIBwIY03upfW-SHwNfBQ4w.jpeg" /></figure><p>so….what’s the best alternative for <strong>MYSQL</strong> migrations in <strong>Golang</strong>? well here comes the use of <strong>Atlas.</strong></p><h3>What is Atlas?</h3><p><strong>Atlas</strong> is a language-independent tool for managing and migrating database schemas using modern DevOps principles.</p><blockquote>Atlas can automatically generate database schema migrations for developers using the official <a href="https://github.com/ariga/atlas-provider-gorm">GORM Provider</a>. It does this by comparing the current state of the database with its desired state to determine the necessary changes.</blockquote><h4>Installation</h4><p>Install Atlas from macOS or Linux by running:</p><pre>curl -sSf https://atlasgo.sh | sh</pre><p>Install the provider by running:</p><pre>go get -u ariga.io/atlas-provider-gorm</pre><p>The Atlas GORM Provider can operate in two modes:</p><p>1. <strong>Standalone</strong>: If all your <strong>GORM</strong> models are within a single package and either embed gorm.Model or include <strong>GORM</strong> struct tags, you can directly use the provider to load your <strong>GORM</strong> schema into Atlas.</p><p>2. <strong>Go Program</strong>: If your <strong>GORM</strong> models are distributed across multiple packages or do not embed gorm.Model or include <strong>GORM</strong> struct tags, you can use the provider as a library within your Go program to load your <strong>GORM</strong> schema into <strong>Atlas</strong>.</p><p>you can read more about them <a href="https://atlasgo.io/guides/orms/gorm">here</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*parBwZr-jOkN-ZmBxHE-Ug.jpeg" /></figure><h3>Example Workflow</h3><p>In your project directory, create a new file named atlas.hcl with the following contents</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5a34a84a7deb1c0a6fbe3a78bb67e1e9/href">https://medium.com/media/5a34a84a7deb1c0a6fbe3a78bb67e1e9/href</a></iframe><p>Next, we will have to write our schema rules which will help us define our user and book <strong>model</strong> and the relationship between them also you can define your <strong>indexes </strong>here, go ahead and create a directory in your project root folder <strong>schema </strong>named schema.sql</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/aa712ef0c1bf042fffe441510dfa313b/href">https://medium.com/media/aa712ef0c1bf042fffe441510dfa313b/href</a></iframe><p>now, your project structure should look something like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/254/1*klQYKDC-lEROt8aAqcQPbA.png" /></figure><p>you can see now that we don’t have any migration files inside our <strong>migration directory, </strong>this folder will soon have migration files as soon as we run migrations.</p><p>Now, let us run our migrations using the following command</p><pre>atlas migrate hash --dir file://migrations</pre><p>The above command will create a hash value, which uniquely represents the content of all migration files in the <strong>migrations directory.</strong></p><p>Next, run the command to generate the differences (diff) between the current database schema and the desired schema</p><pre>atlas migrate diff --env gorm --dir file://migrations --to file://schema --dev-url mysql://username:password@localhost:3306/database?multiStatements=true</pre><p><strong>Note: here it is very important to note that if you use the MySQL driver with an existing database client, you must create the client with the parameter </strong>multiStatements=true.</p><p>The purpose of this command is to:</p><ol><li><strong>Calculate the Difference</strong>: Compare the current state of the database (accessed via --dev-url) with the desired state of the schema (specified in --to).</li><li><strong>Generate Migrations</strong>: Create migration files in the --dir directory that will transform the current database schema into the desired schema.</li><li><strong>Environment Configuration</strong>: Use settings defined in the gorm environment to manage the connection and other parameters for the database.</li></ol><p>after running the above command you will see a <strong>migration file </strong>generated along with a hash file which ensures the integrity and consistency of your database schema changes throughout your development and deployment process inside the <strong>migrations directory.</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/091849dc40496e63d8d523f0285f053a/href">https://medium.com/media/091849dc40496e63d8d523f0285f053a/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3528c6b9df9e2c8d5a42af91f17b6a81/href">https://medium.com/media/3528c6b9df9e2c8d5a42af91f17b6a81/href</a></iframe><p>Now, it’s time to apply the migrations to our <strong>MYSQL</strong> database using the following command</p><pre>atlas migrate apply --dir file://migrations --url mysql://username:password@localhost:3306/database?multiStatements=true</pre><p>The purpose of this command is to apply all pending migration scripts located in the specified directory to the specified database. This updates the database schema to match the desired state as defined by the migration files.</p><p>You will see the following output in your terminal</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/878/1*tNINzDcaGvIqyWk0oDjBnA.png" /><figcaption>Output</figcaption></figure><p>Your final project directory structure should look like</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/255/1*z6HBsVtAhR7Lp5ONvxg98Q.png" /></figure><p>Now, you can see your tables generated in your database along with a table named atlas_schema_revisions which has the information about the last migration applied to the database</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bwWiNFHRGtsSt2SFtudZ1Q.png" /><figcaption>atlas_schema_revisions</figcaption></figure><p>This is how we plan our migrations in <strong>Golang </strong>using <strong>GORM</strong> and<strong> Atlas. </strong>If you want to read more about Atlas you can refer to the documentation <a href="https://atlasgo.io/docs">here</a></p><p><strong>Keep learning and keep growing and also keep exploring more!</strong></p><p><strong>All the very best!</strong></p><p>For more exciting and informative articles and tips follow me on <a href="https://swapnilkant11.medium.com/"><strong>Medium</strong></a><strong> and </strong><a href="https://www.linkedin.com/in/swapnil-kant-279a3b148/"><strong>Linkedin</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc935d438a93" width="1" height="1" alt=""><hr><p><a href="https://javascript.plainenglish.io/atlas-powered-mysql-migration-planning-for-gorm-in-go-bc935d438a93">Atlas-Powered MySQL Migration Planning For GORM In Go</a> was originally published in <a href="https://javascript.plainenglish.io/">JavaScript in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Export Data In CSV Using Rails and Angular.js]]></title>
            <link>https://javascript.plainenglish.io/export-data-from-webpage-in-csv-using-an-api-dd89ead814a1?source=rss-6d10c8780163------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd89ead814a1</guid>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[rails]]></category>
            <category><![CDATA[angular]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[Swapnil Kant]]></dc:creator>
            <pubDate>Sat, 18 Mar 2023 10:17:03 GMT</pubDate>
            <atom:updated>2023-03-22T06:39:34.794Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*DD-2yza9mZFuuKCktSXuOw.jpeg" /></figure><p>Hello folks! While looking for proper documentation for exporting all data from a webpage designed in angular.js in CSV using an <strong>API</strong> in rails, I figured out there are a lot of incomplete resources and a need for proper code structure and optimizations! So, after searching for many resources on the internet, I have figured out a way to execute this feature, some of you might be aware of this implementation, so <strong>I have tried to extend this article by also explaining how to optimize the code when we have millions of data to be exported!</strong> So, let’s start by discussing the sub-features we would be implementing, and let’s do it individually!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/259/1*UISGsl-ydlN3N4P4KveURQ.jpeg" /></figure><h4>Sub-Tasks Involved</h4><p>To design an <strong>API</strong> that would export the data in a CSV file we are already aware that for our use case, we need mainly two things, the first one being an <strong>API</strong> and the second being a button (on the client-side) clicking which would download the data in a CSV file.</p><p>Now, for the above two tasks we will break them into smaller tasks which would be as follows:<br>1. <strong>Create a route</strong> that would integrate your client-side and backend applications and would enable the export of your data to CSV format.<br>2. <strong>Create a method</strong> that would enable a user to download the data from the webpage with just a click! This will include all your logic to fetch the data from the database, depending on you how you need the data :)<br>3. <strong>Create a method in your model </strong>that would help set up all your headers and data on the webpage to your CSV file.<br>4. <strong>Integrate our API </strong>on the client side to fetch the data through an <strong>export </strong>button<br>5. <strong>Create a method on the client side </strong>that would receive your data from the <strong>API</strong> we integrated into CSV format and would arrange them into the file according to the logic written here!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*FekA-7-jNFfepQ6cK2bIjQ.jpeg" /></figure><p>Now, after discussing the sub-tasks involved let’s look into the code and see how it works on every line!</p><h3><strong>Creating your backend API</strong></h3><p>While creating your backend <strong>API</strong> and fetching the data from your database, I would suggest you always use <strong>eager loading </strong>instead of <strong>lazy loading </strong>to optimize your query if you are fetching data from multiple databases and have associations involved (you can read more about <strong>eager and lazy loading </strong><a href="https://rubyastra.wordpress.com/2021/02/09/eager-load-vs-lazy-load/">here</a>) because when we download some data we expect to get all data at once and there is no pagination involved. Here let’s take an example you are trying to fetch the records of all the bookings made by the user to CSV as shown below</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b25148dcfaaa45a084b9046e3760f7a2/href">https://medium.com/media/b25148dcfaaa45a084b9046e3760f7a2/href</a></iframe><p>here you can see I have included the bookings, which could only be possible if you include associations within your user and booking model, as user has_many<strong> </strong>bookings and bookings belongs_to<strong> </strong>user, so every time we run this <strong>API</strong> there are a total of <strong>2 queries </strong>running hence, a time complexity of O(1) which is a constant! also, make <strong>a note</strong> here we are using respond_to<strong> </strong>to format our data into CSV so, basically respond_to is a method of the superclass ActionController which takes format as an argument and serializes your data to CSV format!</p><p>So, now we already have our method created so, let’s create a route that would trigger this method! to get all data we would create a <strong>get </strong>route as shown below</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/909d0cdd2dca5d73a622ba7bf930c114/href">https://medium.com/media/909d0cdd2dca5d73a622ba7bf930c114/href</a></iframe><p>Now, we will be creating a method in our <strong>user</strong> model named as_csv which will import the headers of our CSV file and our users data and their bookings here column_names is the name of the columns in the users table of your database as shown below</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d94877ef1debe35f6c2e851771605864/href">https://medium.com/media/d94877ef1debe35f6c2e851771605864/href</a></iframe><p>Now, we are done with creating our <strong>API</strong> on the <strong>backend </strong>so, now, we will test our <strong>API</strong> on Postman to make sure your <strong>API</strong> would be in the format <strong>.csv </strong>for example my <strong>API</strong> would be as shown below</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6e73b6af3acc540556bc97158d1ce2c5/href">https://medium.com/media/6e73b6af3acc540556bc97158d1ce2c5/href</a></iframe><p>Now, when you hit this on your Postman you will see the response in comma separated or the CSV format, as you needed!</p><p>Awesome work guys! you just made your <strong>API</strong> which would send you the data in CSV format.</p><h3>Creating Views</h3><p>Now, we will integrate this <strong>API</strong> and develop functionality to download a CSV file on the go! for that we will have to create a button on our webpage to click on which we will get all our data exported to a CSV file!</p><p>Creating a button <strong>Export </strong>to fetch all our data to a CSV file as shown below</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e335c1b9f155889595d01e2bdd46e794/href">https://medium.com/media/e335c1b9f155889595d01e2bdd46e794/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2e14c6fa6c2059dd4580d0b976fcc96f/href">https://medium.com/media/2e14c6fa6c2059dd4580d0b976fcc96f/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ed4272063fcc522fb8c3d9b9c1d87ca7/href">https://medium.com/media/ed4272063fcc522fb8c3d9b9c1d87ca7/href</a></iframe><p>Here we have all our client-side functions to respond to the button click <strong>Export </strong>now which fetches the data from the <strong>API </strong>and returns to the <strong>ExportCsv </strong>function so, we will need to build a function <strong>ExportCsv </strong>as written on line 4 of the above code to download the CSV file with our data, so let’s create as shown below</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4639397027ca69409ac52cc35b817af8/href">https://medium.com/media/4639397027ca69409ac52cc35b817af8/href</a></iframe><p>Now, let’s understand the above code here we push all our CSV data from the API to an array data and while we traverse through the array we keep appending the data to csv variable and then create a file using the <strong>Blob </strong>object which is a file-like object of immutable, raw data; they can be read as text or binary data, or converted into a <a href="https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream">ReadableStream</a> so its methods can be used for processing the data which in return creates an URL to download the data in the filename that we mentioned which triggers this method on line 4 of the <strong>UserController.js</strong> file.</p><p>Hence, we have made our own <strong>API</strong> to export all our data to a CSV file, however, you can add your own modifications according to your data requirement, this was just an example of how you can start thinking and come up with an idea to design your own <strong>API.</strong></p><p>Now, let us discuss our second problem statement which tasks with how to optimize the code when we have millions of data to be exported! So, here’s one of the ways of doing so</p><h3>Use Sidekiq as a Background Job Processing</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gD-KanpUcqM-nlaEOHGjpw.png" /></figure><p>If you have been programming in Ruby for quite some time you must have come across this name as <strong>Sidekiq </strong>yes!</p><p><strong>Sidekiq</strong> is Simple, efficient background processing for Ruby. <strong>Sidekiq</strong> uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails to make background processing dead simple.</p><p>Now, imagine you are using this API which you just designed to export millions and billions of user data so if you start hitting this API to fetch data it would take a lot of your time to get the data or even the <strong>TTL (Time To Live) </strong>might exceed the time of your API response and hence, it would fail. So now your question might be how to solve this problem, well we can solve this by putting all your data fetching logic inside a <strong>Sidekiq</strong> worker because it has the capability to concurrency by using multiple threads in its process and then send the response data to the <strong>client side.</strong></p><p>Apart from that, if you have less <strong>RAM</strong> space as compared to the data you want to export, you can break them into chunks of data by using <strong>batch processing </strong>in rails<strong> </strong>picking up the data in <strong>batches</strong> and processing them, you can read more about <strong>batch processing</strong> in rails <a href="https://apidock.com/rails/ActiveRecord/Batches/find_in_batches">here</a></p><p><strong>Keep learning and keep growing and also keep exploring more!</strong></p><p><strong>All the very best!</strong></p><p>For more exciting and informative articles and tips follow me on <a href="https://swapnilkant11.medium.com/"><strong>Medium</strong></a><strong> and </strong><a href="https://www.linkedin.com/in/swapnil-kant-279a3b148/"><strong>Linkedin</strong></a></p><p><em>More content at </em><a href="https://plainenglish.io/"><strong><em>PlainEnglish.io</em></strong></a><em>. Sign up for our </em><a href="http://newsletter.plainenglish.io/"><strong><em>free weekly newsletter</em></strong></a><em>. Join our </em><a href="https://discord.gg/GtDtUAvyhW"><strong><em>Discord</em></strong></a><em> community and follow us on </em><a href="https://twitter.com/inPlainEngHQ"><strong><em>Twitter</em></strong></a>, <a href="https://www.linkedin.com/company/inplainenglish/"><strong><em>LinkedIn</em></strong></a>,<em> and</em><strong><em> </em></strong><a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong><em>YouTube</em></strong></a><strong><em>.</em></strong></p><p><strong><em>Learn how to build awareness and adoption for your startup with </em></strong><a href="https://circuit.ooo/?utm=publication-post-cta"><strong><em>Circuit</em></strong></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd89ead814a1" width="1" height="1" alt=""><hr><p><a href="https://javascript.plainenglish.io/export-data-from-webpage-in-csv-using-an-api-dd89ead814a1">Export Data In CSV Using Rails and Angular.js</a> was originally published in <a href="https://javascript.plainenglish.io/">JavaScript in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cron To Go Review — A Heroku Scheduler Alternative]]></title>
            <link>https://medium.com/coderbyte/cron-to-go-review-a-heroku-scheduler-alternative-85c3a5318584?source=rss-6d10c8780163------2</link>
            <guid isPermaLink="false">https://medium.com/p/85c3a5318584</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[developer]]></category>
            <category><![CDATA[addons]]></category>
            <category><![CDATA[heroku]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Swapnil Kant]]></dc:creator>
            <pubDate>Wed, 06 Apr 2022 18:00:09 GMT</pubDate>
            <atom:updated>2022-04-12T01:25:59.379Z</atom:updated>
            <content:encoded><![CDATA[<h3>Cron To Go Review — A Heroku Scheduler Alternative</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*m37FDmj9RqASWwbT.png" /><figcaption>Cron To Go Scheduler Dashboard</figcaption></figure><p>We all at some point in our programming journey have come across <strong>Heroku-</strong> a cloud-native platform for developing applications in various languages. It offers us many inbuilt features in one platform such as Github, Docker Integrations, Command-Line, Web UI, and various other <a href="https://elements.heroku.com/addons">add-ons</a> to choose from. <strong>Heroku</strong> is one of the most popular platforms for free hosting of websites and offers a free scheduling add-on of its own for running jobs on our apps at scheduled time intervals, much like Cron in a traditional server environment.</p><p>Some of the major problems with <strong>Heroku Scheduler</strong></p><blockquote>The scheduler has no guarantee that jobs will execute at their scheduled time, or at all, there may be a chance that a job will be skipped, and there may be a chance that a job will run twice.</blockquote><p>One of the good alternatives to this is a <a href="https://devcenter.heroku.com/articles/scheduled-jobs-custom-clock-processes"><strong>custom clock process </strong></a>on <strong>Heroku</strong> and in this article, I am going to talk about <a href="https://crontogo.com/">Cron To Go Scheduler</a> an add-on created by <a href="https://crazyantlabs.com/">Crazy Ant Labs</a> its features, setup, and integration with Nodejs code.</p><h4>What are Crons?</h4><p>Crons are the command-line utilities also known as cron jobs, they are the schedulers that are used by the developers to schedule jobs that are responsible to run on a fixed time, date, or periodically in some intervals. It typically automates system maintenance or administration. It is most suitable for scheduling repetitive tasks.</p><p>Before we talk about what exactly Cron To Go Scheduler is, let&#39;s figure out what Heroku Dynos are.</p><blockquote><strong>Dynos </strong>are isolated, virtualized Linux containers that are designed to execute code based on a user-specified command. They are the building blocks that power any Heroku app, from simple to sophisticated.</blockquote><h4>What is Cron To Go Scheduler?</h4><p>Cron To Go Scheduler is an add-on created for Heroku users to run scheduled jobs based on one-off dynos using cron expressions. It enables the dynos to run only when it is needed and it combines the simplicity of using cron to schedule jobs with the scale and elasticity of the cloud.</p><h4>Setting up the Environment</h4><p>Setting up your Cron To Go Scheduler is quite easy and has a fully managed environment with flexible scheduling, let us see how to configure and setup the environment, there are two ways to add Cron To Go Scheduler to your application which is:</p><h4>Adding Cron To Go Scheduler using Heroku UI</h4><ol><li>Go to the Heroku Dashboard and select any one of the existing projects, if there is no existing project, create one</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g7ifHHek-zCBltslxMy1JA.png" /><figcaption>Heroku Dashboard- Image by author</figcaption></figure><p>2. After selecting the project go to the <strong>installed add-ons </strong>section and click on <strong>Configure Add-ons</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/1*V_aZfMyj77qcJP3n0Upv6Q.png" /><figcaption>Heroku Overview Dashboard- Image by author</figcaption></figure><p>3. Now, click on <strong>Find more add-ons</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-GUgNX2rP7Cd6hVW46j6BA.png" /><figcaption>Heroku Resources Dashboard- Image by author</figcaption></figure><p>4. Now, select <strong>Cron To Go Scheduler </strong>from the available add-ons from the <strong>Data Stores </strong>section</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m-DKCbYrwV2EkScn0xCdOg.png" /><figcaption>Heroku add-ons Datastores- Image by author</figcaption></figure><p>5. Next, click on <strong>install </strong>Cron To Go Scheduler on the rightmost top corner of the window</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*slhvyNcTFayeFRA3MnItEw.png" /><figcaption>Cron To Go Scheduler add-on- Image by author</figcaption></figure><p>6. Now, select any <strong>add-on plan </strong>and select the application in the search bar with which the add-on has to be connected and click on <strong>Submit Order Form</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NLRSiea3WM6hc2e4md-Yvw.png" /><figcaption>Cron To Go Scheduler Order Form- Image by author</figcaption></figure><p>7. Next, wait for a few seconds to get the add-on started, and now, you can add your jobs and manage them to get started! after deploying the add-on you will be directed to your dashboard which looks something like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jptcuRfIcdQNBSuPPtvSJg.png" /><figcaption>Cron To Go Scheduler Dashboard- Image by author</figcaption></figure><h4>Adding Cron To Go Scheduler using Heroku CLI</h4><ol><li>Login to your <strong>Heroku </strong>account using the command and enter the credentials such as email and the password on the terminal</li></ol><pre>$ heroku login -i</pre><p>2. Now, use the command to add the Cron To Go Scheduler add-on to your <strong>Heroku </strong>dashboard</p><pre>$ heroku addons:create crontogo -a APP_NAME</pre><p>Once ready you can access the Cron To Go Scheduler Dashboard directly from Heroku</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4v1r_eJk5ukEs2KFebKvLg.png" /><figcaption>Cron To Go Scheduler Dashboard- Image by author</figcaption></figure><p>In your Cron To Go Scheduler dashboard, you can find all your information related to the jobs created, last run, webhooks, and other settings in the platform.</p><p>You can also access the dashboard using the command</p><pre>$ heroku addons:open crontogo -a APP_NAME</pre><p>and then, visit the dashboard using the web</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gp2SKGvXtANnVyqwDQ6sZw.png" /><figcaption>Command to access the Cron To Go dashboard- Image by author</figcaption></figure><h4>How to Create Jobs?</h4><p>Jobs in Cron To Go Scheduler can be created using the <strong>Cron To go dashboard</strong> or the <strong>APIs</strong></p><p>You will have to provide the name of your job which you are creating as a <strong>Nickname </strong>and add the time interval when you want your job to be executed, enter the script in the <strong>Command </strong>which you want to run then select the <strong>Dyno Size </strong>according to your needs and usage in your application and <strong>Timeout </strong>and then create your job by clicking on the button <strong>click.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zEIceUAkDLaMnlN6ovwiYw.png" /><figcaption>Job creation window on Cron To Go Scheduler- image by author</figcaption></figure><p>While you are selecting <strong>Dyno Size </strong>you might come across different types of dynos that are, <strong>free </strong>having a size of 512MB, <strong>hobby </strong>having a size of 512MB, <strong>standard-1x </strong>having a size of 512MB, <strong>standard-2x </strong>having a size of 1024MB, <strong>performance-m </strong>having a size of 2.5GB, <strong>performance-I </strong>having a size of 14GB, it depends on which plan are you currently working or having your application working upon!</p><h4>How can one Monitor and check the Logs of the application?</h4><p>Monitoring and checking the logs for your application can be easily done from within the UI (by clicking the menu button […] for the job and then choosing view job history or view job logs). You can also use the CLI:</p><pre>$ heroku logs -t -a &lt;APP_NAME&gt; -d crontogo</pre><p>then, you can use the dashboard to view the streamed dyno log from <strong>Heroku </strong>or you can also install logging add-ons.</p><h4>Application Features</h4><p>Cron To Go can run any command that can be run in your application. For Node.js, you can simply add a file to your app and execute it using the command</p><pre>$ node &lt;SCRIPT_NAME&gt; &lt;METHOD_NAME&gt;</pre><h4>Removing the Add-on</h4><p>The add-on can be removed using the following command</p><pre>$ heroku addons:destroy crontogo</pre><h4>What are Webhooks?</h4><p>Webhooks enable us to receive notifications whenever particular events occur with our jobs. You can subscribe to notifications regarding the following events:</p><ul><li>Job Execution Failed — occurs whenever a job execution fails to complete.</li><li>Job Execution Started — occurs whenever a job execution is started.</li><li>Job Execution Succeeded — occurs whenever a job execution completes successfully.</li></ul><p>Webhook notifications are sent as HTTPS POST requests to a URL of your choosing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TbDOKsRvyuwO6WGKNeDUOw.png" /><figcaption>Webhook in Cron To Go Scheduler dashboard- Image by author</figcaption></figure><p>for creating a <strong>webhook </strong>in Cron To Go Scheduler you can refer to the image above where you need to add the <strong>Nickname </strong>as a descriptive name for the webhook.<strong>, Endpoint URL </strong>is the server endpoint to send all your webhook notifications to<strong>, Authorization</strong> <strong>Header</strong> is a header that will be included with all webhook notifications and selecting the topics for which the notifications have to be sent.</p><p>After the <strong>webhook </strong>has been created you will be getting a secret key that is displayed only once, <strong>so you are advised to copy this key and store it in a safe file. </strong>Each request is signed by Cron To Go in the X-Hub-Signature header.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nNFtCGD0r9RQn13wyxe-HA.png" /><figcaption>Secret key while creating a Webhook- image by author</figcaption></figure><p>After successfully creating a webhook, you may <strong>pause/resume, rotate secret, ping webhook, </strong>and <strong>view deliveries.</strong></p><h4>Receiving Webhooks</h4><p>While we get a POST request to your server endpoint with the details of the event. You can verify this by matching the authorization header with the value you passed when subscribing to notifications and also by checking the HMAC SHA256 signature of the request body.</p><h4>Why do we need this?</h4><p>Cron To Go Scheduler comes up with various features which are very unique and reliable some of them are:</p><ul><li><strong>One-Click</strong> <strong>Migration</strong>: It provides easy import of jobs from <strong>Heroku Scheduler, Advanced Scheduler,</strong> and <strong>Temporizes Scheduler</strong> for a smooth migration.</li><li><strong>Flexible</strong> <strong>Scheduling</strong>: It supports the Unix cron format so that you can define schedules <strong>with a 60 seconds precision </strong>allowing you to run your jobs run <strong>multiple times a day</strong>, on <strong>specific days and times of the week or of the month</strong>.</li><li><strong>No Code Platform: No code</strong> changes are required to add, edit or remove a scheduled job.</li><li><strong>Different Time-Zones Feature</strong>: You can <strong>schedule</strong> your jobs in different time zones.</li><li><strong>Notifications and Webhooks: </strong>You can receive <strong>notifications</strong> directly to your <strong>email</strong> or via <strong>webhooks</strong>.</li></ul><h4>Summary</h4><p>Now, talking about the summary of this article we have discussed various features and functionalities of Cron To Go Scheduler from talking what it means to discuss about how to create jobs, to scheduling the jobs and managing them, and also how to destroy the jobs and also its features, this is one of the best product created by <strong>Crazy Ant Labs </strong>which helps you <strong>manage all your automated jobs at one place, helps in scaling up your platform easily according to the needs </strong>and <strong>allow you to monitor and get the quick and correct logs of your jobs at one place.</strong></p><h4>Conclusions</h4><p>Cron To Go Scheduler is one of the best and most fully-managed job schedulers for your Heroku applications which is flexible, scalable, and easy to get started with I am really impressed with the product on its features of <strong>fully</strong> <strong>managed, scalability, notifications, and webhook features, security, flexible scheduling, monitoring, and logging features, etc </strong>features which are not present in other job scheduling add-ons.</p><p><strong>Keep learning and keep growing and also keep exploring more!</strong> <strong>All the very best!</strong></p><p>For more interesting and informative articles and tips follow me on <a href="https://swapnilkant11.medium.com/"><strong>Medium</strong></a><strong> </strong>and <a href="https://www.linkedin.com/in/swapnil-kant-279a3b148/"><strong>Linkedin</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=85c3a5318584" width="1" height="1" alt=""><hr><p><a href="https://medium.com/coderbyte/cron-to-go-review-a-heroku-scheduler-alternative-85c3a5318584">Cron To Go Review — A Heroku Scheduler Alternative</a> was originally published in <a href="https://medium.com/coderbyte">Tech x Talent</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[SFTP To Go Add-On By Crazy Ant Labs On Heroku]]></title>
            <link>https://medium.com/javarevisited/sftp-to-go-add-on-by-crazy-ant-labs-on-heroku-b178410c6480?source=rss-6d10c8780163------2</link>
            <guid isPermaLink="false">https://medium.com/p/b178410c6480</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[cloud-storage]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[heroku]]></category>
            <category><![CDATA[startup]]></category>
            <dc:creator><![CDATA[Swapnil Kant]]></dc:creator>
            <pubDate>Thu, 24 Mar 2022 03:55:58 GMT</pubDate>
            <atom:updated>2022-03-28T14:58:38.953Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iuA1ubp-6Ttd4vFWYksLdg.png" /><figcaption>SFTP To Go Dashboard</figcaption></figure><p>We all at some point in our programming journey have come across <strong>Heroku</strong> a cloud-native platform for developing applications in various languages. It offers us many inbuilt features in one platform such as Github, Docker Integrations, Command-Line, Web UI, and various other <a href="https://elements.heroku.com/addons">add-ons</a> to choose from. <strong>Heroku </strong>has been one of the most loved places for free hosting of websites and is said to be a <strong>polyglot platform</strong> as it has features for a developer to build, run and scale applications in a similar manner across most languages.</p><p>One of the major problems that <strong>Heroku </strong>has is the storage of files</p><blockquote><strong>Heroku </strong>has an “ephemeral” file system, where although we can write files to our disk, there is no guarantee that those files will persist after the application is restarted.</blockquote><p>Now, in rescue to this, although <strong>Heroku </strong>has several <a href="https://elements.heroku.com/addons#data-stores">storage add-ons</a>, in this article, I am going to talk about <a href="https://sftptogo.com/">SFTP To Go</a> add-on created by <a href="https://crazyantlabs.com/">Crazy Ant Labs</a> its features, setup, and integration with Nodejs code.</p><h3>What is SFTP To Go?</h3><p><a href="https://sftptogo.com/">SFTP To Go</a> is secure, scalable, and durable cloud storage that is primarily used to secure file access, transfer, and management either automatically or manually without the need to maintain and handle APIs.</p><h3>Setting up your Environment</h3><p>Setting up your SFTP To Go is quite easy and also follows its principle of zero management, let us see how to configure and setup our environment, there are basically two ways to add SFTP To Go to your application which are:</p><h4>Adding SFTP To Go using Heroku UI</h4><ol><li>Go to the Heroku Dashboard and select any one of the existing projects, if there is no existing project, create one</li></ol><figure><a href="https://javarevisited.blogspot.com/2018/02/10-courses-to-prepare-for-programming-job-interviews.html"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g7ifHHek-zCBltslxMy1JA.png" /></a><figcaption>Heroku Dashboard</figcaption></figure><p>2. After selecting the project go to the <strong>installed add-ons </strong>section and click on <strong>Configure Add-ons</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/1*V_aZfMyj77qcJP3n0Upv6Q.png" /><figcaption>Heroku Overview Dashboard- Image by author</figcaption></figure><p>3. Now, click on <strong>Find more add-ons</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-GUgNX2rP7Cd6hVW46j6BA.png" /><figcaption>Heroku Resources Dashboard- Image by author</figcaption></figure><p>4. Now, select SFTP To Go<strong> </strong>from the available add-ons from the <strong>Data Stores </strong>section</p><figure><a href="https://javarevisited.blogspot.com/2020/08/10-examples-of-essential-linux-commands.html"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4sCp3-0-SHjmSZJ9dZL8Rw.png" /></a><figcaption>Heroku add-ons Datastores- Image by author</figcaption></figure><p>5. Next, click on <strong>install </strong>SFTP To Go<strong> </strong>on the rightmost top corner of the window</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2kT3zO4EpyxDp9MVpL8FFQ.png" /><figcaption>SFTP To Go add-on- Image by author</figcaption></figure><p>6. Now, select any <strong>add-on plan </strong>and select the application in the search bar with which the add-on has to be connected and click on <strong>Submit Order Form</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J7mATWPI0l4dYgtChN34ng.png" /><figcaption>SFTP To Go Order Form- Image by author</figcaption></figure><p>7. Next, wait for a few seconds to get the add-on started and now, we can copy our <strong>URI </strong>and <strong>passcode </strong>to get started!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iuA1ubp-6Ttd4vFWYksLdg.png" /><figcaption>SFTP To Go Dashboard- Image by author</figcaption></figure><h4>Adding SFTP To Go using Heroku CLI</h4><ol><li>Login to your <strong>Heroku </strong>account using the command and enter the credentials such as email and the password on the terminal</li></ol><pre>heroku login -i</pre><p>2. Now, use the command to add the SFTP To Go<strong> </strong>add-on to your <strong>Heroku </strong>dashboard</p><pre>heroku addons:create sftptogo:bronze -a APP_NAME</pre><p>in the above command, <strong>bronze </strong>is the name of the plan given, the other plans which are available are M<strong>icro, Mini, Small, Starter, Professional, and Premium.</strong></p><p>Once ready you can access the SFTP To Go Dashboard directly from Heroku</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iuA1ubp-6Ttd4vFWYksLdg.png" /><figcaption>SFTP To Go Dashboard- Image by author</figcaption></figure><p>You can also view your add-ons on the dashboard here</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ki6-7Uts6iz_u4I8IX6GYw.png" /><figcaption>Different add-ons in the Dashboard- Image by the author</figcaption></figure><p>In your SFTP<strong> </strong>To Go<strong> </strong>dashboard, you can find all your information related to the credentials (SFTP / FTPS ACCESS and <a href="https://medium.com/javarevisited/7-best-aws-s3-and-dynamodb-courses-for-beginners-in-2021-a8a44b6066da">Amazon S3</a>), Network Security configuration, and Webhooks</p><h4>Verify the Access</h4><p>Now, once the add-on is installed the repository is ready to be accessed through the command line after you enter the given <strong>URI </strong>and <strong>password </strong>given in your SFTP To Go<strong> </strong>dashboard</p><pre>sftp *****************.sftptogo.com<br>sftp&gt; ls<br>dir<br>sftp&gt; mkdir storageFile<br>storageFile</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*JC5Ww67jCDPdnkzUzc7hOQ.png" /><figcaption>Command to access the Repository- Image by author</figcaption></figure><p>Here you can create your own directories and files for storage. We can also know the URL to your provisioned SFTP service, including the root username, password, and host by the following command</p><pre>heroku config:get SFTPTOGO_URL -a APP_NAME</pre><h4>Application Features</h4><p>While using SFTP To Go<strong> </strong>with Nodejs, the following npm package needs to be installed to the package.json file using the following command</p><pre>npm install ssh2-sftp-client</pre><p>and its usage as</p><pre>let Client = require(&#39;ssh2-sftp-client&#39;);<br>let sftp = new Client();<br><br>sftp.connect({<br>  host: &#39;*********.sftptogo.com&#39;,<br>  port: &#39;22&#39;,<br>  username: &#39;swapnilkant11&#39;,<br>  password: &#39;******&#39;<br>}).then(() =&gt; {<br>  return sftp.list(&#39;/&#39;);<br>}).then(data =&gt; {<br>  console.log(data, &#39;the data info&#39;);<br>}).catch(err =&gt; {<br>  console.log(err, &#39;catch error&#39;);<br>});</pre><h3>Adding Credentials to the <strong>SFTP To Go </strong>dashboard</h3><p>One can add other credentials also from the SFTP To Go<strong> </strong>dashboard other than the <strong>root</strong></p><figure><a href="https://www.java67.com/2020/07/top-5-courses-to-learn-linux-in-depth.html"><img alt="" src="https://cdn-images-1.medium.com/max/881/1*nuwGitmVRpiTNYCRgbZqPQ.png" /></a><figcaption>Adding credentials to SFTP To Go- Image by author</figcaption></figure><p>one can enter the <strong>username</strong> and the <strong>home directory, </strong>choose the permission from the list and click on <strong>Add credential </strong>now, there will be an additional credential added to the dashboard. Other permissions include <strong>read-only, write-only, </strong>and <strong>read and write</strong></p><h3>What are Webhooks?</h3><p>Events such as file deletion, file updation can be notified through an inbuilt feature in SFTP To Go<strong> </strong>called <strong>webhooks. </strong>One can configure <strong>webhooks </strong>from the dashboards by entering the Endpoint URL, authorization header, and type of events to monitor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*5CAxnQaDeOIxRzYAjpgV8g.png" /><figcaption>Webhook in SFTP To Go dashboard- Image by author</figcaption></figure><h3>Why do we need this?</h3><p>SFTP To Go comes up with various features which are very unique and reliable some of them are:</p><ul><li><strong>One-Click</strong> <strong>Setup</strong>: It provides seamless and quick access to our SFTP server, our secure SFTP server is just a few clicks away also providing us with <strong>one-click</strong> password rotation</li><li><strong>Amazon</strong> <strong>S3</strong> <strong>Access</strong>: This allows us to access our files using Amazon S3 service APIs which means that we can access the same files using a different, more modern interface</li><li><strong>Run</strong> <strong>At</strong> <strong>Scale</strong>: Built on top of the <a href="https://medium.com/javarevisited/5-best-aws-courses-for-beginners-and-experienced-developers-to-learn-in-2021-563212409fbd">Amazon cloud platform</a> can scale as traffic and file sizes increase and also, allows us to cap our storage and bandwidth quotas at fixed monthly costs and scales it up after discussion with the user</li><li><strong>Webhooks</strong>: Comes up with a webhook that automatically notifies us when there is any change in the file involving updation and deletion</li><li><strong>Network Inbound Rules: </strong>Defines a set of IP addresses that are safe to have access to our server</li></ul><h3>Security at SFTP To Go</h3><p>SFTP To Go comes up with the following security features:</p><ul><li><strong>Data Security: </strong>For data encryption, SFTP To Go uses SSL/TLS encryption on all their websites and microservices, and also the platform never uses our data to content or derive information from it for marketing or advertising and regular verification of security certificates and encryption algorithms is carried out to keep our data safe.</li><li><strong>Network Security: </strong>By default, all access is denied, and only explicitly allowed ports and protocols are allowed based on business requirements.</li><li><strong>System Security: </strong>The principle of least privilege is followed both internally by restricting their staff’s access to the minimum permissions needed to perform the work, as well as externally by asking the user for the minimum permissions required by their service when accessing 3rd party systems on the user’s behalf.</li></ul><h3>Summary</h3><p>Now, talking about the summary of this article we have the following features of SFTP To Go:</p><ol><li>Comes up with easy and fast setup features</li><li>Can be accessed from anywhere by adding the IP address which uses the SFTP server</li><li>Webhooks provides easy and fast notification features including simple to set up and manage</li><li>Comes up with extensive security features including password rotation, IP filtering, SSH keys, multiple credentials for fine-grained access control.</li></ol><h3>Conclusions</h3><p>SFTP To Go is one of the best and fully-managed cloud SFTP storage for your Heroku applications which is flexible, scalable, and easy to get started with and I am really impressed with the product on its features of <strong>zero management, scalability features, webhook feature, security, etc </strong>features which are not present in other cloud storage add-ons.</p><p>Would love to see:</p><ul><li>free tier available: allow developers to start small projects at no cost</li><li>file update notifications: allow webhooks to notify when an existing file has been modified</li></ul><p><strong>Keep learning and keep growing and also keep exploring more!</strong></p><p><strong>All the very best!</strong></p><p>For more interesting and informative articles and tips follow me on <a href="https://swapnilkant11.medium.com/"><strong>Medium</strong></a><strong> </strong>and<strong> </strong><a href="https://www.linkedin.com/in/swapnil-kant-279a3b148/"><strong>Linkedin</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b178410c6480" width="1" height="1" alt=""><hr><p><a href="https://medium.com/javarevisited/sftp-to-go-add-on-by-crazy-ant-labs-on-heroku-b178410c6480">SFTP To Go Add-On By Crazy Ant Labs On Heroku</a> was originally published in <a href="https://medium.com/javarevisited">Javarevisited</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Azure Data Factory as an ETL Tool and its Use Cases]]></title>
            <link>https://medium.com/javarevisited/azure-data-factory-as-an-etl-tool-and-its-use-cases-f36a7a421cee?source=rss-6d10c8780163------2</link>
            <guid isPermaLink="false">https://medium.com/p/f36a7a421cee</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[developer]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[azure]]></category>
            <dc:creator><![CDATA[Swapnil Kant]]></dc:creator>
            <pubDate>Sun, 06 Mar 2022 18:24:09 GMT</pubDate>
            <atom:updated>2022-03-07T15:50:01.242Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TASY1GYqydn2gaIlWWbaEg.png" /><figcaption>Azure Data Factory</figcaption></figure><h4><strong>1.1. Introduction</strong></h4><p>Nowadays we see that most of the developers are developing an interest in <a href="https://javarevisited.blogspot.com/2020/06/top-5-course-to-crack-google-cloud-professional-data-engineer-certification.html">data engineering</a> or <a href="https://medium.com/javarevisited/my-favorite-data-science-and-machine-learning-courses-from-coursera-udemy-and-pluralsight-eafc73acc73f">data science</a> although the era of software developers would never gonna end shortly, but! surely due to increasing demands of consistent electronic content creation, transactional data generation, and streams of data logs data science and data engineering will be in demand.</p><p>Many companies know that they have ample data with them, but are not sure how to use them! Although there are many <strong>ETL </strong>tools in the market, how does one know which one is the best among all? Which one will provide us with the optimal result with less cost? Which one will help us manage all our data in-house and provide accurate information from our data? <br>There are many such questions that a company has to go through before they make any decisions on the tool they need to use for their data!</p><p>One of the very famous <strong>ETL </strong>tools is <strong>Azure Data Factory</strong></p><p>Azure Data Factory allows us to:<br>· <strong>Copy</strong> data from many supported sources both on-premises and cloud sources<br>· <strong>Transform</strong> the data (cf. below paragraphs)<br>· <strong>Publish</strong> the copied and transformed data, sending it to a destination data storage or analytics engine<br>· <strong>Monitor</strong> the data flow using a rich graphical interface</p><h4>2.2 What is Azure Data Factory?</h4><p>Azure Data Factory is the cloud-based ETL and data integration service that allows us to create data-driven workflows for enabling data movement and transforming data at scale. It might be somewhat similar to <strong>Talend </strong>but, it has more features and is more powerful than <strong>Talend</strong></p><p>Let us first understand the basic components of <strong>Azure Data Factory:-</strong></p><figure><a href="https://javarevisited.blogspot.com/2020/09/top-5-courses-to-learn-microsoft-azure.html#axzz7M6Z8nhCJ"><img alt="" src="https://cdn-images-1.medium.com/max/751/1*YliqaqS0QijtIySG-cMvSw.png" /></a><figcaption>Fig 1.1 Figure showing three different activities</figcaption></figure><ul><li><strong>Activity: </strong>An activity is just like a logical operation or the action that we perform on our data. Some of the main important activities in <strong>Azure Data Factory </strong>are:<br>- <strong>Copy Activity: </strong>This activity involves copying the data from different datasets for further processing<br>- <strong>Lookup Activity: </strong>This activity involves the addition of datasets from data sources supported by the <strong>Azure Data Factory</strong><br>- <strong>Validation Activity: </strong>This activity is to ensure that the pipeline is only to be executed if it passes the necessary validations from the attached dataset reference, or the timeout has been reached<br>- <strong>Get Metadata Activity: </strong>This activity retrieves the metadata of any data from the <strong>Azure Data Factory</strong><br>- <strong>Conditional Activity: </strong>This activity checks for the given condition, if it turns out to be <strong>true </strong>it executes a certain set of activities and another set of activities when the condition evaluates to <strong>false</strong></li></ul><figure><a href="https://javarevisited.blogspot.com/2021/11/top-5-courses-to-crack-ai-900-microsoft.html"><img alt="" src="https://cdn-images-1.medium.com/max/295/1*123Ssn47mY2DSKEAcTqEFQ.png" /></a><figcaption>Fig 2.2 Figure showing a linked service</figcaption></figure><ul><li><strong>Linked Services:</strong> Linked services are a very important component to link your data store to the <strong>Azure Data Factory </strong>or <strong>Synapse Workspace</strong>. They are much like connection strings, which define the connection information needed for the service to connect to external resources</li></ul><figure><a href="https://www.java67.com/2020/07/5-free-courses-to-learn-microsoft-azure-cloud.html"><img alt="" src="https://cdn-images-1.medium.com/max/705/1*GhHxKJ0zZQ9esAx9hjkHVg.png" /></a><figcaption>Fig 3.3 Figure showing a pipeline</figcaption></figure><ul><li><strong>Pipelines: </strong>It’s the logical groups of activities that perform a specific job. The pipeline is basically the complete execution of the job that can be performed by individual activities. The pipelines are very useful when you talk about triggering the whole process from copying data to its transformation without individually triggering different activities separately</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/977/1*VL_N7rq5lq6uv7T8m9ToOw.png" /><figcaption>Fig 4.4 Figure showing one of the datasets</figcaption></figure><ul><li><strong>Datasets: </strong>Datasets are the named view of data that simply points or references the data you want to use in your activities as inputs and outputs</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fQAyjcnoVeYri7J2_Hg2DA.png" /><figcaption>Fig 5.5 Figure showing one of the dataflows</figcaption></figure><ul><li><strong>Mapping Dataflows: </strong>Mapping data flows are visually designed data transformations in <strong>Azure Data Factory</strong>. It helps to design the data transformation logics without actually coding and writing scripts</li><li><strong>Triggers: </strong>The triggers in ADF are another way that you can execute a pipeline run. Triggers represent a unit of processing that determines when a pipeline execution needs to be kicked off</li></ul><h4>3.3 How is Azure Data Factory different from other ETL tools?</h4><p>There are some features that distinguish <strong>Azure Data Factory</strong> from other tools.<br>· It also has the ability to run SSIS packages<br>· It auto-scales (fully managed PaaS product) based on the given workload.<br>· It allows to run up to once per minute<br>· It bridges on-premises &amp; <a href="https://medium.com/javarevisited/10-best-microsoft-azure-cloud-certification-it-professionals-can-aim-b7b6765d8ef1">Azure Cloud </a>seamlessly through a gateway<br>· It can handle big data volumes<br>· It can connect &amp; work together with other compute services (Azure Batch, HDInsights) to even run truly big data computations during ETL<br>· It supports both pre-and post-load transformations<br>· It integrates with about 80 data sources, including SaaS platforms, <a href="https://medium.com/javarevisited/top-5-sql-and-database-courses-to-learn-online-48424533ac61">SQL</a> and <a href="https://medium.com/javarevisited/5-best-nosql-database-programmers-and-developers-can-learn-42a0bdfa9a12?source=rss------programming-5">NoSQL databases</a>, generic protocols, and various file types<br>· It supports around 20 cloud and on-premises data warehouse and database destinations<br>· Pricing for <strong>Azure Data Factory’s</strong> data pipeline is calculated based on number of pipeline orchestration runs; compute-hours for flow execution and debugging; and number of Data Factory operations, such as pipeline monitoring, so we only need to pay as per we are using the resources, no less no more!</p><h4>4.4 Various Use-Cases of Azure Data Factory</h4><p>The various use cases of <strong>Azure Data Factory</strong> are as follows:<br>· Used in supporting the data migration for advanced analytics projects.<br>· Can be used to reform the EEL process from <a href="https://medium.com/javarevisited/top-10-free-courses-to-learn-microsoft-sql-server-and-oracle-database-in-2020-6708afcf4ad7">SQL server </a>integration services to extra data<br>· Can be used as a solution for getting data from a client’s server, or online data, to an Azure Data Lake. We create pipelines to orchestrate the data flow from source to target<br>· It can be used for carrying out various data integration processes and also, is one of the most famous <strong>ETL </strong>tools<br>· It can be used in integrating data from different ERP systems and loading it into Azure Synapse for reporting, it also supports analysis and reporting through <a href="https://medium.com/javarevisited/7-best-courses-to-learn-microsoft-power-bi-for-beginners-and-experienced-developers-83695c9428dc"><strong>power BI</strong></a> as well<br>· The ADF tools are impressively well-integrated, allowing quick development of <strong>ETL</strong>, <a href="https://medium.com/javarevisited/top-10-courses-to-learn-big-data-and-hadoop-best-of-lot-23ef8691633f">big data</a>, data warehousing, and <a href="https://medium.com/javarevisited/10-free-machine-learning-courses-for-beginners-181f83b4c816">machine learning solutions</a> with the flexibility to grow and adapt to changing or enhanced requirements</p><h4>5.5 Conclusion</h4><p>Data Factory offers us the possibility to easily integrate cloud data with on-premises data. It’s unique in its ease of use yet its ability to transform and enrich complex data. It delivers data integration, which is scalable, available and at low costs. Today this service is a crucial building block in any data platform &amp; machine learning tasks.</p><p><strong>Keep learning and keep growing and also keep exploring more!</strong></p><p><strong>All the very best!</strong></p><p>For more interesting and informative articles and tips follow me on <a href="https://swapnilkant11.medium.com/"><strong>Medium</strong></a><strong> and </strong><a href="https://www.linkedin.com/in/swapnil-kant-279a3b148/"><strong>Linkedin</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f36a7a421cee" width="1" height="1" alt=""><hr><p><a href="https://medium.com/javarevisited/azure-data-factory-as-an-etl-tool-and-its-use-cases-f36a7a421cee">Azure Data Factory as an ETL Tool and its Use Cases</a> was originally published in <a href="https://medium.com/javarevisited">Javarevisited</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Recursion and How does it Work?]]></title>
            <link>https://medium.com/javarevisited/what-is-recursion-and-how-does-it-work-ea20f1b4e697?source=rss-6d10c8780163------2</link>
            <guid isPermaLink="false">https://medium.com/p/ea20f1b4e697</guid>
            <category><![CDATA[data-structures]]></category>
            <category><![CDATA[recursion]]></category>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[blog]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Swapnil Kant]]></dc:creator>
            <pubDate>Wed, 16 Feb 2022 17:32:37 GMT</pubDate>
            <atom:updated>2022-02-17T15:51:11.805Z</atom:updated>
            <content:encoded><![CDATA[<p>Have you all while watching a code in any programming language observed where a function being defined is applied within its own definition? and maybe you just have ignored that, aah! it’s just the function call what is that something I have to do with?</p><p>Well, this is one of the most important concepts in programming, basically when you talk about <a href="https://medium.com/javarevisited/5-best-mathematics-and-statistics-courses-for-data-science-and-machine-learning-programmers-bf4c4f34e288">mathematics </a>and <a href="https://medium.com/javarevisited/8-best-computer-science-courses-for-beginners-to-learn-online-696379aa4e96">computer science</a>, in which a function calls itself directly or indirectly or a function being defined is applied within its own definition.</p><p>Now, most of us have written code that involves writing <strong>recursive</strong> <strong>functions</strong>, but before writing a well defined <strong>recursive function </strong>we will need to keep in mind the following <strong>two </strong>conditions which are very important, which are:</p><ol><li>The<strong> Base Case </strong>for our function, <strong>recursive functions </strong>apparently defines an infinite number of function calls, so to end the function call at some step you will need a <strong>base case </strong>for the same!</li><li>The <strong>Recursive Step </strong>Now, to reach the base case you will need a <strong>recursive step </strong>which will eventually lead to the <strong>base case </strong>and terminate the process.</li></ol><figure><a href="https://javarevisited.blogspot.com/2021/11/top-5-courses-to-learn-recursion-for.html"><img alt="" src="https://cdn-images-1.medium.com/max/480/1*VhKFV2tae8DYilKU8kMzFA.gif" /></a></figure><p>You might have heard of the <strong>Fibonacci Sequence, </strong>which is one of the best examples of <strong>recursion</strong></p><pre>Fib(0) = 0 as base case 1<br>Fib(1) = 1 as base case 2<br>For all integers <em>n</em> &gt; 1, Fib(<em>n</em>) = Fib(<em>n</em> − 1) + Fib(<em>n</em> − 2)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*WlaNpRuLDddq6dy_CHUq3g.gif" /></figure><p>Now, let’s take a look and understand <a href="https://javarevisited.blogspot.com/2012/12/recursion-in-java-with-example-programming.html">how <strong>recursion </strong>works</a>, although it’s easy to look into <strong>recursive functions </strong>and write down codes for solving various <a href="https://medium.com/javarevisited/50-data-structure-and-algorithms-interview-questions-for-programmers-b4b1ac61f5b0"><strong>algorithmic problems</strong></a><strong>, </strong>have you ever tried to understand it and see how it actually works behind the screen, and how you can make necessary changes to manipulate you function to perform something more than just stated? well, if not then don’t worry! I will make your work easy and help you understand how it actually works by taking an example of the <strong>Pre-Order traversal of a Binary Tree!</strong></p><p>Now, before jumping to our code and understanding more about <strong>recursion, </strong>let’s take a look at a new term that is <strong>recursion stack or the call stack</strong></p><h4>Definition:</h4><p><strong>The Call Stack </strong>is something very similar to what we all might have used in solving various <strong>algorithmic problems, </strong>that is <a href="https://javarevisited.blogspot.com/2022/02/-stack-and-queue-data-structure-interview-questions.html"><strong>Stack</strong></a><strong>, </strong>we all know, that it uses the <strong>LIFO rule </strong>to perform functions, which stands for “<strong>Last In First Out”, </strong>where you add things one after another and take off the most recently added item from it! Before moving to the <a href="https://javarevisited.blogspot.com/2016/07/binary-tree-preorder-traversal-in-java-using-recursion-iteration-example.html#axzz6dXsEfLvJ"><strong>Pre-Order traversal</strong></a><strong>, </strong>let’s know, what is preorder traversal?</p><h4><strong>Definition:</strong></h4><p><strong>Pre-Order</strong> <strong>traversal</strong> is a way of traversing in a <strong>binary tree </strong>where we start from the <strong>root node </strong>and traverse through the <strong>left child </strong>nodes of the <strong>parent </strong>node at each <strong>level </strong>and then to its <strong>right child.</strong></p><pre>Algorithm:<br>1. Visit the root.<br>   2. Traverse the left subtree, i.e., call Preorder(left-subtree)<br>   3. Traverse the right subtree, i.e., call Preorder(right-subtree)</pre><h4>Code For Preorder Traversal</h4><p>Now, let&#39;s have a look at the code:</p><pre><strong>void</strong> preorderTraversal(<strong>struct</strong> node* node){</pre><pre><strong>if</strong> (node == NULL)<br>   <strong>return</strong>;</pre><pre>/* first print data of node */<br>cout &lt;&lt; node-&gt;val &lt;&lt; &quot; &quot;;</pre><pre>/* recursion performed on left subtree */<br>preorderTraversal(node-&gt;left);</pre><pre>/* now perform recursion on right subtree */<br>preorderTraversal(node-&gt;right);</pre><pre>}</pre><p>Now, let’s have a look at the <strong>Pre-Order traversal of a Binary Tree </strong>and understand how <strong>recursion </strong>works!</p><figure><a href="https://www.java67.com/2022/01/how-to-find-maximum-sum-level-in-given-binary-tree-java.html"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CC2epfGPVnDHtKiaOr53GQ.jpeg" /></a><figcaption>Fig 0.0: A Binary Tree</figcaption></figure><p>Now, we have a given<a href="https://javarevisited.blogspot.com/2015/10/how-to-implement-binary-search-tree-in-java-example.html#axzz7DyRjLUX9"> <strong>binary tree</strong></a><strong> </strong>as shown above!</p><figure><a href="https://javarevisited.blogspot.com/2016/12/how-to-count-number-of-leaf-nodes-in-java-recursive-iterative-algorithm.html"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*865_FXy0BHNYicvxmWSZBA.jpeg" /></a><figcaption>Fig 1.1</figcaption></figure><p>Now, in the <strong>first step, </strong>we have a <strong>binary tree </strong>on the left-hand side and a <strong>call stack </strong>on the right-hand side, now the pointer points to the <strong>root node </strong>as shown, we move to the <strong>root node’s </strong>left child and push the <strong>root </strong>to our <strong>call stack</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L6nVgU40rBOGrr4aloA-KQ.jpeg" /><figcaption>Fig 2.2</figcaption></figure><p>In the <strong>second step, </strong>we pushed the <strong>current node </strong>into the <strong>call stack</strong> and move to its <strong>left child node</strong></p><figure><a href="https://dev.to/javinpaul/21-binary-tree-interview-questions-for-programmers-42bh"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4lwid_Eti0uVvpjgbzgcIw.jpeg" /></a><figcaption>Fig 3.3</figcaption></figure><p>In the <strong>third step, </strong>we pushed the <strong>current</strong> last <strong>left node</strong> to the stack and move to its <strong>left child node</strong></p><figure><a href="https://www.java67.com/2016/07/binary-tree-preorder-traversal-in-java-without-recursion.html"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*af4Jd7VjfcyNMO3bKQMw9A.jpeg" /></a><figcaption>Fig: 4.4</figcaption></figure><p>In the <strong>fourth step, </strong>we can observe that there is neither a <strong>left child </strong>nor<strong> </strong>the<strong> right child </strong>of 20, so we will pop out the current node and <strong>recursively </strong>move to the code line <strong>preorderTraversal(node-&gt;right) </strong>where node 10’s <strong>right child </strong>is pushed to our <strong>call stack, </strong>i.e. 25 is <strong>pushed </strong>into the <strong>call stack.</strong></p><figure><a href="https://www.java67.com/2016/07/how-to-implement-preorder-traversal-of-binary-tree-in-java.html"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CHV_NVC4SYCE22U1bv1asw.jpeg" /></a><figcaption>Fig 5.5</figcaption></figure><p>In the next <strong>pre final step, </strong>since we have no <strong>left </strong>or<strong> </strong>the<strong> right child </strong>of 25, we will pop out the current node and <strong>recursively </strong>move back to the <strong>root node’s right child node </strong>15 and push it into the <strong>call stack, </strong>and <strong>pop </strong>out <strong>root node </strong>5 from the <strong>call stack.</strong></p><figure><a href="https://www.java67.com/2021/07/recursion-programming-exercises-in-java.html"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w9UB3TspTY6w5sdt2OhPWA.jpeg" /></a><figcaption>Fig 6.6</figcaption></figure><p>In the <strong>final step </strong>we see that again, there are no <strong>left </strong>and the <strong>right child </strong>of node 15, so we <strong>pop</strong> out the current node and get the output as shown in the above diagram, which is our final output.</p><p>So, you can easily understand how <strong>recursion </strong>works, there are many more examples of <strong>recursion, </strong>I have tried to explain it in an easy way by taking a complex problem that many students face while writing the code for<a href="https://www.java67.com/2020/02/top-40-binary-tree-interview-questions.html"> <strong>binary trees</strong></a><strong>.</strong></p><p><strong>Keep learning and keep growing and also keep exploring more!</strong></p><p><strong>All the very best!</strong></p><p>For more interesting and informative articles and tips follow me on <a href="https://swapnilkant11.medium.com/"><strong>Medium</strong></a><strong> and </strong><a href="https://www.linkedin.com/in/swapnil-kant-279a3b148/"><strong>Linkedin</strong></a></p><p><a href="https://javarevisited.blogspot.com/2021/09/top-10-recursion-interview-questions.html?m=0">Top 25 Recursion Interview Questions for Coding Interviews</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea20f1b4e697" width="1" height="1" alt=""><hr><p><a href="https://medium.com/javarevisited/what-is-recursion-and-how-does-it-work-ea20f1b4e697">What is Recursion and How does it Work?</a> was originally published in <a href="https://medium.com/javarevisited">Javarevisited</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Find Pair With A Given Sum In A Sorted Doubly Linked List]]></title>
            <link>https://medium.com/javarevisited/find-pair-with-a-given-sum-in-a-sorted-doubly-linked-list-13fbac7b8a7d?source=rss-6d10c8780163------2</link>
            <guid isPermaLink="false">https://medium.com/p/13fbac7b8a7d</guid>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[blog]]></category>
            <category><![CDATA[students]]></category>
            <category><![CDATA[interview]]></category>
            <category><![CDATA[data-structures]]></category>
            <dc:creator><![CDATA[Swapnil Kant]]></dc:creator>
            <pubDate>Sun, 12 Sep 2021 16:33:24 GMT</pubDate>
            <atom:updated>2021-09-14T14:28:10.737Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*245dhBXbnU5l1sGf31-f2w.jpeg" /><figcaption>Doubly-Linked List</figcaption></figure><p>The linked list is one of the most important concepts and data structures to learn while preparing for interviews. Having a good grasp of <a href="https://medium.com/javarevisited/top-20-linked-list-coding-problems-from-technical-interviews-90b64d2df093">Linked Lists</a> can be a huge plus point in a coding interview.</p><h4>Problem Statement:</h4><p>Given a sorted doubly linked list of positive distinct elements, the task is to find pairs in a doubly-linked list whose sum is equal to the given value <strong>num.</strong></p><h4>Discussed Solution Approaches:</h4><ul><li>The brute force approach using <a href="https://javarevisited.blogspot.com/2015/09/how-to-loop-two-dimensional-array-in-java.html"><strong>nested loops</strong></a></li><li>Time optimized approach using <strong>Hashing</strong></li><li>Space and Time optimized approach using<strong> the two-pointer</strong> method</li></ul><h4>The key takeaway from this coding problem:</h4><ul><li>Learn about <strong>Hashing</strong></li><li>Learn about the <a href="https://javarevisited.blogspot.com/2013/05/find-if-linked-list-contains-loops-cycle-cyclic-circular-check.html"><strong>two-pointer </strong>approach</a></li></ul><pre>For example:</pre><pre>Suppose we have a sorted doubly-linked list as<br>5 &lt;-&gt; 7 &lt;-&gt; 10 &lt;-&gt; 15 &lt;-&gt; 20</pre><pre>and we are given the value of <strong>num </strong>is 17 i.e. <strong>num = 17. </strong>Now we have to find two integers from the sorted doubly-linked list which gives the total sum as <strong>17.</strong></pre><pre>So, we can clearly see that integers <strong>7 and 10 </strong>together add up to <strong>17, </strong>hence, in this case, the two numbers will be <strong>7 and 10</strong></pre><p><strong>Can you do it in the time complexity of O(n) with the space complexity of O(1)?</strong></p><p>Now, for time being forget the time complexity you have been given to find out the solution, for now, think of the most basic way to find a solution to the problem.</p><figure><a href="https://javarevisited.blogspot.com/2017/07/top-10-linked-list-coding-questions-and.html#axzz6fY0boe26"><img alt="" src="https://cdn-images-1.medium.com/max/480/1*eYP3RO9mk2OHjEUoUGdaKw.gif" /></a><figcaption>Let’s look at the code :p</figcaption></figure><h4>Naive Solution:</h4><p>The <strong>brute-force </strong>solution of the above problem would be using a nested loop to take a single integer and find the other integers whose sum is equal to the given <strong>num.</strong></p><p>Let’s look at the approach of this method:</p><pre>- Pick the first integer from the doubly-linked list<br>- Calculate the difference between the given num and the current integer in the doubly-linked list and store it in a variable <strong>diff</strong><br>- Now, search for <strong>diff</strong> by traversing the complete linked list which will be the second number in the list (if present)</pre><p><strong>The time complexity for this problem will be O(n²), n is the total number of nodes in the doubly linked list.</strong></p><p>Now, since this was a brute force solution, if you think a bit closely on optimization of the time complexity we can have a bit optimized solution.</p><p>Can you think of any such solution?</p><p>If you are not able to think let me give you a bit of a hint, we can solve the above problem with the help of <a href="https://javarevisited.blogspot.com/2010/10/difference-between-hashmap-and.html"><strong>hashing </strong></a>in <strong>O(n) time.</strong></p><h4>Hashing Solution:</h4><p>Let us have a glance at this hashing algorithm.</p><pre>start function findPair(reference to the headNode, number)</pre><pre>  unordered_set&lt;int&gt; nodeHash;<br>  <br>  struct node *headRef = headNode;<br>  start while(headRef-&gt;next is not NULL)<br>     insert headRef-&gt;data to nodeHash<br>  end while</pre><pre>  bool nodeFound = false;</pre><pre>  start while(headRef-&gt;next is not NULL)<br>      int diff = number - headRef-&gt;data<br>      start if(diff is present in nodeHash)<br>          print(diff and headRef-&gt;data)<br>          set nodeFound to true<br>          break the loop<br>      end if<br>  end while</pre><pre>  start if(nodeFound is false)<br>      print(&quot;Not Found&quot;)<br>  end if</pre><pre>end findPair</pre><p>Now, analyzing the above approach we see that this algorithm has a <strong>time complexity of O(n)</strong> which is quite good enough to solve this problem, but looking into the space complexity of the <a href="https://javarevisited.blogspot.com/2019/04/top-20-searching-and-sorting-algorithms-interview-questions.html">algorithm </a>we see that it takes an <strong>O(n) space</strong> which is not an optimized solution in terms of space complexity.</p><p>Can we do it even better?</p><p>Think of an approach where we can do this problem in <strong>O(n) time and O(1) space.</strong></p><p>Yes, we can do it in constant space by making use of the two-pointer approach and simply traversing and finding the pair if it exists.</p><p>Let us see this efficient approach.</p><h4>Two-Pointer Solution:</h4><p>Let us have a glance at this most efficient algorithm in terms of space and time.</p><pre>start function findPair(reference to the headNode, number)<br>    struct node *headPointer = headNode<br>    struct node *tailPointer = headNode</pre><pre>    start while(tailPointer-&gt;next is not NULL)<br>        tailPointer = tailPointer-&gt;next<br>    end while</pre><pre>    bool foundNode = false</pre><pre>    start while(headPointer is not equal to tailPointer and tailPointer-&gt;next is not equal to headPointer)<br>          start if(headPointer-&gt;data + tailPointer-&gt;data == number)<br>                print(headPointer-&gt;data and tailPointer-&gt;data)<br>                set foundNode as true<br>                break the loop<br>          end if<br>          start else<br>                start if(headPointer-&gt;data + tailPointer-&gt;data &lt; number)<br>                      headPointer = headPointer-&gt;next<br>                end if<br>                start else<br>                      tailPointer = tailPointer-&gt;prev<br>                end else<br>          end else<br>    end while</pre><pre>    start if(foundNode is false)<br>          print(&quot;Not Found&quot;)<br>    end if</pre><pre>end findPair</pre><p>Now, since you already know the <a href="https://medium.com/javarevisited/20-algorithms-coding-problems-to-crack-you-next-technical-interviews-23191f229788">algorithm </a>behind the problem so, you can write your own functional <strong>code </strong>in your desired programming language.</p><h4>Analyze:</h4><ul><li>Time Complexity: O(n), where <strong>n</strong> is the size of the given doubly-linked list this is because we are traversing the list only once.</li><li>Space Complexity: O(1), since we are not using any extra space to store the integers or the nodes.</li></ul><p><strong>Keep learning and keep growing and also keep exploring more!</strong></p><p><strong>All the very best!</strong></p><p>For more interesting and informative articles and tips follow me on <a href="https://swapnilkant11.medium.com/"><strong>Medium</strong></a><strong> and </strong><a href="https://www.linkedin.com/in/swapnil-kant-279a3b148/"><strong>Linkedin</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=13fbac7b8a7d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/javarevisited/find-pair-with-a-given-sum-in-a-sorted-doubly-linked-list-13fbac7b8a7d">Find Pair With A Given Sum In A Sorted Doubly Linked List</a> was originally published in <a href="https://medium.com/javarevisited">Javarevisited</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Count and Rearrange]]></title>
            <link>https://medium.com/javarevisited/count-and-rearrange-2efb209330d3?source=rss-6d10c8780163------2</link>
            <guid isPermaLink="false">https://medium.com/p/2efb209330d3</guid>
            <category><![CDATA[students]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[interview]]></category>
            <category><![CDATA[arrays]]></category>
            <category><![CDATA[algorithms]]></category>
            <dc:creator><![CDATA[Swapnil Kant]]></dc:creator>
            <pubDate>Tue, 10 Aug 2021 03:56:15 GMT</pubDate>
            <atom:updated>2021-08-10T11:34:37.488Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fIetqus0mfmpZNt7ahW9DQ.png" /></figure><h4>Problem Statement:</h4><p>Manju is given a task to find the count of a given number from the group of sorted numbers and then push the numbers to the front of the array to easily be picked up and counted.</p><h4>Discussed Solution Approaches:</h4><ul><li>The brute force approach using loop</li><li>The optimized approach using<a href="https://en.wikipedia.org/wiki/Binary_search_algorithm"> binary search</a></li></ul><h4>The key takeaway from this coding problem:</h4><ul><li>Learn about<a href="https://medium.com/javarevisited/binary-search-in-java-algorithm-eca288cb9bc2"> binary search</a></li><li>Learn about <a href="https://javarevisited.blogspot.com/2015/03/how-to-reverse-array-in-place-in-java.html">swapping elements in an array</a></li></ul><pre>For example:</pre><pre>If the numbers are 1, 1, 2, 2, 2, 2, 6 and you are given number 2 to count and push it to the front then, the final output would look like</pre><pre>Output: 2, 2, 2, 2, 1, 1, 6</pre><p><strong>Can you do it in the time complexity of O(logn) with the space complexity of O(1)? considering only finding and counting the numbers</strong></p><p>Now, for time being forget the time complexity you have been given to find out the solution, for now, think of the most basic way to find a solution to the problem.</p><p>Let’s break the problem into two subproblems:-<br>1. Find the count of the given number<br>2. Arrange the given number at the starting of all the other numbers</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/490/1*ZeYBlVyHQw9trIm4pwomGA.gif" /><figcaption>Let’s Code</figcaption></figure><h4>Naive Solution:</h4><p>The<strong> brute-force</strong> solution to the problem would be using <a href="https://javarevisited.blogspot.com/2020/01/how-to-implement-linear-or-sequential-search-in-java.html#axzz6VYKcmyZz"><strong>linear search</strong></a><strong> </strong>to count the occurrence of the number and then rearrange the number to the front of all other numbers.</p><p>Let’s look at the algorithm of this method:</p><pre>start function searchElementandSwapp(array, array_size, number){<br>     count = 0<br>     initialIndex = 0</pre><pre>     /* Get the count of the number and the index where it was first found */</pre><pre>     start for (index is 0 to index is less than array_size increment index by 1)<br>          start if(array[index] is equal to number and count = 0)<br>                increment count by 1<br>                set initialIndex as index<br>          end if<br>          start if(array[index] is equal to number and count != 0)<br>                increment count by 1<br>          end if<br>     end for</pre><pre>     /* Now swapp the numbers */</pre><pre>     start for (index is 0 to index is less than array_size and array[initialIndex] is equal to number increment  by 1)<br>        start if(array[initialIndex] is equal to number and initialIndex is less than array_size)<br>             swapp array[index] and array[initialIndex]<br>             increment initialIndex by 1<br>        end if<br>     end for</pre><pre>end searchElementandSwapp</pre><p><strong>This algorithm will take O(n) time to get executed and complete the operation with space of O(1)</strong></p><p>This algorithm will surely fail for the large size of the array and is inefficient for the larger array size, so what should we think to optimize it further?</p><p>Have you ever heard of a <a href="https://javarevisited.blogspot.com/2019/04/top-20-searching-and-sorting-algorithms-interview-questions.html"><strong>binary search </strong>algorithm</a>?</p><p>This problem satisfies all the rules of where binary search can be used as the array is completely sorted and we could find the element in a time of<strong> O(logn)</strong> exactly what we want!</p><h4>Binary Search Method:</h4><p>Let’s discuss the algorithm for first find the element that where it occurs in the array:</p><pre>start function searchElement(array, start_index, end_index, number)</pre><pre>       start if(end_index is less than 1)<br>           return -1<br>       end if</pre><pre>       mid_index = start_index + (end_index - start_index) / 2<br>  <br>       start if(array[mid_index] is equal to number)<br>            return mid_index<br>       end if</pre><pre>       start if(array[mid_index] is less than number)<br>            searchElement(array, mid_index + 1, end_index, number)<br>       end if</pre><pre>       start else<br>            searchElement(array, start_index, mid_index - 1, number)<br>       end else</pre><pre>end searchElement</pre><p>Now, after searching we will need to <a href="https://medium.com/javarevisited/20-array-coding-problems-and-questions-from-programming-interviews-869b475b9121">count the occurrence of a given number in the array</a>, for which we could write our algorithm as</p><pre>start function countElement(array, array_size, number)</pre><pre>     getIndex = searchElement(array, 0, array_size - 1, number)</pre><pre>     /* if the element is not present */</pre><pre>     start if(getIndex is equal to -1)<br>         return 0<br>     end if<br>     <br>     count = 0<br>     leftCountIndex = getIndex<br>     rightCountIndex = getIndex<br>     initialIndex = 0 /* this will let us know the first position where the number was found to swapp it later on */</pre><pre>     /* count left side */<br> <br>     start while(leftCountIndex is greater than or equal to 0 and array[leftCountIndex] is equal to number)<br>           increment count by 1<br>           set initialIndex to leftCountIndex<br>           decrement leftCountIndex by 1<br>     end while</pre><pre>     /* count right side */</pre><pre>     start while(rightCountIndex is less than the array_size and   array[rightCountIndex] is equal to number)<br>           increment count by 1<br>           incrementrightCountIndex by 1<br>     end while</pre><pre>     print count </pre><pre>     return initialIndex</pre><pre>end countElement</pre><p>Now, we have the <strong>initialIndex</strong> with you so we can easily use it to swap the numbers to the front <strong>(as shown in the naive approach swapping we will use the same way)</strong>, and boom! you have solved the problem in the desired time.</p><p>Now, since you already know the <a href="https://medium.com/javarevisited/20-algorithms-coding-problems-to-crack-you-next-technical-interviews-23191f229788">algorithm </a>behind the problem so, you can write your own functional <strong>code </strong>in your desired programming language.</p><h3>Analyze:</h3><ul><li>Time Complexity: O(logn), where <strong>n</strong> is the size of the given array this is because we are using the binary search approach.</li><li>Space Complexity: O(1), since we are not using any extra space to store the elements.</li></ul><p><strong>Keep learning and keep growing and also keep exploring more!</strong></p><p><strong>All the very best!</strong></p><p>For more interesting and informative articles and tips follow me on <a href="https://swapnilkant11.medium.com/"><strong>Medium</strong></a><strong> and </strong><a href="https://www.linkedin.com/in/swapnil-kant-279a3b148/"><strong>Linkedin</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2efb209330d3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/javarevisited/count-and-rearrange-2efb209330d3">Count and Rearrange</a> was originally published in <a href="https://medium.com/javarevisited">Javarevisited</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding the Algorithm behind Merge Sort for Linked Lists]]></title>
            <link>https://medium.com/javarevisited/understanding-the-algorithm-behind-merge-sort-for-linked-lists-79ef7f6f2f94?source=rss-6d10c8780163------2</link>
            <guid isPermaLink="false">https://medium.com/p/79ef7f6f2f94</guid>
            <category><![CDATA[sorting-algorithms]]></category>
            <category><![CDATA[data-structures]]></category>
            <category><![CDATA[students]]></category>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[interview]]></category>
            <dc:creator><![CDATA[Swapnil Kant]]></dc:creator>
            <pubDate>Sat, 07 Aug 2021 06:41:14 GMT</pubDate>
            <atom:updated>2021-10-20T05:52:48.857Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*7bSuYJt1jMocSW7m-4Sgow.jpeg" /><figcaption>Merge Sort of Linked lists</figcaption></figure><h4>What is a Merge Sort?</h4><p><strong>Merge sort</strong> is a famous and most optimized sorting algorithm which works on the principle of <a href="https://www.geeksforgeeks.org/divide-and-conquer-algorithm-introduction/">divide and conquer</a>. It is also known as the <a href="https://www.geeksforgeeks.org/divide-and-conquer-algorithm-introduction/">divide and conquer</a> algorithm with <strong>time complexity</strong> of <strong>O(n*logn).</strong></p><h4>How does it work?</h4><p><strong>Merge sort</strong> works in two easy steps:</p><ul><li>Divide the input list into two sub-lists, each sub-list having the same size.</li><li>Merge the two sub-lists into one big list, and each element in the list is ordered/sorted.</li></ul><p>Now, working of <a href="https://www.java67.com/2018/03/mergesort-in-java-algorithm-example-and.html"><strong>merge sort</strong> for an array</a> can be easily understood, for those who have not any idea on how<strong> merge sort</strong> works for an array can refer to this <a href="https://www.geeksforgeeks.org/merge-sort/">link</a></p><figure><a href="https://www.java67.com/2018/03/mergesort-in-java-algorithm-example-and.html"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*61Mf0zjVfd1s3_SzUNGxPA.png" /></a><figcaption>Pictorial Representation of Merge Sort</figcaption></figure><p>Now, moving on to how <strong>merge sort</strong> works for a <a href="https://medium.com/javarevisited/top-20-linked-list-coding-problems-from-technical-interviews-90b64d2df093">linked list</a>, we will first have a look at the <strong>simple approach</strong> of merge sort on a linked list</p><h4>Approach:</h4><pre>Step1: Call divideList() and find the mid node of the given linked list and also divide the list into two halves</pre><pre>Step2: Recursively call sortMerge() on both left and right sub-linked list and store the new head of the left and right linked list.</pre><pre>Step3: Call finalMerge() given the arguments new heads of left and right sub-linked lists and store the final head returned after merging.</pre><pre>Step4: Return the final head of the sorted linked list.</pre><p>In the above paragraphs we have discussed how <strong>merge sort </strong>works so in a similar manner here also we first <a href="https://javarevisited.blogspot.com/2017/03/how-to-reverse-linked-list-in-java-using-iteration-and-recursion.html">divide the linked list into two sub-list</a> and then recursively sort the two sub-lists and then finally merge both of them and return the final list.</p><p>Discussing the algorithm it looks something like this.</p><h4>Algorithm:</h4><p>In the very <strong>first step</strong> as stated you will have to divide the list into two sub-lists as explained with the algorithm below:</p><pre>start fundtion divideList(headList){<br>    slowPointer = headList<br>    fastPointer = next of headList<br>    <br>    start while(slowPointer is not NULL and (fastPointer is not NULL and next of fastPointer is not NULL)){<br>          slowPointer = next of slowPointer<br>          fastPointer = next of next of fastPointer<br>    end while</pre><pre>    return slowPointer</pre><pre>end divideList</pre><p>The above algorithm to divide a linked list into two halves is popularly known as the <a href="https://javarevisited.blogspot.com/2013/05/find-if-linked-list-contains-loops-cycle-cyclic-circular-check.html#axzz5jI43Qiwi"><strong>tortoise and hare algorithm</strong></a></p><p>Now, after dividing the list into two sub-lists we will be sorting each of the two sub-lists based on the actual <strong>merge sorting </strong>algorithm as explained with the algorithm below:</p><pre>start function finalMerge(firstListHead, secondListHead) <br>    firstHalf = new node<br>    secondHalf = new node<br>    <br>    firsthalf = secondHalf /* points to the head node later after sorting */</pre><pre>   start while(firstListHead is not NULL &amp;&amp; secondListHead is not NULL){<br>       start if(data of firstListHead node &lt;= data of secondListHead node)<br>            next of secondHalf = firstListHead<br>            firstListHead = next of firstListHead<br>       end if<br>       start else</pre><pre>             next of secondHalf = secondListHead<br>             secondListHead = next of secondListHead<br>       end else<br>       secondHalf = next of secondHalf <br>   end while</pre><pre>   /* If first list is not over */<br>   start while(firstListHead is not NULL){<br>       next of secondHalf = firstListHead<br>       firstListHead = next of firstListHead<br>   end while<br>  <br>   /* If second list is not over */<br>   start while(secondListHead is not NULL){<br>       next of secondHalf - secondListHead<br>       secondListHead = next of secondListHead<br>   end while</pre><pre>   return next of firstHalf</pre><pre>end sortMerge</pre><p>Now, after sorting the lists we will have to finally get the head reference of the complete sorted list and this function will also help us in getting the reference to the<strong> first half of the list and second half</strong> of the list using the <strong>divideList() </strong>function. the <a href="https://javarevisited.blogspot.com/2019/04/top-20-searching-and-sorting-algorithms-interview-questions.html">algorithm </a>looks something like this:</p><pre>start function sortMerge(headList){<br>   start if(next of headList is NULL)<br>        return headList<br>   end if<br>   <br>   midNode = new node<br>   secondHead = new node<br>   <br>   midNode = divideList(headList)<br>   secondHead = next of midNode<br>   next of midNode = NULL<br>   <br>   finalHead = finalMerge(sortMerge(midNode), sortMerge(secondHead))</pre><pre><br>   return finalHead</pre><pre>end sortMerge</pre><p>Now, eventually, this explains the complete algorithm on how <strong>merge sort </strong>works for a linked list and how the lists get sorted by the <strong>divide and conquer </strong>approach.</p><p>Now, since you already know the <a href="https://medium.com/javarevisited/20-algorithms-coding-problems-to-crack-you-next-technical-interviews-23191f229788">algorithm </a>behind the problem so, you can now, write your own functional <strong>code </strong>in your desired programming language.</p><h4>Dry Run:</h4><p>Now, consider this algorithm by taking an example of an unsorted linked list and dry run your test case as I have done here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v2dUmhmPvyOxiJMH3NccVA.jpeg" /><figcaption>Fig: 1.1</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R3y04oLdVJTEIslX4S8OvA.jpeg" /><figcaption>Fig: 1.2</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TLIZOAiFtlDlYRTq8SEf7w.jpeg" /><figcaption>Fig: 1.3</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qixXz67rAKKqDVpATL8AJw.jpeg" /><figcaption>Fig: 1.4</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*riWTwIYGCxi42lM_QBrcLw.jpeg" /><figcaption>Fig: 1.5</figcaption></figure><p>Now, you could easily relate that how the list is being sorted one by one and this would make you more clear on how this algorithm works!!</p><h3>Analyze:</h3><ul><li>Time Complexity: O(nlogn), where n is the number of nodes in the linked list.</li><li>Space Complexity: O(logn), where n is the number of nodes in the linked list. Since we are dividing the list into two sub-lists so we have the given final space complexity as O(logn)</li></ul><p><strong>Keep learning and keep growing and also keep exploring more!</strong></p><p><strong>All the very best!</strong></p><p>For more interesting and informative articles and tips follow me on <a href="https://swapnilkant11.medium.com/"><strong>Medium</strong></a><strong> and </strong><a href="https://www.linkedin.com/in/swapnil-kant-279a3b148/"><strong>Linkedin</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79ef7f6f2f94" width="1" height="1" alt=""><hr><p><a href="https://medium.com/javarevisited/understanding-the-algorithm-behind-merge-sort-for-linked-lists-79ef7f6f2f94">Understanding the Algorithm behind Merge Sort for Linked Lists</a> was originally published in <a href="https://medium.com/javarevisited">Javarevisited</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>