<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Narendiran Krishnan on Medium]]></title>
        <description><![CDATA[Stories by Narendiran Krishnan on Medium]]></description>
        <link>https://medium.com/@narenltk?source=rss-dc3776c53375------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*vkKKu6aEFGVmMWgTuWYsDg.jpeg</url>
            <title>Stories by Narendiran Krishnan on Medium</title>
            <link>https://medium.com/@narenltk?source=rss-dc3776c53375------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 16:54:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@narenltk/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Grafana to Visualize the AWS Cloud Watch Metrics]]></title>
            <link>https://medium.com/geekculture/grafana-to-visualize-the-aws-cloud-watch-metrics-6b55533fde36?source=rss-dc3776c53375------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b55533fde36</guid>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[visualization]]></category>
            <category><![CDATA[cloudwatch]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Narendiran Krishnan]]></dc:creator>
            <pubDate>Wed, 12 May 2021 16:09:14 GMT</pubDate>
            <atom:updated>2021-05-14T21:52:57.641Z</atom:updated>
            <content:encoded><![CDATA[<h4>Use Credentials File Method to visualize the Cloud Watch Metrics</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/474/1*M6QKhWlJpxI2ZhLHNY0EEA.png" /><figcaption>Some Random dude on Google Images</figcaption></figure><p>Well, everyone’s aware of Public Cloud and Private Cloud. Since it’s 2021 and most of us are working with the help of Cloud Platform. Well the reason why I am talking about something that is irrelevant is because some people want to monitor both Public and Private cloud using a single tool. Even though in AWS there is something known as Cloud Watch with which we can track everything. Still, when the requirement is like your client is using any Hybrid Cloud or else he wants to visualize the metrics, then the admins can’t directly give access to the Cloud Watch.</p><p>Hence there is this one stop solution for this and it is the Open Source tool Grafana. With Grafana we can connect multiple Cloud applications for monitoring and visualizing it. Moreover, it is easy for the user to edit and monitor the dashboard according to their needs.</p><p>On the other hand it is an Open Source tool and so we don’t have to invest anything for this.</p><p>These monitoring mostly help us with the cost optimization part, since we can track which of our applications has been used more i.e., which applications or servers are over-specced and so we can think of an alternate way or else downsize or even shut down at certain times.</p><p>It is not that Grafana alone can be used, we can also use some other applications like Zabbix. Am not gonna cover details about the list of Open Source tools. Since this blog is based on Grafana.</p><p>For this Tutorial, we assume that you have your Grafana Server installed on AWS EC2.</p><p>If not, no worries there is another blog that I wrote about it, kindly have a look at it.</p><p><a href="https://medium.com/geekculture/install-grafana-in-any-cloud-vm-for-monitoring-868f6436162d">Install Grafana in Any Cloud VM for Monitoring</a></p><p>3 ways of using Grafana with AWS CloudWatch are as follows,</p><ul><li>With the ARN Number of IAM Policy</li><li>With Credential File</li><li>By entering the Credentials into the Data Source (in Grafana Dashboard)</li></ul><p>Here in this blog we will be doing method 2 i.e. with Credential File</p><p>Alright now let’s start with making changes in the IAM …</p><blockquote><strong>Note:</strong></blockquote><blockquote><strong>Just a small update if you guys are wondering what’s IAM then look below,</strong></blockquote><blockquote><strong>What are IAM roles?</strong></blockquote><blockquote><strong>IAM roles are a secure way to grant permissions to entities that you trust. Examples of entities include the following:</strong></blockquote><blockquote><strong>IAM user in another account</strong></blockquote><blockquote><strong>Application code running on an EC2 instance that needs to perform actions on AWS resources. An AWS service that needs to act on resources in your account to provide its features</strong></blockquote><blockquote><strong>Users from a corporate directory who use identity federation with SAML.</strong></blockquote><blockquote><strong>IAM roles issue keys that are valid for short duration’s, making them a more secure way to grant access. Well if you still don’t get it, then let’s start with so you may understand it better about it.</strong></blockquote><h4><strong>Step 1: Create a new Policy</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/0*H_V4gW352XqfORu-" /><figcaption><strong>create_policy_IAM</strong></figcaption></figure><p>Just follow this, under <strong>Services → select IAM → Policies → Create Policy</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*F1BPQmmrcc5EhOxx" /><figcaption><strong>choose_service</strong></figcaption></figure><p>Now under the services → Cloud Watch</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/0*XdVwyFRk3wTQXi86" /><figcaption><strong>access_level_read_permission</strong></figcaption></figure><p>After that, do select the following Access Level Policies for which we can get the metrics report in Grafana. Then Review your Policy, give a name to your policy and describe what this policy contains so that when we have a look at it in the future we will be aware of what are the features this policy has. This is just to make our job easier.</p><p>Once you’re done with it, you have to verify whether what did has been created or not, just to cross check that is all.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gb8r8CK3DyqdDXj3" /><figcaption><strong>verify_grafana_policy</strong></figcaption></figure><h4><strong>Step 2: Creating a Role</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QQ6o_M5RCrzwTcgC" /><figcaption><strong>create_roles</strong></figcaption></figure><p>Just follow this, under <strong>Services → select IAM → Roles → Create Role</strong></p><p>Now select AWS Service → EC2 (under common use case) → next: Permission</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/998/0*gRZkSSUgE7p6p7UR" /><figcaption><strong>choose_use_cases</strong></figcaption></figure><p>Alright now we have to attach the policy which we created earlier with this Role, as shown below,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/995/0*URyGp1SWwhAUl54r" /><figcaption><strong>attach_permission</strong></figcaption></figure><p>Once the Policy has been attached we gotta review it, where ought to update the Role Name and Description etc.. and once we are done with this we can select Create Role.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/0*Fn_Z9b9TAdxbL1mg" /><figcaption><strong>review_role</strong></figcaption></figure><p>Once that is done we need to verify whether the role which we created is there or not,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/0*ndn5DoWp60CQ5c9T" /><figcaption>verify_grafana_role</figcaption></figure><h4><strong>Step 3: Creating a User</strong></h4><p>First, we have created the Policy then Role and then attached the Policy with the Role and now we are creating the User so that we can attach it to the EC2 Instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZpRiYfkkwMvdXuOa" /><figcaption><strong>create_iam_users</strong></figcaption></figure><p>Now select the Add User.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/987/0*fxqyT4dVCwbZbIF5" /><figcaption><strong>update_user_details</strong></figcaption></figure><p>Give the user name.</p><p>Kindly make sure that you select the programmatic access in the Access Types under select AWS access type. Which will give you the details of the Access Key and Secret Key. Make sure that you make a note of it as it will be generated only once and if you forget it then you need to create it again.</p><p>Then select Next Permission.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/0*tdoPQeai9N0i25ck" /><figcaption><strong>attach_grafana_policy</strong></figcaption></figure><p>Now Set Permission, if you have already created any group means select add user to group or else proceed with Attach existing policies directly → search for the Policy which we created in the last step and proceed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/987/0*KHlU3499CaTSgRCF" /><figcaption><strong>grafana_user_Cred</strong></figcaption></figure><p>Now here you can see the Access Key ID and Secret access key, which I was talking about earlier.</p><p>Yeah, you have to save it since it will be displayed only once just like the proverb “<strong>The Golden Words are not repeated</strong>”.</p><p>I guess now we are all set to go…. Now all that’s left is just attach the User with the EC2 instance which we have.</p><h4><strong>Step 4: Attach the User to EC2 Instance</strong></h4><p>Here I do expect you all to have your EC2 instance ready i.e. it should have been started.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9MmHsfy3Dx9qUaMM" /><figcaption><strong>attach_iam_role</strong></figcaption></figure><p>Now select the instance for which you wanted to update the IAM User and click on</p><p><strong>Action → Security → Modify IAM Role</strong>. (This is the latest since AWS has been keep on updating)</p><p>If you have been referring to the Old videos on YouTube or else some old Medium Blogs then you would have been navigated to</p><p><strong>Action → Instance Settings → Attach / Replace IAM Role.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/855/0*wSsFq0c9t0SLgEse" /><figcaption><strong>select_grafana_role</strong></figcaption></figure><p>Now you gotta search for the Role which you have created and select that and save it.</p><p>Now you ought to verify that too, so kindly check the security group settings of the EC2 instance as shown below,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rF8Ve58fQYOGwVIY" /><figcaption><strong>verify_security_tab_iam_role</strong></figcaption></figure><p>You can verify the IAM Role that is attached to the Security Group.</p><p>Now we are gonna attach the Credential file to the EC2 Instance</p><h4><strong>Step 5: Creating a Credentials File</strong></h4><p>Now log into your EC2 instance to which you have attached the IAM Role in the previous step and do the following as shown below,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/0*YWcF7AHUQULdOT46" /><figcaption><strong>credentials_share_grafana</strong></figcaption></figure><p>Here the code is as follows,</p><pre><strong>sudo nano /usr/share/grafana/.credentials</strong></pre><p>We are creating the Credentials file for our Grafana to access with the<strong> Access ID</strong> and <strong>Secret Key</strong> which we created in the previous step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/598/0*3lINjcyeiMjHGdh-" /><figcaption><strong>aws_access_creditial_default</strong></figcaption></figure><p>Replace the “ <strong>xoxoxo</strong> “ with your own <strong>access key id</strong> and <strong>secret key</strong> and <strong>region</strong>.</p><p>Save this and now we are going to give this the permission to access it.</p><p>The code is as shown below,</p><pre><strong>sudo chmod 0644 /usr/share/grafana/.credentials</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/849/0*M5TlfnX_OkZdJuK2" /><figcaption><strong>chmod_0644_cred</strong></figcaption></figure><h4><strong>Step 6: Configure Grafana with AWS Cloud Watch</strong></h4><p>Now log into to your Grafana Dashboard, with uname and pwd which you have configured,</p><blockquote><strong>Note:</strong></blockquote><blockquote><strong>Default uname and pwd is given below</strong></blockquote><blockquote><strong>Uname → admin</strong></blockquote><blockquote><strong>Pwd → admin</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/779/0*EJXxZGT5eSP99iBA" /><figcaption><strong>select_data_source</strong></figcaption></figure><p>Here you ought to see a dashboard panel like shown above and kindly select the Data Source, i.e Add your first data source.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/0*STJRW-wL1ixTupxY" /><figcaption><strong>search_cloud_watch</strong></figcaption></figure><p>Now search for the Cloud Watch in the search bar, then select it to configure the credential details.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*5j9p8Oj3NoRYQuGo" /><figcaption><strong>udpate_access_secret_key</strong></figcaption></figure><p>Here you gotta give your desired name, then under Authentication Provider select the Access &amp; Secret Key.</p><p>Now given the details of the Access Key and Secret Access Key alone with the Default Region.</p><p>Then click on Save and Test and you should get the following as the output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*iNyu0lZPrx4hKNN5" /><figcaption><strong>verify_udpated_details_correct</strong></figcaption></figure><p>If in any place you find any error which you wanna track then, go to the EC2 instance Terminal and give the following com<strong>mand to check the log reports,</strong></p><pre><strong>cd /var/log/grafana/</strong></pre><pre><strong>ls</strong></pre><pre><strong>vi grafana.log (or else) nano grafana.log</strong></pre><p>This will give you all the details of the error which you have faced.</p><h4><strong>Step 7: Create your beautiful Dashboard</strong></h4><p>Now you need to select the<strong> “ + → Dashboard “</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/569/0*VgZL0HOvpLi0Pqck" /><figcaption><strong>click_dashboard_for_making_panel</strong></figcaption></figure><p>Now you can either select the Add Panel in the top right corner as sometimes panel will be created created with Add Empty Panel and Add Row as shown below,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/973/0*T-PUdN38Oc7MAJXK" /><figcaption><strong>select _add_panel</strong></figcaption></figure><p>Then all the grafana query I will let you guys do it since it is easy though.</p><p>Here I have done an example with the CPU utilization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*aTJDrN0zJ9ua2ztW" /><figcaption><strong>Cpu_utilization</strong></figcaption></figure><p>The Grafana Version that I have used for this blog is 7.5.5</p><blockquote><strong>If you wish to stay connected,</strong></blockquote><ul><li><a href="https://www.linkedin.com/in/narenltk/">Narendiran Krishnan - Coimbatore, Tamil Nadu, India | Professional Profile | LinkedIn</a></li><li><a href="https://github.com/narenltk">narenltk - Overview</a></li></ul><blockquote><strong>you can just google “ narenltk / narendiran krishnan ” or just drop a mail to → narenltk@gmail.com → Happy to help..!!!</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b55533fde36" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/grafana-to-visualize-the-aws-cloud-watch-metrics-6b55533fde36">Grafana to Visualize the AWS Cloud Watch Metrics</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Install Grafana in Any Cloud VM for Monitoring]]></title>
            <link>https://medium.com/geekculture/install-grafana-in-any-cloud-vm-for-monitoring-868f6436162d?source=rss-dc3776c53375------2</link>
            <guid isPermaLink="false">https://medium.com/p/868f6436162d</guid>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[ec2]]></category>
            <dc:creator><![CDATA[Narendiran Krishnan]]></dc:creator>
            <pubDate>Wed, 05 May 2021 15:47:57 GMT</pubDate>
            <atom:updated>2021-05-13T04:37:09.389Z</atom:updated>
            <content:encoded><![CDATA[<h4>Install Grafana on EC2 and start with an Auto Start</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8S0jbn12Gyvo0UM1o14piw.png" /><figcaption><strong>Credit: Some Random dude on Google Images</strong></figcaption></figure><h4>What is Grafana ?</h4><p>Grafana is an awesome tool which helps you Query, Visualize and create alerts on those Visualizations. End users can create complex monitoring dashboards using interactive query builders. In this blog, we will install it in the AWS EC2 Instance. But the way to install it in a different cloud VM is similar as only the name alone differs as shown below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*jauus3N8Q-3HIFDaDD4YAQ.png" /><figcaption><strong>cloud_comparison</strong></figcaption></figure><h4><strong>Grafana Features:</strong></h4><p>It’s an open source tool and so it does have a great community as we can post any kinda questions and the random guy on the internet will definitely answer those questions as soon as possible.</p><p>Moreover, it supports all the major databases in the same dashboard.</p><p>Based on my personal experience this works well with the time series database. I found it most helpful.</p><h4><strong>Steps to install Grafana in EC2 Instance:</strong></h4><ul><li>Create a EC2 instance with Ubuntu / Linux based OS</li><li>Configure the Security Groups</li><li>Install Grafana</li><li>Adding Grafana to APT Repository</li><li>Start and add Auto Start to Grafana</li><li>Logging into Grafana Dashboard</li></ul><h4><strong>Step 1: Create a EC2 Instance</strong></h4><p>You can create an EC2 Instance with Ubuntu and grab whatever is free as I did that only.</p><h4><strong>Step 2: Configure Security Groups</strong></h4><p>Create a security group with open 22 and 3000 port.</p><p><strong>22 port → </strong>ssh access</p><p><strong>3000 port →</strong> Access to the Grafana dashboard via a web browser</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0f7-TfTHf8ULBKQVgrWIcQ.png" /><figcaption><strong>Security_rules</strong></figcaption></figure><h4><strong>Step 3: Install Grafana</strong></h4><p>Always make sure to get the latest version. You can verify that from the following link given below,</p><p><a href="https://grafana.com/grafana/download?platform=linux">Download Grafana | Grafana Labs</a></p><p><strong>code:</strong></p><pre><strong>sudo apt-get install -y adduser libfontconfig1</strong></pre><pre><strong>sudo wget </strong><a href="https://dl.grafana.com/oss/release/grafana_7.5.5_amd64.deb"><strong>https://dl.grafana.com/oss/release/grafana_7.5.5_amd64.deb</strong></a></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*isHMb-bBguHuYCIA6vhP8A.png" /><figcaption><strong>wget_command</strong></figcaption></figure><p><strong>code:</strong></p><pre><strong>sudo dpkg -i grafana_7.5.5_amd64.deb</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9NiPKbffWyivBtGig4GIBw.png" /><figcaption><strong>dpkg_code</strong></figcaption></figure><h4><strong>Step 4: Adding the Grafana sources to our APT Repository</strong></h4><blockquote><strong>Note: APT Repository → It’s a collection of deb packages with metadata which is readable by the apt-* family of tools, namely, apt-get. Having the APT Repository will let you perform the package install, removal, upgrade etc (End of Thinking Capacity)…!</strong></blockquote><p>we need to modify our <strong>/etc/apt/sources.list</strong> file by adding a line to the bottom of it.</p><pre><strong>sudo nano /etc/apt/sources.list</strong></pre><p>Scroll down all the way to the bottom using your down-arrow key until you cannot scroll down and then add the following line at the bottom:</p><pre><strong>deb </strong><a href="https://packagecloud.io/grafana/stable/debian/"><strong>https://packagecloud.io/grafana/stable/debian/</strong></a><strong> stretch main</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/767/1*GfITbzW9AjcZIV1PLNnGjQ.png" /><figcaption><strong>adding_grafana_source_repo</strong></figcaption></figure><p>Now we need to add the Package Cloud key. This allows us to install signed packages.</p><pre><strong>curl </strong><a href="https://packagecloud.io/gpg.key"><strong>https://packagecloud.io/gpg.key</strong></a><strong> | sudo apt-key add -</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*XWa9OJAM4BO3P5oQTmJ0fA.png" /><figcaption><strong>adding_grafana_source_repo</strong></figcaption></figure><p>Once you’re done with this we gotta update and verify it once again just to ensure that everything we did went well.</p><pre><strong>sudo apt-get update</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*E6UzGgdg1G8876Q6RbTL2A.png" /><figcaption><strong>install_grafana</strong></figcaption></figure><h4><strong>Step 5: Starting the Grafana Server and adding it to Auto start</strong></h4><p>Starting Grafana is kinda simple just starting the services as shown below</p><pre><strong>sudo service grafana-server start</strong></pre><p>Now we are going to add it to Auto Start, run:</p><pre><strong>sudo update-rc.d grafana-server defaults</strong></pre><pre><strong>sudo systemctl enable grafana-server.service</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9NhgcU-kW60RCNM3UnwV7A.png" /><figcaption><strong>adding_autostart</strong></figcaption></figure><p>Alright, now it’s time to visualize the output …!!!</p><p><strong>Logging in to Grafana</strong></p><p>Open up your Web Browser and enter the IP of your server followed by port 3000, like</p><blockquote>http://&lt;you IP address&gt;:3000</blockquote><p>You should then be greeted by the Grafana Login Window.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*efVk7FZMqpvKKNGhwXMEow.png" /><figcaption><strong>finally_grafana_board</strong></figcaption></figure><blockquote><strong>Note:</strong></blockquote><blockquote><strong>User Name: admin</strong></blockquote><blockquote><strong>Password: admin</strong></blockquote><p>Once you enter the details you will be prompted to change the User Name and Password, so do it according to your wish</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/526/1*PO4FF_mdJu6Cn-yOdkAeSg.png" /><figcaption><strong>new_password_prompt</strong></figcaption></figure><p>After this, you will finally visualize the Grafana Dashboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fmcwm0-PMXxWVuu0IsdaNw.png" /><figcaption><strong>atlast_grafana</strong></figcaption></figure><p>The Grafana Version that I have used for this blog is 7.5.5</p><p>If you wanna implement Grafana to get the Cloud Watch Metrics in your EC2 instance then check the blog below,</p><p><a href="https://narenltk.medium.com/grafana-to-visualize-the-aws-cloud-watch-metrics-6b55533fde36">Grafana to visualize the AWS Cloud Watch Metrics</a></p><blockquote><strong>If you wish to stay connected,</strong></blockquote><ul><li><a href="https://www.linkedin.com/in/narenltk/">Narendiran Krishnan - Coimbatore, Tamil Nadu, India | Professional Profile | LinkedIn</a></li><li><a href="https://github.com/narenltk">narenltk - Overview</a></li></ul><blockquote><strong>you can just google “ narenltk / narendiran krishnan ” or just drop a mail to → narenltk@gmail.com → Happy to help..!!!</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=868f6436162d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/install-grafana-in-any-cloud-vm-for-monitoring-868f6436162d">Install Grafana in Any Cloud VM for Monitoring</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Node Forever JS on Any Virtual Machine]]></title>
            <link>https://medium.com/geekculture/node-forever-js-on-any-virtual-machine-e5d573583b67?source=rss-dc3776c53375------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5d573583b67</guid>
            <category><![CDATA[forever]]></category>
            <category><![CDATA[ec2]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[web-apps]]></category>
            <category><![CDATA[node]]></category>
            <dc:creator><![CDATA[Narendiran Krishnan]]></dc:creator>
            <pubDate>Mon, 15 Mar 2021 12:09:17 GMT</pubDate>
            <atom:updated>2021-03-31T20:03:28.141Z</atom:updated>
            <content:encoded><![CDATA[<h4>Host your web apps forever from any Virtual Machine</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0Y6PEWUF5veyfi0g6vZPHQ.png" /><figcaption>credit goes to some random dude in this world</figcaption></figure><p>If you ever wanted to host your Node.js application forever, then in this right place. Here you will learn how to run your Node js app forever, even when the console is closed.</p><h4><strong>Forever</strong> → A simple CLI tool for ensuring that a given script runs continuously (i.e. forever)</h4><p>If you have ever faced an issues as shown below while using Virtual Machine, then, this blog will help you to overcome the problem.</p><pre>“ <strong>client_loop: send disconnect: Broken pipe</strong> “</pre><p>Yes, this happens always while working with cloud and this is where the package Forever Js comes in handy.</p><p>Here I will uncover all the useful commands which are required for Forever Js. Now let’s get started, but before getting started, let install forever js.</p><p>Code is as follows,</p><pre><strong>sudo apt-get update</strong></pre><pre><strong>sudo su</strong></pre><pre><strong>npm i forever -g</strong></pre><p>Next step is to create a test node js application which we will use for this forever application.</p><blockquote><strong>Note:</strong></blockquote><blockquote><strong>Guys here I am using an EC2 Server instance and as usual, I just grabbed whatever is under the free tier limit i.e. Ubuntu or any Linux based OS.</strong></blockquote><p>Here is the code, first create a <strong>server.js</strong> file</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*siOxWY6ygFMY78i5RQX0YA.png" /><figcaption>Server js code</figcaption></figure><p>Once you do this just verify whether it is working fine or not</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/825/1*WMIYJZ6kGKcmSle6HfJpkQ.png" /><figcaption>verify whether it works without error</figcaption></figure><p>Now visit to the browser with the IP which is assigned to your EC2 instance so that you can verify that as shown below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/721/1*4xjyTEd3eaSGsHNho3UNJA.png" /><figcaption>verify from your browser</figcaption></figure><p>Alright, I guess we now have our beloved server running successfully.</p><p>Now all that is left is just to make use of the forever js lib.</p><h4><strong>Forever Commands</strong></h4><h4><strong>Forever Start</strong></h4><p>Usually we start our server by running the code as node server.js but here there will be a small change using the code while running and the code is as follows,</p><pre><strong>forever start server.js</strong></pre><h4><strong>Forever list</strong></h4><p>Alright, now we can verify using the forever list command</p><pre><strong>forever list</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*shRtqFX5xkY3WE_CJuYPAQ.png" /><figcaption>forever list</figcaption></figure><p>This command is actually used for listing all the scripts which are running forever and it doesn’t stop here. It also shows us location, process id, up-time and also the details of the log file name of each script that’s running.</p><h4><strong>Forever Stop</strong></h4><p>The next command is the stop, yes, if there is a start, then there will be a stop definitely.</p><p>We can stop using the file name of the script you want to stop.</p><p>The code is as follows,</p><pre><strong>forever stop server.js</strong></pre><p>With this command you have to reach the specific location of the file where the node application is running and only then stop or else it will not work properly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nZMyTdDBkv8b0x170sEUDA.png" /><figcaption>forever stop cmd</figcaption></figure><p>Alright, what if you want to stop a file based on uid.</p><h4><strong>Forever stop uid</strong></h4><p>Well, whenever you start or else list using the forever command you will be displayed with an output of whichever script is running with a uid. So we can also stop using those commands rather than typing out the name of the script file. You can verify by using the start or list command. Now type the following commands,</p><pre><strong>forever stop o</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DlfqjXFSOdfqTTAXcJu9fA.png" /><figcaption>forever stop 0</figcaption></figure><p>Even here you can see the uid which I spoke about.</p><h4><strong>Forever Stopall</strong></h4><p>Alright, if you wish to stop everything that’s running simple stopall, but there are two commands one is forever command and the other one is a usual node command.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dvh1gumQYN2YOp2oLNF5Jw.png" /><figcaption>forever stopall</figcaption></figure><h4><strong>Forever Logs</strong></h4><p>If you wish to find the logs, then you can just use the following code</p><pre><strong>forever logs</strong></pre><p>It will list the list, the path of each and every script which is running along with the uid.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wrjssJ3sITd5nJqwuxpBCg.png" /><figcaption>forever logs</figcaption></figure><p>Now that we stop the forever script and verify our browser and it should throw up the following screen,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wlAxmI0HHpMkuuTo-s9fyA.png" /><figcaption>killall node</figcaption></figure><p>For this you can use any commands, i.e. the normal node command or forever command,</p><pre><strong>killall node</strong></pre><pre><strong>forever stopall</strong></pre><p>Now you can use the forever start command and close the EC2 instance terminal and then verify for yourself.</p><p>I guess I have given details of most of the Forever js Commands and the rest of the few other commands I will give you guys and do try it out.</p><h4>Commands:</h4><pre><strong>forever restart server.js</strong></pre><pre><strong>forever restart 0 --&gt; </strong>restarts uid o script alone</pre><pre><strong>forever restartall --&gt; </strong>restarts all running forever scripts</pre><pre><strong>forever -h --&gt; </strong>-h, –help: Command line help information</pre><pre><strong>forever config --&gt; </strong>lists all forever user configurations</pre><pre><strong>forever clear --&gt; </strong>clears the specified forever config by its key</pre><pre><strong>forever cleanlogs --&gt; </strong>deletes all historical forever log files</pre><pre><strong>forever set --&gt; </strong>defines a forever config value for an unique key</pre><p>You can find the server.js file in my GitHub.</p><blockquote><strong>GitHub </strong><a href="https://github.com/narenltk/forever_node_js"><strong>Link</strong></a></blockquote><p>If you want to learn how certificate binding is done for web apps using openssl then follow the blog below,</p><p><a href="https://javascript.plainenglish.io/generate-ssl-certificate-using-node-js-for-web-apps-73d452ad5898">Generate SSL Certificates using Node.js for web apps</a></p><blockquote><strong>If you wish to stay connected,</strong></blockquote><ul><li><a href="https://www.linkedin.com/in/narenltk/">Narendiran Krishnan - Coimbatore, Tamil Nadu, India | Professional Profile | LinkedIn</a></li><li><a href="https://github.com/narenltk">narenltk - Overview</a></li></ul><blockquote><strong>you can just google “ narenltk / narendiran krishnan ” or just drop a mail to → narenltk@gmail.com → Happy to help..!!!</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5d573583b67" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/node-forever-js-on-any-virtual-machine-e5d573583b67">Node Forever JS on Any Virtual Machine</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setup Your Own CoTurn Server Using AWS EC2 Instance]]></title>
            <link>https://medium.com/swlh/setup-your-own-coturn-server-using-aws-ec2-instance-29303101e7b5?source=rss-dc3776c53375------2</link>
            <guid isPermaLink="false">https://medium.com/p/29303101e7b5</guid>
            <category><![CDATA[ec2]]></category>
            <category><![CDATA[webrtc]]></category>
            <category><![CDATA[peer-to-peer]]></category>
            <category><![CDATA[servers]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Narendiran Krishnan]]></dc:creator>
            <pubDate>Fri, 26 Feb 2021 15:08:08 GMT</pubDate>
            <atom:updated>2021-05-28T07:47:59.160Z</atom:updated>
            <content:encoded><![CDATA[<h4>You will know the small 2021 update of CoTURN Server</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*Yc5hvWrDJ5vqZe60a3RJ9Q.jpeg" /><figcaption>Some random dude in google images</figcaption></figure><p>NAT (Network Address Translation) traversal servers in webRTC are the reason that the media gets properly connected and those servers are STUN and TURN.</p><p>If you guys are wondering what this webRTC still then have a read through my blog on it</p><p><a href="https://narenltk.medium.com/webrtc-architectural-overview-peer-to-peer-in-the-browser-6f0dee9d5ddf">webRTC → Architectural overview: Peer-To-Peer in the Browser</a></p><h3>What is Turn Server ?</h3><p>Traversal Using Relays around NAT is a protocol that assists in the traversal of network address translators or firewalls for multimedia applications.</p><p>It may be used with the Transmission Control Protocol and User Datagram Protocol. It is most useful for clients on networks masqueraded by symmetric NAT devices.</p><h3>When Turn Server is needed?</h3><p>On a typical webRTC app, about 20% of connections require a TURN server. It may work fine for you, but try accessing your webRTC service from a cell phone connection (which will usually require TURN), and you’ll see that not all connections are equal when it comes to p2p.</p><p>For peer to peer, you will need to install and run a TURN server.</p><p>But the question is why ?</p><p>Because some firewalls block certain types of traffic. Many just block UDP, but some may even block TCP.</p><h3>Turn server Hardware</h3><p>It is not mandatory that we should have a higher configuration or something. You can make the setup even with basic configurations like using a free tier instance, which is available on the AWS for EC2 i.e. t2.micro.</p><p>If you wish to test whether your browser supports this webRTC then you can test it out here.</p><p><a href="https://test.webrtc.org/">WebRTC Troubleshooter</a></p><p>All right Now let’s start to implement it from the EC2 instance,</p><h3><strong>Step 1:</strong></h3><p>Here you can just grab whatever is available on the free tier for EC2 instance.</p><h3><strong>Step 2: EC2 Instance security group settings</strong></h3><p>Then only changes while launching an instance is in the security groups, I have given the details of the security groups port then we need to</p><pre><strong>UDP → 32355–65535, 3478–3479</strong></pre><pre><strong>TCP → 32355–65535, 3478–3479</strong></pre><h3><strong>Step 3: Install coturn</strong></h3><p>CoTurn is an Open Source Implementation of TURN and STUN Servers.</p><p>Code is as follows,</p><pre><strong>sudo apt-get -y update</strong></pre><pre><strong>sudo apt-get install coturn</strong></pre><h3><strong>Step 4:</strong> <strong>Stop the CoTurn Server &amp; make the changes</strong></h3><p>Before we make any changes we need to stop the coturn server first.</p><p>Then we need to uncomment the <strong>TURNSERVER_ENABLED=1</strong> and move the following location,</p><p>Code is as follows,</p><pre><strong>sudo systemctl stop coturn / sudo systemctl stop coturn.services</strong></pre><pre><strong>sudo vi /etc/default/coturn</strong></pre><p>Now un-comment and it should look like the following,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/1*oBFWdqzEbUGItmwlXRofcA.png" /><figcaption>UNCOMMENT IT</figcaption></figure><h3><strong>Step 5: Make changes in the turnserver.conf file / coturn setting file</strong></h3><p>Move to that location or else open the file from the following location,</p><p>Code is as follows,</p><pre><strong>cd /etc/</strong></pre><p>Then you need to open either using a <strong>vim editor / nano editor / vi editor</strong> in super user mode i.e. <strong>“sudo su”</strong></p><p>Code is as follows,</p><pre><strong>sudo nano turnserver.conf</strong></pre><p>Remove the comments from the following lines, and add any necessary information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/1*g7SDwB-2WZBeL0khI5wQYg.png" /><figcaption>port details to be uncommented</figcaption></figure><p>Now you need to update your own public IP of the EC2 instance which you have configured.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*huZ8R4fDAUjyLvR00NYOHA.png" /><figcaption>listening ip</figcaption></figure><p>Here also you need to configure the same give the EC2 instance Public ip, i.e. listening ip and relay ip should be the same, but in this example it’s different since i restarted my EC2 instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/718/1*Nb64-MdWxHQFxplqF2mnhA.png" /><figcaption>relay ip</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*vPb-oLEwwe55hf5Uw4UWKA.png" /><figcaption>finger and lt cred mech</figcaption></figure><blockquote><strong>Here there is an update given by the official creator since it is 2021, they gave an update it seems</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/690/1*_wB8rSkRFRcRsd4cZNzLvg.png" /><figcaption>2021 update by CoTurn</figcaption></figure><p>Here you can see that the <strong>“no-loopback-peers”</strong> option has been replaced by <strong>“allow-loopback-peers”</strong>.</p><p>It is up to you to comment that or un-comment but I highly recommend you all to comment that line.</p><p>Once you have done all these properly then you’re good to go.</p><h3><strong>Step 6: start the coturn server</strong></h3><p>Since we have stopped our server in step 4 we need to start it up,</p><p>Code is as follows,</p><pre><strong>sudo systemctl start coturn / sudo systemctl start coturn.services</strong></pre><p>Then you need to verify it using the following code,</p><pre><strong>sudo systemctl status coturn / sudo systemctl status coturn.services</strong></pre><p>Once you hit enter you should see the out and it should show you the output as “ <strong>active (running) “.</strong></p><p>You can check and verify this in the following website,</p><p><a href="https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/">Trickle ICE</a></p><p>Here you can see the current working Stun Server i.e. the free public server which is hosted by our Uncle Google. Well, there will be an option to change the details in the ICE Server. You need to first</p><ol><li>Reset to defaults</li><li>Select and remove the details in the ICE Servers</li><li>Then enter the turn server details</li></ol><pre><strong>turn:&lt; PUBLIC IP / EC2 Public Instance IP / Domain Name&gt;:3478?transport=tcp </strong></pre><pre><strong>example:<br>turn:narenltk.com:3478</strong></pre><p>4. Then given the username as “ <strong>&lt;USERNAME&gt;</strong> ” and password as “ <strong>&lt;PASSWORD&gt;</strong> “</p><p>5. Then hit “<strong>add Server</strong>” select “<strong>Get Candidates</strong>”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/992/1*0Idboio5bd9pm_OOMB-zFQ.png" /><figcaption>done message</figcaption></figure><p>You should always see the “<strong>Priority</strong>” column should give a result “<strong>Done</strong>” or if it is not configured properly, then you will end up with an output “<strong>Unreachable</strong>”.</p><blockquote><strong>But if you need to update your own uname, password or you need to update your own certificate and key with domain all can be done just proceed to scroll and look further options and you will find the following details below,</strong></blockquote><pre><strong>user</strong></pre><pre><strong>cert</strong></pre><pre><strong>pkey</strong></pre><pre><strong>server-name</strong></pre><p>I will let you guys do the changes on this.</p><blockquote><strong><em>Moreover always update the Certificate and Private Key in the “ PEM “ format.</em></strong></blockquote><p>If you wanna generate your own SSL and then add it then follow the blog below,</p><p><a href="https://js.plainenglish.io/generate-ssl-certificate-using-node-js-for-web-apps-73d452ad5898">Generate SSL Certificates using Node.js for web apps</a></p><p>Once you have made all the changes you to add the details in the following way to your code,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/786/1*4Hod5_z9L_VDFkpKvHEvfA.png" /><figcaption>code update details</figcaption></figure><blockquote><strong>If you wish to stay connected,</strong></blockquote><ul><li><a href="https://www.linkedin.com/in/narenltk/">Narendiran Krishnan - Coimbatore, Tamil Nadu, India | Professional Profile | LinkedIn</a></li><li><a href="https://github.com/narenltk">narenltk - Overview</a></li></ul><blockquote><strong>you can just google “ narenltk / narendiran krishnan ” or just drop a mail to → narenltk@gmail.com → Happy to help..!!!</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=29303101e7b5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/setup-your-own-coturn-server-using-aws-ec2-instance-29303101e7b5">Setup Your Own CoTurn Server Using AWS EC2 Instance</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Generate SSL Certificates using Node.js for web apps]]></title>
            <link>https://javascript.plainenglish.io/generate-ssl-certificate-using-node-js-for-web-apps-73d452ad5898?source=rss-dc3776c53375------2</link>
            <guid isPermaLink="false">https://medium.com/p/73d452ad5898</guid>
            <category><![CDATA[security]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Narendiran Krishnan]]></dc:creator>
            <pubDate>Wed, 10 Feb 2021 11:05:41 GMT</pubDate>
            <atom:updated>2021-02-11T18:48:41.579Z</atom:updated>
            <content:encoded><![CDATA[<h4>Here you will learn how certificate binding is done for web apps using openssl</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GQhHVFsVxuckvBoMSC-DAQ.png" /><figcaption>Some random dude on the Internet</figcaption></figure><p>Hey everyone here, I will give you all a heads up with the basic setup of a <strong>Node.js app</strong> with <strong>SSL certificate</strong>. We will be using the <a href="https://en.wikipedia.org/wiki/OpenSSL"><strong>OpenSSL</strong></a> for this and along with <strong>Express</strong> and <strong>Nodemon</strong> for this app.</p><p>You all must be aware or else must have heard about the term “<a href="https://en.wikipedia.org/wiki/HTTPS">HTTPS</a>”, yes this is something which validates a website whether that is safe or secure to visit. But have you ever wondered what makes it safe and secure ?? Well, here I will guide you through how to making your website more secure and safe. This will just give you an idea on what exactly it is and using this OpenSSL kindly don’t proceed to the production environment.</p><p>Before looking deep into the example, let’s look into some terminologies so every time we don’t have to Google search something.</p><ul><li><strong>SSL</strong> → Secure Socket Layer, this is a predecessor of TLS (Transport Layer Security), which is nothing more than a cryptographic protocol designed for secured communication over the network.</li><li><strong>HTTPS</strong> → Hypertext Transfer Protocol Secure is an application that is an extension or a hybrid version of HTTP, i.e. it is a combination of both “HTTP” and “SSL / TLS” so the birth of a new protocol has been named as HTTPS.</li></ul><p>Alright, less learning and more implementation guys, let’s now take a deep dive into what we have actually planned to do.</p><p>Here’s the prerequisite that you will need to use for this:</p><ul><li><a href="https://nodejs.org/en/download/">Node</a>.js</li><li><a href="https://www.openssl.org/source/">OpenSSL</a> (Not to worry about this if your a Linux and Mac user)</li><li>Node <a href="https://www.npmjs.com/package/express">Express</a></li><li><a href="https://www.npmjs.com/package/nodemon">Nodemon</a></li><li><a href="https://git-scm.com/downloads">Git</a> (just for cloning the git repo)</li></ul><p>Well is that, I have implemented this on a AWS EC2 instance ( I have done this on a free tier EC2 Instance) you can also do the same thing for free of cost.</p><p>Well, in order to get the certificate, there are just 3 steps to be followed and it is as follows:</p><ul><li>Generate a private key</li><li>Create a CSR (certificate signing request) using private key</li><li>Generate the SSL certification from CSR.</li></ul><p>Yeah, it’s just as simple as it looks.</p><p>For the best practice we will create a folder and then put everything in that folder, for instance once you git clone the project into your Local Machine or else your EC2 instance it should look as shown below,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/461/0*dWLj6Lf5mFVwYVA1" /><figcaption><strong>Credit goes to Narendiran Krishnan itself</strong></figcaption></figure><p>Well, I request you all don’t try to use my <strong>.pem</strong> files and moreover, I have removed all the files inside the cert folder and <strong>node_modules</strong> it won’t be there since all the dependencies will be removed and we will create from scratch here and you can use this structure as a reference.</p><p>If you didn’t clone my git repo then just make the directory as follows:</p><ul><li>ssl_</li><li>ssl_ → cert ( inside ssl_)</li></ul><p>Let’s just focus on creating the certificate and key and then move on to node.</p><h4>Step 1:</h4><p>Move to cert folder and then you need to give the following commands:</p><pre><strong>Openssl {generate RSA} {output} {output file name with .pem extension}</strong></pre><p>Code is:</p><pre><strong>openssl genrsa -out key.pem</strong></pre><p>Once you give this command, you should see the following output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/0*ZJHSszrwTrFl9Lly" /><figcaption><strong>Generate your RSA</strong></figcaption></figure><h4><strong>Step 2</strong></h4><p>Look the code structure below after <strong>step 1:</strong></p><pre><strong>Openssl {new CSR sign in request} -key {that we have just now generated} {keyname} -out {output the certificate signing request} name {name with .pem extension}</strong></pre><p>Now the next command is as follows:</p><pre><strong>openssl req -new -key key.pem -out csr.pem</strong></pre><p>Once you give this command, you should see the following questions asked and kindly give your details and not mine:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/0*5fMC9fjIwHP-qoel" /><figcaption><strong>generating csr.pem</strong></figcaption></figure><h4>Step 3:</h4><p>Once you’re done with <strong>step 2</strong>, all that is left is to create an SSL certificate from CSR which was created using the private key.</p><p>Before we jump into the code we need to understand the public key standards,</p><p><em>X509 → which defines the format of the public key certificates, X.509 certificates are used in many Internet protocols, including TLS/SSL, which is the basis for HTTPS, the secure protocol for browsing the web. They are also used in offline applications, like electronic signatures.</em></p><p>Now let’s look into the code structure:</p><pre><strong>Openssl {public key standards} {request} {number of days certificate will be valid} {pass in certificate signing request file} csr.pem (in our case) {signkey} {key which we generated in step 1} key.pem (in our case) {output} {name of the file with .pem extension}</strong></pre><p>Code looks as follows,</p><pre><strong>openssl x509 -req -days 365 -in csr.pem -signkey key.pem -out cert.pem</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*aTx9SAdSr99nZeVg" /><figcaption><strong>creating an SSL certificate</strong></figcaption></figure><p>If you have reached so far without any error then, you have followed my blog properly.</p><p>Yes, it’s time for the victory lap now as you have climbed half the well already.</p><p>Now the rest is just to create the Node.js app and host that up in the EC2 server instance.</p><p>If you have just cloned my git repo then I have already given details about how to run the application in my git repo just follow it and don’t follow these instructions as I am doing things from scratch here.</p><p>First, we need to do <strong>npm initialisation</strong> in out ssl_ folder and give the following command and fill in the details as per your wish,</p><pre><strong>npm init</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/691/0*lyTfatF-3bDQ9sS4" /><figcaption>npm init</figcaption></figure><p>Once done now, need to install the dependencies like node express and nodemon</p><p>Code is as follows.</p><pre><strong>npm i nodemon express</strong></pre><p>Or</p><pre><strong>npm i -g nodemon express</strong></pre><p>Once you’re done with this you need to make the following changes in package.json file that has been created as shown below,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/544/0*odKHzXaeVgLbpquV" /><figcaption><strong>package.json</strong></figcaption></figure><p>Here you would have made the changes in scripts alone.</p><p>Once you are done with the dependencies I have created a server.js file as shown below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/945/0*CjqmztII-pwLrgLj" /><figcaption><strong>server.js</strong></figcaption></figure><p>Once your done with the code, start running the code in the terminal as shown below,</p><p>Code is</p><pre><strong>npm start </strong></pre><pre><strong>or </strong></pre><pre><strong>nodemon server.js </strong></pre><pre><strong>or </strong></pre><pre><strong>node server.js</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/0*Jong5hZP0bRp_MNc" /><figcaption><strong>node server.js</strong></figcaption></figure><p>Alright it does stop here, to verify it you need to visit the localhost and verify it.</p><p><strong><em>“</em></strong><a href="http://localhost:3030/"><strong><em>http://localhost:3030</em></strong></a><strong><em>”</em></strong></p><p>This will throw up the following error,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2z7P-OaB6y61HGQ_" /><figcaption><strong>error</strong></figcaption></figure><p>Now try the same with the “<a href="https://localhost:3030/">https://localhost:3030</a>” you will get the following error.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gJm7067YanngPTgP" /><figcaption><a href="https://localhost:3030/"><strong>https://localhost:3030</strong></a></figcaption></figure><p>Here you can see that Firefox has not allowed you, but in the meanwhile you can accept the risk and continue or if you’re not still satisfied with it you may view the certificate and verify it for yourself. Which will give you the details of the validity, public key info i.e. which algorithm and key size and also the fingerprints i.e. the <strong>SHA-256 &amp; SHA-1</strong> details will be given.</p><p>Once you accept all and proceed with it, you get the following output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/641/0*t7WiiJhrre9SZuuA" /><figcaption><strong>google narenltk</strong></figcaption></figure><p>Apart from this there are few places where people use SSL, i.e. the application of SSL which is where they used are as follow:</p><ul><li>Web Servers</li><li>Application servers</li><li>Databases</li></ul><p>Why should we protect the above mentioned services with SSL?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/789/1*2XxnEEpdzc_lE4eJJCTDgQ.png" /><figcaption><strong>with and without ssl comparison</strong></figcaption></figure><p>This technique as well as the code can be used in a real time production environment, but you can’t use the certificate and key though. I highly recommend something like <a href="https://aws.amazon.com/certificate-manager/"><strong><em>Amazon Certificate Manager</em></strong></a>.</p><p>If you wanna know about video streaming using opencv4nodejs have a look at the blog below:</p><p><a href="https://medium.com/swlh/video-streaming-using-opencv4nodejs-with-node-js-express-and-socket-io-3806abb049a">Video Streaming Using Opencv4nodejs With Node js, Express, and Socket IO</a></p><p>GitHub link is below:</p><p><a href="https://github.com/narenltk">narenltk - Overview</a></p><p>If you wish to stay connected:</p><p><a href="https://www.linkedin.com/in/narenltk/">Narendiran Krishnan - Coimbatore, Tamil Nadu, India | Professional Profile | LinkedIn</a></p><p><em>You can also just google “ narenltk / narendiran krishnan ” or just drop a mail to → narenltk@gmail.com → Happy to help..!!!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=73d452ad5898" width="1" height="1" alt=""><hr><p><a href="https://javascript.plainenglish.io/generate-ssl-certificate-using-node-js-for-web-apps-73d452ad5898">Generate SSL Certificates using Node.js for web apps</a> was originally published in <a href="https://javascript.plainenglish.io/">JavaScript in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Video Streaming Using Opencv4nodejs With Node js, Express, and Socket IO]]></title>
            <link>https://medium.com/swlh/video-streaming-using-opencv4nodejs-with-node-js-express-and-socket-io-3806abb049a?source=rss-dc3776c53375------2</link>
            <guid isPermaLink="false">https://medium.com/p/3806abb049a</guid>
            <category><![CDATA[ejs]]></category>
            <category><![CDATA[socketio]]></category>
            <category><![CDATA[expressjs]]></category>
            <category><![CDATA[opencv]]></category>
            <category><![CDATA[nodejs]]></category>
            <dc:creator><![CDATA[Narendiran Krishnan]]></dc:creator>
            <pubDate>Mon, 01 Feb 2021 13:31:53 GMT</pubDate>
            <atom:updated>2021-02-10T11:08:36.387Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MD1_DUWo4JDYANfAjFYhQQ.jpeg" /><figcaption>Credit → some random dude on google images</figcaption></figure><p>Hey guys, welcome to 2021 and I know it is very late to wish everyone !!!</p><p>It’s a new year and the right time to learn something new, I mean yes so let’s look into this new concept, its OpenCV. Yes it is OpenCV not with python and since it is a new year let’s learn a new language i.e. let’s learn <strong>OpenCV</strong> with <strong>Node JS</strong>. Since OpenCV has played a vital role in the Computer Vision applications let’s do the same in web applications like Face Detection, Face Land Mark Detection, Face Recognition and even Object Recognition with Deep Neural Networks etc…</p><p>Well we all know how to get a camera feed from our lappy or simply local machine. But we have all done that using OpenCV with Python and not even with C++. If you guys have done it, that’s really great.</p><p>Here in this tutorial we will get the camera feed and put that into a node app and host it up using <strong>Express Node js</strong> and <strong>Socket IO</strong>.</p><blockquote>Let’s cut the talk and get to implementation part, before diving deep into the AI part of <strong>opencv4nodejs</strong> let’s first the basic like steps to be followed,</blockquote><h4>Steps:</h4><ul><li>Import the required libraries.</li><li>Capture the Video feed from the local machine.</li><li>Get the Video feed and stream it into a web app.</li><li>Create a basic html file or create a EJS template.</li><li>Include Socket IO library in the Client side</li><li>Connect your client side to either the localhost or any network</li><li>Make client listen to that image and encode it</li><li>Now set a constant value for FramesPerSecond</li></ul><h4>Step 1:</h4><p>Here the required Libraries are</p><ul><li>Opencv4nodejs</li><li>Express</li><li>http</li><li>Socket IO</li><li>Path</li></ul><p>Import them as shown</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/491/0*7yPPSJ1O2byRMnDh" /><figcaption><strong>Library Required</strong></figcaption></figure><h4>Step 2:</h4><p>Now we need to Capture the Video feed from the local machine,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/0*ZrUk5LxQJezhD5Ma" /><figcaption><strong>Capture Video</strong></figcaption></figure><p>You guys can see doesn’t this look like python code, if you’re a true python programmer you definitely remember this and the answer is hell yes … !!!</p><p>I guess OpenCV cheated on us guys .. !!!</p><h4>Step 3:</h4><p>Get the Video feed and stream it into a web app using proper time interval i.e we are going to do a public broadcast to anyone who is listening to our server.</p><p>This is our server side by the way.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/0*Y74RrKgFe4Snl3-u" /><figcaption><strong>setInterval</strong></figcaption></figure><p>Here we have obtained the video capture and encoded the image. We are encoding because all the images that have been captured are always read in the matrix format and that is the main reason we need to encode them to base 64 and that string is then broadcasted with the almighty Socket IO.</p><p>Looks like all the setup needed for the server side has been done. It is time to move to the client side.</p><h4>Step 4:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/269/0*9G4eJwNJ3c-m-Vw9" /><figcaption><strong>Folder Structure</strong></figcaption></figure><p>For the client side instead of creating a normal HTML file I have created a EJS template and if you’re going to do a git repo kindly follow my <a href="https://github.com/narenltk">GitHub</a>, where I have given the required details of how to use it if you are doing a git clone.</p><h4>Step 5:</h4><p>Now include Socket IO library in the Client side <strong><em>i.e. use the </em></strong><a href="https://cdnjs.com/libraries/socket.io"><strong><em>Link</em></strong></a></p><p>I will let you guys do the research about the Content Delivery Network Node JS (cdnjs) of Socket IO i.e whichever version you want you may use and rather than using what I have used.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/888/0*Uk_hVPLwSA18FSQj" /><figcaption><strong>index.ejs</strong></figcaption></figure><h4>Step 6:</h4><p>Connect your client side to either the localhost with assigned port or else make it listen to any network as I have done.</p><blockquote><strong>You can refer to the image given above for this.</strong></blockquote><h4>Step 7:</h4><p>Now need our client to listen to that image and encode it as shown,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/931/0*JfP-jlYYuXFV9CqK" /><figcaption><strong>socket script in index.ejs</strong></figcaption></figure><h4>Step 8:</h4><p>I guess we are all done. Now all we need to do is set a constant value for FramesPerSecond and you can also change it and have fun. This is on the server side.</p><blockquote><strong><em>Note →</em></strong></blockquote><blockquote><strong>Try to make the FramesPerSecond = 10 and change it to 50 then make it 100. You will enjoy the output.</strong></blockquote><p>In this project if your start the server itself the camera will start to run and you don’t need to do transform it will give you a normal camera feed i.e. not a mirror image feed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/969/0*FUwy965n9CK1NcMm" /><figcaption><strong>server.js</strong></figcaption></figure><p>Have fun with <strong>OpenCV4nodejs</strong>…</p><p>If you wish to host it using ngrok I have given the details in the <a href="https://github.com/narenltk"><strong><em>GitHub</em></strong></a><strong><em>.</em></strong></p><p>If you wish to add ssl certificate to your website have a look at link below</p><p><a href="https://narenltk.medium.com/generate-ssl-certificate-using-node-js-for-web-apps-73d452ad5898">Generate SSL Certificate using Node JS for web apps</a></p><p><strong><em>Code Details, take a look at my GitHub</em></strong></p><p><a href="https://github.com/narenltk">narenltk - Overview</a></p><p>Ref: <a href="https://www.npmjs.com/package/opencv4nodejs">https://www.npmjs.com/package/opencv4nodejs</a></p><blockquote><strong>If you wish to stay connected,</strong></blockquote><p><a href="https://www.linkedin.com/in/narenltk/">Narendiran Krishnan - Coimbatore, Tamil Nadu, India | Professional Profile | LinkedIn</a></p><blockquote><strong>you can also just google “ narenltk / narendiran krishnan ” or just drop a mail to → narenltk@gmail.com → Happy to help..!!!</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3806abb049a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/video-streaming-using-opencv4nodejs-with-node-js-express-and-socket-io-3806abb049a">Video Streaming Using Opencv4nodejs With Node js, Express, and Socket IO</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[webRTC → Architectural overview: Peer-To-Peer in the Browser]]></title>
            <link>https://narenltk.medium.com/webrtc-architectural-overview-peer-to-peer-in-the-browser-6f0dee9d5ddf?source=rss-dc3776c53375------2</link>
            <guid isPermaLink="false">https://medium.com/p/6f0dee9d5ddf</guid>
            <category><![CDATA[peer-to-peer]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[streaming]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[webrtc]]></category>
            <dc:creator><![CDATA[Narendiran Krishnan]]></dc:creator>
            <pubDate>Sun, 20 Dec 2020 15:42:06 GMT</pubDate>
            <atom:updated>2020-12-30T10:11:36.879Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*Lj1HiHRk7o6rtBauUPCnXw.png" /><figcaption><strong>Credits to someone on the Google Images</strong></figcaption></figure><p><strong>Alright guys it is already the end of 2020 and our beloved Corona’s celebrating its 1st year anniversary and which has made us take the longest summer vacation in the history of mankind I guess.</strong></p><blockquote>Yuppp.. summer is over and “winter is coming” be prepared..!!! This has made us, use the two way live streaming a lot recently. We all have frequently started to use applications like <strong>Zoom calls, Google Meet, GoToMeeting and Discord</strong> <strong>etc.. (End of Thinking Capacity)</strong>. Yeah we have used these guys a lot and are still gonna use a lot.</blockquote><p>But have you ever wondered what really goes on behind this, i.e. how exactly this is working. What we should exactly do is to understand that we should open up our Google Browser and say Thanks Google, yuppp..</p><p>Google is responsible for the OpenSource Project, i.e. the Peer to Peer connection. Once again it is all thanks to “<strong>almighty Uncle Google</strong>.”</p><p>Before we go in detail, let’s see who all have actually made use of this OpenSource Project,</p><ul><li>Google Meet / Hangouts, yeah they are supposed to use their product.</li><li>GoToMeeting</li><li>Discord</li><li>Amazon Chime</li><li>Facebook → Messenger, Facebook Live</li><li>Zoom (Yeah even these guys are using it, have a look at the tweet by the official creator of webRTC)</li></ul><h3></h3><p>Looks like @zoom_us has switched it&#39;s web client from web sockets to #WebRTC data channels. Performance a lot better compared to their old web client. pic.twitter.com/SQhP9XhHXP</p><h4><strong>But you may also ask a question: is YouTube using webRTC ?</strong></h4><p>It’s a hypothetical question since they are only using it for <strong>VP9 </strong>video codec, i.e. Video Processor version 9 and currently everyone else is using <strong>VP8 </strong>and our beloved <strong>Uncle Google</strong> alone is testing out things with <strong>YouTube </strong>with the <strong>VP9</strong>. When it comes to Google something fishy always goes on.</p><p>Alright it’s <a href="https://webrtc.org/"><strong>webRTC</strong></a><strong> </strong>which has been used by all these guys.</p><h4><strong>What is webRTC?</strong></h4><p>“webRTC” stands for <strong>Web Real Time Communication</strong>, which is plugin free and a networking technology introduced <strong>by Google → for the Googlers and rest of the world</strong> for real-time audio, video, and data transmission across the web and native browsers.</p><p>In webRTC there’s no need for servers to be involved in transferring real time data from one client to another client. Here the connections are peer to peer and so sending data is very simple with the help of webRTC API which is available for all the platforms without the use 3rd party app and support almost all the platforms.</p><p>Initially the servers are required for the client computers to get connected and once they’re connected they can communicate with each other without the server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/0*TQJR43OiGbE4K1lS" /><figcaption><strong>webRTC Overview</strong></figcaption></figure><blockquote><strong>Note: Session Description Protocol (SDP):</strong></blockquote><blockquote><strong>SDP is a standard for describing the multimedia content of the connection such as resolution, formats, codecs, encryption, etc. so that both peers can understand each other once the data is transferring. This is, in essence, the metadata describing the content and not the media content itself.</strong></blockquote><blockquote><strong>Technically, then, SDP is not truly a protocol, but a data format used to describe connections that shares media between devices.</strong></blockquote><p>To understand this webRTC in simple way, let’s consider a scenario where,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/426/0*StnNDd7nLaNiC0Ug" /><figcaption><strong>scenario</strong></figcaption></figure><p>Here for establishing a connection the client 1 has to create a offer i.e. <strong>SDP </strong>(<strong>Session Description Protocol</strong>) which is the media configuration of the client 1, i.e. with which he/she can send the Video / Audio / Both.</p><p>1st the <strong>C1 </strong>will send an offer (<strong>SDP</strong>) to the server / signaling server and that will forward it to <strong>c2 </strong>and <strong>c2 </strong>will store it in its local memory. Once this is done it has to offer the media configuration of <strong>c2 </strong>to <strong>c1 </strong>in the same way back.</p><p>Now these <strong>C1 </strong>and <strong>C2</strong> has the details of each others “media configuration” and for the connection to be established between them, they are supposed to share their own Network Information i.e. Network Configuration.</p><blockquote><strong>Here just go with the flow as I haven’t given details about STUN and TURN Servers and will make another Blog on those stuffs.</strong></blockquote><h4><strong>What is Network Configuration ?</strong></h4><p>Network configuration is the process of assigning network settings, policies, flows, and controls.</p><blockquote><strong>Note: ICE→ Interactive Connectivity Establishment</strong></blockquote><p>Once the <strong>C1 </strong>connects with a Signalling server and an offer (<strong>SDP</strong>) has been shared, something known as the <strong>ICE Candidates</strong> will come from the webRTC i.e. which is generated from the <strong>STUN </strong>and <strong>TURN </strong>servers.</p><p>In order to get the <strong>ICE Candidate</strong> from the STUN and TURN servers for the <strong>C1</strong>, all it needs to do is provide the URL of STUN and TURN to webRTC API i.e.</p><p>Once the <strong>C1 </strong>creates the offer (SDP) with a signaling server, the webRTC will start to get ICE Candidates from the STUN and TURN servers.</p><p>Now the job for this <strong>c1 </strong>is to send the Network Configuration to <strong>c2 </strong>and this happens the same way.</p><p>This is the reason why a server is needed for the 1st time connection establishment. Since both <strong>c1 </strong>and <strong>c2 </strong>now have each other Network Configuration they establish a “<strong>Peer — Peer Connection</strong>”.</p><h4><strong>API of webRTC:</strong></h4><ul><li>getUserMedia</li><li>RTCPeerConnection</li><li>RTCDataChannel</li></ul><p>Out of the 3 webRTC API, <strong>getUserMedia()</strong> is the one that allows the browser to capture streaming voice and video using JavaScript Commands.</p><p>In simple terms, <strong>getUserMedia()</strong> is the <strong>1st Link in the chain</strong>.</p><p>Before RTCPeerConnection can route a browser’s real time data to another browser, users streaming data has to be captured by getUserMedia.</p><p>With the getUserMedia, the browser is able to capture data from the webcam &amp; microphone and it’s turned into basic JavaScript Object.</p><p>In simple terms the getUserMedia, RTCPeerConnection and RTCDataChannel are HTML 5 API that gives JavaScript enhanced media capabilities.</p><h4><strong>Internet Engineering Task Force (IETF):</strong></h4><p>This Internet Engineering Task Force is the <strong>open standards organization</strong>, which develops and promotes voluntary Internet Standards, in particular the standards that comprise the Internet protocol suite (TCP/IP).</p><p>The IETF, decides what standard the webRTC is going to follow, whether vp8 / vp9 or H.264/H.265.</p><p>Both the audio codec ( Opus ) and video codec ( VP8 / VP9 ) are officially standardized by IETF.</p><h3><strong>Audio and Video Codec</strong>:</h3><h4><strong>Codec:</strong></h4><p>A codec is a standard or tool for encoding and decoding multimedia files.</p><p>When comparing VP8 and x264, VP8 also shows 5–25 times lower encoding speed with 20–30% lower quality on average. For example x264 High-Speed preset is faster and has higher quality than any of VP8 presets on average.</p><p>Opus is a lossy audio codec that has some significant advantages over other lossy codecs such as MP3 or AAC. First, Opus is an open standard, and as such is royalty-free. Another advantage of Opus is its remarkable audio quality, especially at low bitrates. Moreover, this quality is achieved at very low latencies, which makes Opus a logical choice for interactive music and speech transmission. However, it is also very well suited for storage and streaming applications.</p><blockquote><strong>“Sharing some facts about the Audio and Video Bitrates”</strong></blockquote><h4><strong>Video Bitrates:</strong></h4><ul><li>VP8 with Chrome — it will take up 1.77 Mbps when the camera source is VGA.</li><li>VP8 has no limit on frame rate or data rate.</li><li>The WebRTC video codec is powered by libvpx, which is the only software library that can encode VP8 video streams.</li></ul><h4><strong>Audio Bitrates:</strong></h4><ul><li>Opus isn’t in wide use yet, but it’s considered a next-generation codec.</li><li>Opus provides 26.5 ms latency, when using its default settings (20 ms frame size), making it highly suitable for Voice over IP (VoIP) communications.</li><li>Opus has a variable bitrate (VBR) mode that can range from 6 kbit/s to 510 kbit/s to help minimize bandwidth while maintaining consistent quality for applications that require that.</li></ul><p>I have given the detail of the STUN and TURN Servers in another Blog and working on that as of now. Before leaving you all just like that have a look at the comparison based on my understanding.</p><h4><strong>Comparison of STUN &amp; TURN</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/919/1*B_sQuM2OFj9BNTZCxn09BA.png" /><figcaption>Comparison based on my research</figcaption></figure><blockquote><strong>If you wish to stay connected,</strong></blockquote><ul><li><a href="https://www.linkedin.com/in/narenltk/">Narendiran Krishnan - Coimbatore, Tamil Nadu, India | Professional Profile | LinkedIn</a></li><li><a href="https://github.com/narenltk">narenltk - Overview</a></li></ul><blockquote><strong>you can just google “ narenltk / narendiran krishnan ” or just drop a mail to → narenltk@gmail.com → Happy to help..!!!</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6f0dee9d5ddf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TensorFlow’s Object Detection API Using Google Collab.]]></title>
            <link>https://medium.com/swlh/tensorflows-object-detection-api-using-google-collab-cb92d7f7b3cf?source=rss-dc3776c53375------2</link>
            <guid isPermaLink="false">https://medium.com/p/cb92d7f7b3cf</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[opencv]]></category>
            <category><![CDATA[python3]]></category>
            <category><![CDATA[covid19]]></category>
            <category><![CDATA[tensorflow]]></category>
            <dc:creator><![CDATA[Narendiran Krishnan]]></dc:creator>
            <pubDate>Sat, 08 Aug 2020 14:34:41 GMT</pubDate>
            <atom:updated>2020-12-16T03:59:17.490Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*-Hx9AfkE7-htkv0E_jzC9Q.jpeg" /></figure><p>Here I will walk you through the steps to create your own Custom Object Detector with the help of Google’s TensorFlow Object Detection API using Python 3 not on your CPU. It is with the free Google Cloud which provides you around 12 Hrs of free GPU for training your model.</p><p>You can find the code to the entire code here in my GIT HUB repo.</p><p><a href="https://github.com/narenltk/Tensorflow-Object-Detection-API-Google_Colab">narenltk/Tensorflow-Object-Detection-API-Google_Colab</a></p><p>Before we proceed, I recommend you to kindly read my previous Blog which is also TensorFlow’s Object Detection API, where I trained using CPU.</p><p><a href="https://medium.com/@narenltk/train-your-own-dataset-for-object-detector-with-tensorflows-object-detector-api-53f74467f6d4">Train your own dataset for Object Detector with TensorFlow’s Object Detector API</a></p><p>I recommended you guys to read this because we are going to follow the same steps here and there is going to be a small difference, where we are just going to train the model here in the Google Collab which is free for 12 Hrs per day.</p><p>Steps to be followed to implement this project:</p><ol><li>Setting up Object Detection Directory</li><li>Configure path variables</li><li>Training Data</li><li>Updating Google Drive</li><li>Setting up Google Collab</li><li>Configuring Training</li><li>Training your model</li><li>Inference Graph</li><li>Testing our model</li></ol><blockquote><strong>I have refereed multiple tutorials and blogs and worked on this and special thanks for the author of pythonprogramming.net, Edje Electronics, </strong><a href="https://medium.com/u/acb7414d6046"><strong>Nathaniel O Solomon</strong></a><strong> and </strong><a href="https://medium.com/u/4ff6d2f67626"><strong>Dat Tran</strong></a><strong>.</strong></blockquote><p>Here the first 3 steps are going to be the same, there is not going to be any changes i.e. this is not entirely going to be done in a cloud environment and it is more of a combination of both.</p><p>Before setting up the Google Drive and Collab, I have to say here I have used a different Data Set and Since it is still Covid-19 situation and so I have decided to use Masked Dataset.</p><p><a href="https://github.com/prajnasb/observations">prajnasb/observations</a></p><blockquote><strong>Here in his git repo go to “ /experiements/data/ “ you will find the details of the mask. Well he has done some more of the complex data augmentations work in “ mask_classifier/Data_Generator/ “ you guys can check that too.</strong></blockquote><p>I have got only 2 classes for this mask detection and you need to update those details in the first level in “ <strong>labelmap.pbtxt</strong> “ as well as in “ <strong>faster_rcnn_inception_v2_pets.config</strong>” and “ <strong>generate_tfrecord.py </strong>“</p><p>You can also refer to my previous blog where I have given the details of how it has to be done.</p><p>However I will also show you how to do it here too.</p><p>Now go to the folder Training under as shown below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/0*X2pZ6TEyttml_rZS" /><figcaption><strong>folder you need to access</strong></figcaption></figure><p>In these both you have to make changes with the class name, i.e. with mask and no mask.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/333/0*RjLu804xcjQNY1J7" /><figcaption><strong>labelmap.pbtxt file</strong></figcaption></figure><p>This is the changes that is supposed to be done in “ <strong>labelmap.pbtxt</strong> “</p><p>In the “ <strong>faster_rcnn_inception_v2_pets.config</strong> “ file, we need to do many changes according to Google Drive.</p><p>The first change is, line number 9 where you need to change the number of classes to 2,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/529/0*wjtRtHzzwgcoO1zV" /><figcaption><strong>change the number of classes line 9</strong></figcaption></figure><p>Then scroll down to line 126 and you need to do the following changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/0*DkWX22HoGYxhjI4o" /><figcaption><strong>start changing from line 126</strong></figcaption></figure><p>You have to make sure that you give the path exactly the same way I have given here. Moreover I should note that line 132.</p><p><strong>Num_examples </strong>→ this is nothing but the total images which you have used for testing.</p><p>Now you need to search for “ <strong>generate_tfrecord.py</strong> “ which will be in the Object Detection folder.</p><p>Here are a few extra lines of codes that you’re supposed to change since, most of them in my previous Blog are kinda deprecated, that doesn’t mean that it will not run. It will give you warning error.</p><p>The first thing is to update the number of classes, as shown from line 32.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/0*HZptr7N7I99pQ_8E" /><figcaption><strong>with mask and no mask</strong></figcaption></figure><p>Now these steps are to be followed only if your using my other Git Repo, if not you can skip this step.</p><p>Now these steps are to be followed only if your using my other Git repo, if not you can skip this step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/931/0*NPsQ_i_xncyU9_SB" /><figcaption><strong>tf.io.gfile.GFile</strong></figcaption></figure><p>The code is “ <strong>tf.io.gfile.GFile</strong> “ → this is the new code which ain’t deprecated yet.</p><p>Next is line 88 where you can see the difference in the code it is nothing big, but still a difference is a difference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/673/0*GfaOgrytMXMM19bc" /><figcaption><strong>line 88</strong></figcaption></figure><p>Last but not least the last line,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/406/0*wKrg3SHwQYxyV_5I" /><figcaption><strong>Last line of the program</strong></figcaption></figure><p>Now once you’re done with these changes your good to go.</p><h4>Updating Google Drive</h4><p>Now you need to save all the changes that you made to the code and now your going to create a new folder in your google drive as tensorflow1</p><p>Then Import all the files and folder inside tensorflow1 in your Local Machine to your Google Drive, I guess it will be around 350 Mb once your done uploading into Drive.</p><p>Once you done it you will be having all the files and folder as shown below,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/810/0*ThBfIiEtfNS7wfT0" /><figcaption>Google Drive</figcaption></figure><p>I guess we are done updating the datas required in Google Drive,</p><p>Now you need to select new and follow the 1, 2 and 3 as follows,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/965/0*1TknDJmR45BUeMJI" /><figcaption><strong>Create Google Colaboratory Notebook</strong></figcaption></figure><p>Now you will be redirected to your Google Colab Notebook, which is pretty much 100% similar to jupyter notebook all the shortcuts are all the same.</p><p>Now type the code as given below,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/0*6H2knj1vp0yV6uFe" /><figcaption><strong>Mount Your Google Drive</strong></figcaption></figure><blockquote><strong>from google.colab import drive</strong></blockquote><blockquote><strong>drive.mount (‘/content/gdrive’)</strong></blockquote><p>Now if you run this cell you will find a URL given below you need to select that and allow that for accessing your Google Drive you need to give some auth key just copy and paste that auth code and you’re done.</p><p>Now you can access any files from your Google Drive.</p><p>Now we need to select the <strong>Version 1.x</strong> in <strong>TensorFlow </strong>as this project hasn’t been updated to <strong>TensorFlow 2.x yet</strong>.</p><p>So get along and follow the steps as shown below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/617/0*8W4Pn_mYLPOrcbnG" /><figcaption><strong>Check TensorFlow Version</strong></figcaption></figure><p>Code is as given below and you use this technique to select any specific version you want.</p><blockquote><strong>%tensorflow_version 1.x</strong></blockquote><h4><strong>Setting up Google Collab</strong></h4><p>First move to the tensorflow1 location in you gdrive,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/0*xVEO0SN_vOzNg3xN" /><figcaption><strong>Setting up Google Collab</strong></figcaption></figure><p>Once you’re done with that you need to install a protobuf-compiler but whatever you’re supposed to always add a “ <strong>!</strong> “.</p><p>Now we need to compile the model definition. So we need to change the directory to the research folder and run the following code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/0*CMAm174V_0WdbE0N" /><figcaption><strong>change directory</strong></figcaption></figure><blockquote><strong>!protoc object_detection/protos/*.proto — python_out=.</strong></blockquote><h4>Set the environment variables</h4><blockquote><strong>import os</strong></blockquote><blockquote><strong>os.environ[‘PYTHONPATH’] += ‘:/content/gdrive/My Drive/tensorflow1/models/research/:/content/gdrive/My Drive/tensorflow1/models/research/slim’</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*m3ayPmp-8YEbq80w" /><figcaption><strong>Google Colab environment setup</strong></figcaption></figure><p>For a session restart do the following</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/0*pfO9ezDnCtubGPAX" /><figcaption><strong>session restart</strong></figcaption></figure><p>Always add a “ ! “ before. Don’t forget this.</p><p>Now in our Google Drive we need to change the directory to object detection directory that contains your train and test images with a respective xml file of each image. We need to create a TensorFlow record file from the xml file we have.</p><p>Use the following code to change directory,</p><blockquote><strong>%cd /content/gdrive/My Drive/tensorflow1/models/research/object_detection</strong></blockquote><p>For creating a training record, we are using the following code,</p><blockquote><strong>!python generate_tfrecord.py — csv_input=images/train_labels.csv — image_dir=images/train — output_path=train.record</strong></blockquote><p>Now for creating a testing record, we are using the following code,</p><blockquote><strong>!python generate_tfrecord.py — csv_input=images/test_labels.csv — image_dir=images/test — output_path=test.record</strong></blockquote><p>Since all the work that is supposed to be done for training are all done.</p><p>Now all that is left is only training the model.</p><h4>Training your model</h4><p>Now make sure that you are in the object_detection directory and then you can execute the following code,</p><blockquote><strong>!python train.py — logtostderr — train_dir=training/ — pipeline_config_path=training/faster_rcnn_inception_v2_pets.config</strong></blockquote><p>Well I have trained the model up to <strong>64335 </strong>steps,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/0*xIxcRzTURiCtKqy9" /><figcaption><strong>Training Output</strong></figcaption></figure><p>Since it is a 12 Hrs free GPU, you can proceed.</p><p>Next step is similar to my previous blog. which is <strong>step 8 Inference Graph</strong></p><blockquote><strong>You can follow this step if your testing in Google Colab.</strong></blockquote><h4>Testing our model</h4><p>Once you have the frozen graph, you need to open the “<strong>Object_detection_image.py</strong>” file in the object detection folder.</p><p>Copy the entire code and paste it in a new cell in your Colab. Once your done with that run with a image.</p><p>you will have an output as shown below,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*VTBm5yzYbdznPMHBVEGaTw.jpeg" /><figcaption><strong>Finally Output.</strong></figcaption></figure><p>If you guys want to work with TensorFlow 2.x in your Local Machine then you can follow the blog given below and they have also added few information which I haven’t covered here like,</p><ul><li>how to use LionBridge for data annotation</li><li>script for converting the XML / JSON another Open Source contribution</li><li>how to update the Batch size and training steps</li><li>how to update the loss function that is used for classification</li><li>along with how to train your model if you have a GPU</li></ul><p><a href="https://bit.ly/3naBnQG">How to Train Your Own Object Detector Using TensorFlow Object Detection API - neptune.ai</a></p><blockquote><strong>If you wish to stay connected,</strong></blockquote><p><a href="https://www.linkedin.com/in/narenltk/">Narendiran Krishnan - Coimbatore, Tamil Nadu, India | Professional Profile | LinkedIn</a></p><blockquote><strong>you can just google “ narenltk / narendiran krishnan ” or just drop a mail to → narenltk@gmail.com → Happy to help..!!!</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cb92d7f7b3cf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/tensorflows-object-detection-api-using-google-collab-cb92d7f7b3cf">TensorFlow’s Object Detection API Using Google Collab.</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[VGG16 → from scratch using Transfer Learning with Keras and TensorFlow]]></title>
            <link>https://narenltk.medium.com/vgg16-from-scratch-using-transfer-learning-with-keras-and-tensorflow-ffee5ae460fb?source=rss-dc3776c53375------2</link>
            <guid isPermaLink="false">https://medium.com/p/ffee5ae460fb</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[transfer-learning]]></category>
            <category><![CDATA[python3]]></category>
            <category><![CDATA[keras]]></category>
            <dc:creator><![CDATA[Narendiran Krishnan]]></dc:creator>
            <pubDate>Thu, 30 Jul 2020 10:25:51 GMT</pubDate>
            <atom:updated>2020-12-21T16:48:11.647Z</atom:updated>
            <content:encoded><![CDATA[<h3>VGG16 → from scratch using Transfer Learning with Keras and TensorFlow 2</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*HzxRI1qHXjiVXla-_NiMBA.jpeg" /><figcaption><strong>VGG16 Model</strong></figcaption></figure><p>If we are gonna build a computer vision application, i.e. for example, let’s take an example like Image Classification, we could use Transfer Learning instead of training from the scratch. By this way we often make faster progress in training the model since we are just making use of someone else’s trained model and we can use that to do new tasks for us.</p><p>The Computer Vision research community has been posting a lot of data sets on the internet like <strong>Image Net</strong> or <strong>MS COCO</strong> or <strong>PASCAL </strong>types of data sets. Well, these are some of the data sets that are famous across the globe as of now on which people have done enough research and are being used practiced in day to day life with real time solutions.</p><p>In Transfer Learning the trick is very simple: we don’t train all the layers of the model. We just freeze all the layers and just train the lower layers of the model, i.e. making use of the trained model weights and so this makes retraining very simple.</p><p>Well, here in this example I have actually done a Dog and Cat classification using VGG-16 and used ImageNet in this example.</p><p>Before starting the example, let me tell you about ImageNet.</p><h4>What is ImageNet ?</h4><p>ImageNet is a large image dataset which has been organized according to the WordNet hierarchy. They actually have around 14,197,122 images totally dated today, i.e. 14 Million images. There are actually around 100,000 synsets in WordNet, where each synset has around 1000 images just for illustration.</p><blockquote><strong>This is actually designed by people with really good IQ. Well that’s just an indirect way to say proudly that “NOT ME…!!!”</strong></blockquote><p>Moreover, every single image that is actually used in ImageNet is manually annotated by us humans. We can be proud of that since bots ain’t doing this job.</p><p>Well, now the Cat and Dog dataset actually has around 1000 → cat and dog training images and for validation it has around 500 → cat and dog images.</p><blockquote><strong><em>Link for</em> </strong><a href="https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip"><strong>dataset</strong></a></blockquote><p>Once you have downloaded all the dataset you can proceed with the implementation. Here I have built from scratch, according to the VGG-16.</p><p>We will be using TensorFlow and Keras.</p><h4>Points to be noted here,</h4><blockquote><strong>If you’re using TensorFlow Version 2.x then there ain’t any changes with the code.</strong></blockquote><blockquote><strong>If you’re deploying it in your Local Machine make sure you have enough RAM.</strong></blockquote><blockquote><strong>If you’re using cloud, then you need not worry. Since I’ve deployed it in Google Colab.</strong></blockquote><p>All the links which you need to implement this I have uploaded in my GitHub</p><p><a href="https://github.com/narenltk">narenltk - Overview</a></p><p>First thing is importing the dataset, if you’ve downloaded the dataset manually and then importing the data set you can use OpenCV with Glob.</p><blockquote><strong><em>If you want to use your Local Machine then I recommend you guys to create a Virtual Environment and get the work done. It helps in solving a lots of problem and if your don’t know or couldn’t remember how to do it you can refer the link below.</em></strong></blockquote><p><a href="https://medium.com/@narenltk/why-do-we-need-a-virtual-environment-ec5f4cbb9dc0">Why do we need a virtual environment ?</a></p><p>Here I have downloaded and unzipped the following way and it is applicable in both Linux and Windows Machine. I have been working with Windows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/0*_PGJc78bay3K-OJJ" /><figcaption><strong>Dataset</strong></figcaption></figure><p>Here most of the coding will be based on only Keras which uses TensorFlow at the back-end.</p><p>Once you’re done with this you need to use the Training Data and Validation Data for which I am going to use ImageDataGenerator.</p><p><strong>Keras </strong>the <strong>Deep Learning</strong> Library by default provides us a way to fit models using Image Data Augmentation with ImageDataGenerator class.</p><p>ImageDataGenerator actually expands the training dataset so the performance improves.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/949/0*zt5iHvNNNlvHmt0N" /><figcaption><strong>ImageDataGenerator</strong></figcaption></figure><p>Here I have given the target_size as <strong>224</strong>, this is because the input to the ConvNets is a fixed size which is <strong>224*224 RGB</strong> image as stated in the VGG 16 architecture paper.</p><p>You can also have a look at the details given in the documentation page of Keras.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/0*Xt-8vWJXjvmeVJmz" /><figcaption><strong>ImageDataGenerator</strong></figcaption></figure><p><a href="https://keras.io/api/preprocessing/image/">Keras documentation: Image data preprocessing</a></p><p>There are 2 ways to my knowledge for implementing the VGG-16.</p><ol><li>You can either write code from scratch with the help of Keras.</li></ol><p>2. You can just import the VGG-16 function from Keras</p><p><strong>Keras supports you in all ways.</strong></p><p>Here in this example I have done with method 1 as stated above. However, I will show the easy way first.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/0*FTZrRdq8olHwaJE5" /><figcaption><strong>Simple Way</strong></figcaption></figure><p>Now in order to show you how to create from scratch is done as shown below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pX1XR6sqFM-e6spGpwU0wA.jpeg" /><figcaption><strong>VGG 16 model</strong></figcaption></figure><p>Here you can do a summary of the model and verify it.</p><p>If you’re the <strong>vgg16 </strong>by importing <strong>keras</strong> then you need to pop up the last layer which is the final Fully Connected layer.</p><p>To find the Summary of the model you need to give the model name i,e, model.summary() as shown below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/884/0*Pt349hIVYbsdkc26" /><figcaption><strong>model.summary()</strong></figcaption></figure><p>Now that we have come this far without any bugs, all we need now is a Checkpoint, Early stopping and fitting the model for training.</p><h4><strong>ModelCheckPoint</strong></h4><p><strong>ModelCheckPoint </strong>is which saves the model by monitoring the specific parameter of the model.</p><p>Here I am monitoring the validation accuracy by passing the <strong>val_loss </strong>to <strong>ModelCheckPoint</strong>. The model will be saved when the current epoch validation loss is lesser than the previous epoch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/0*K8rEw5PVtAwMPvQu" /><figcaption><strong>ModelCheckPoint</strong></figcaption></figure><h4><strong>EarlyStopping</strong></h4><p><strong>EarlyStopping </strong>will stop the training of the model earlier itself when there is no increase in the parameters which we have set for monitoring. You can define 100 epochs or 1000 epochs or even 10000 epochs, but training will stop based on this function alone.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/667/0*axiDvbSNwsFR7J1r" /><figcaption><strong>EarlyStopping</strong></figcaption></figure><p>Here we have set the monitor as val_accuracy in <strong>EarlyStopping </strong>and <strong>val_loss</strong>, you can do your research which method to use for monitoring it can be either same or else different but make sure to find out the answer yourself.</p><p>But you can also stop even before there are no changes in monitoring value, by simply adding a parameter <strong>patience = 20</strong> / <strong>patience = 10</strong>. Which stops the training once the model reaches 20 / 10 epochs.</p><h4>model.fit</h4><p>Now the final part here is <strong>model.fit_generator</strong> since we have used <strong>ImageDataGenerator </strong>class.</p><p>In the fit_generator we need to pass the <strong>training data</strong>, <strong>validation data</strong>, <strong>steps_per_epoch </strong>along with the epochs, the steps and other things you can do based on the system requirements and based on that the training time will extend.</p><blockquote>But unfortunately this <strong>fit_generator </strong>actually works with <strong>TensorFlow 1.x</strong> version and for <strong>2.x version</strong> it is different.</blockquote><p>For <strong>TensorFlow 2.x</strong> you need to simply call <strong>model.fit</strong> and not <strong>model.fit_generator</strong>. Well problem solved I guess. <strong>TensorFlow 2.x</strong> is actually small minor changes in the codes here and there.</p><p>Nothing much of a change as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*f6-YjcXsM-Kc6v7n" /><figcaption><strong>model.fit</strong></figcaption></figure><p>Once you’re done with this you have to save the weights here I have been working with Google Colab and so i have saved the model weights in google drive and the format to which it has to be saved you can mention i.e. you can save the weights in <strong>.hdf5, .h5, .pkl</strong> etc..</p><blockquote><strong>Ref:</strong></blockquote><blockquote><a href="https://keras.io/api/"><strong>https://keras.io/api/</strong></a></blockquote><blockquote><a href="http://image-net.org/about-overview"><strong>http://image-net.org/about-overview</strong></a></blockquote><blockquote><strong>If you wish to stay connected,</strong></blockquote><p><a href="https://www.linkedin.com/in/narenltk/">Narendiran Krishnan - Coimbatore, Tamil Nadu, India | Professional Profile | LinkedIn</a></p><blockquote><strong>you can also just google “ narenltk / narendiran krishnan ” or just drop a mail to → narenltk@gmail.com → Happy to help..!!!</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ffee5ae460fb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[WhatsApp Integration with Rasa Chat bot]]></title>
            <link>https://blog.chatbotslife.com/whatsapp-integration-with-rasa-chat-bot-3ff8d8df4cc3?source=rss-dc3776c53375------2</link>
            <guid isPermaLink="false">https://medium.com/p/3ff8d8df4cc3</guid>
            <category><![CDATA[bots]]></category>
            <category><![CDATA[whatsapp]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[rasa]]></category>
            <dc:creator><![CDATA[Narendiran Krishnan]]></dc:creator>
            <pubDate>Sat, 27 Jun 2020 13:28:41 GMT</pubDate>
            <atom:updated>2020-07-07T13:39:59.333Z</atom:updated>
            <content:encoded><![CDATA[<h3>Whatsapp Integration with Rasa Chat bot</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_dSYveHwcyajSzy23yYmYA.jpeg" /></figure><p><strong>RASA</strong>: An Open source conversational AI, which made our work a lot easier. Many say it doesn’t require any programming skills. It is up to you to decide whether you need programming skills or not.</p><p>Rasa mainly has 2 main modules,</p><ol><li><em>Rasa NLU → which understands messages</em></li><li><em>Rasa Core → it just decides what to do next.</em></li></ol><p>I guess like me, you must also be not interested to look into theory blah blah blah stuff, so let’s get started.</p><p>Before proceeding with this, let me tell you about the version which I have used here.</p><blockquote><strong>Python 3.7.3</strong></blockquote><blockquote><strong>TensorFlow 2.x</strong></blockquote><p>Well, if you haven’t installed Rasa for building your conversational AI then, you can check out my other blog where I have written how to</p><ol><li>Install RASA <a href="https://medium.com/@narenltk/rasa-chat-bot-installation-on-windows-10-21b4e8f16f5c">Link</a></li><li>Virtual Environment <a href="https://medium.com/@narenltk/why-do-we-need-a-virtual-environment-ec5f4cbb9dc0">Link</a></li><li>My GitHub <a href="https://github.com/narenltk">Link</a></li></ol><p>Well, once all that is done you’re good to go.</p><p>With <strong>Whats App Business API</strong>, everyone is actually trying to integrate their chat bot with whats app and am also one of those techie out there who would love to do this and succeeded finally.</p><h3>Trending Bot Articles:</h3><blockquote><a href="https://chatbotslife.com/building-a-discord-bot-for-chatops-pentesting-or-server-automation-2ac3b69603e">1. Building a Discord Bot for ChatOps, Pentesting or Server Automation</a></blockquote><blockquote><a href="https://chatbotslife.com/8-best-it-workflow-automation-practices-to-follow-for-chatbots-designed-with-workativ-assistant-5f1b2078f6dc">2. 8 Best IT Workflow Automation Practices to follow for Chatbots</a></blockquote><blockquote><a href="https://chatbotslife.com/how-to-prevent-chatbot-attacks-12531596f07f">3. How to prevent chatbot attacks?</a></blockquote><blockquote><a href="https://chatbotslife.com/adding-a-conversational-interface-to-your-app-with-dialogflow-9cfb032fce66">4. Adding a conversational interface to your app with Dialogflow</a></blockquote><blockquote><strong><em>There’s no proper documentation yet to do this work and zero help from YouTube too.</em></strong></blockquote><blockquote><strong><em>Yupppp, you can go to YouTube and check out there ain’t just one video of Rasa integration with Whats app yet and so I just got the job done and will help you now what to do.</em></strong></blockquote><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*lbONfz8Hh_mI7O96" /><figcaption><strong>Screen shot from my mobile</strong></figcaption></figure><p><strong>Step 1:</strong></p><p>First now you need to get your own trial account with Twilio for integration of Whats App.</p><p><a href="https://www.twilio.com/"><strong>Link</strong></a></p><p><strong>Step 2:</strong></p><p>Create your Project <a href="https://www.twilio.com/console/projects/create"><strong>Link</strong></a></p><p><strong>Step 3:</strong></p><p>Enable Whats app sandbox under Programmable SMS</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lqY0Wkw9Uj4fyfLu" /><figcaption><strong>Programmable SMS</strong></figcaption></figure><p>You need to send that join &lt;your twilio whats app name&gt; to that number. It will establish the connection.</p><p><strong>Step 4:</strong></p><p>Select Phone Number <a href="https://www.twilio.com/console/phone-numbers/incoming"><strong>Link</strong></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oS8LLkV-JvW8Yfs2" /><figcaption><strong>Phone Number</strong></figcaption></figure><p><strong>Step 5:</strong></p><p>Now you have to add Twilio channel to your RASA “<strong>credentials.yml</strong>”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/791/0*PjwJfH1DXIDlxkzI" /><figcaption><strong>credentials.yml</strong></figcaption></figure><p><a href="https://rasa.com/docs/rasa/user-guide/connectors/twilio/"><strong>Ref</strong></a></p><p><strong>Step 6:</strong></p><p>Now you can launch your Rasa chat bot,</p><p>Open up <strong><em>3 different terminals</em></strong> and type out the following commands,</p><blockquote><strong>Terminal / cmd 1: rasa run -m models — enable-api</strong></blockquote><blockquote><strong>Terminal / cmd 2: rasa run actions</strong></blockquote><blockquote><strong>Terminal / cmd 3: ngrok http 5005 → 5005 is your local host port number</strong></blockquote><p><a href="https://rasa.com/docs/rasa/user-guide/messaging-and-voice-channels/#testing-channels-on-your-local-machine-with-ngrok"><strong>Ref</strong></a></p><p>You terminal / cmd should look like the following,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OytiSKkKRcrY21q3" /><figcaption><strong>cmd</strong></figcaption></figure><p><strong>Step 7:</strong></p><p>Go the <strong>Step 2</strong> i.e. Phone Number which you have activated,</p><p>Under Managed Number you will find out number which is active</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_WmNhT7uqSTIHuAI" /><figcaption><strong>Managed Number</strong></figcaption></figure><p>Once you open that, you need to add your Rasa Twilio Webhook which looks like this,</p><blockquote><strong>http://&lt;YOUR ID&gt;.ngrok.io/webhooks/twilio/webhook/</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/971/0*1JmBfeUl94daiHif" /><figcaption><strong>Update Voice Webhook</strong></figcaption></figure><p>Then once again scroll down and update the same as shown below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/981/0*imy7k4hv41ZBWadJ" /><figcaption><strong>Update Messaging Webhook</strong></figcaption></figure><p><strong>Step 8:</strong></p><p>Select Programmable SMS</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FwrHVmsXAsCQUdah" /><figcaption><strong>Programmable SMS</strong></figcaption></figure><p>Then copy the ngrok link which you have hosted the <strong>http </strong>and not the <strong>https </strong>link and your supposed to update that in the Whats app beta sandbox,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*C6Ssd8gbEJb5cMea" /><figcaption><strong>Whats app beta sandbox</strong></figcaption></figure><p>Once you have update it save,</p><p>I guess you’re all good to go now, you can now chat with your own chat bot now.</p><blockquote>If you wish to stay connected, <a href="https://www.linkedin.com/in/narenltk/">LinkedIn</a> Or just drop a mail to narenltk@gmail.com, Happy to help..!!!</blockquote><h3>Don’t forget to give us your 👏 !</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/0*ZxaGcKC3UQEydQIX" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F3e59ef%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F3e59ef&amp;image=http%3A%2F%2Fapi.screenshotlayer.com%2Fapi%2Fcapture%3Faccess_key%3Dfe59908dad3baab69ffab249a2224b03%26viewport%3D1024x612%26width%3D1000%26url%3Dhttps%253A%252F%252Fupscri.be%252F3e59ef%253Fscreenshot&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href">https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href</a></iframe><figure><a href="https://chatbotslife.com/bot-communities-mastermind-group-d2dae9876709#.53x0py6ou"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*6XUspT9JOSq0w0Fi35HIaA.png" /></a></figure><figure><a href="https://m.me/ChatbotsLife"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*c1LDMH5vbnIz9rmAka8Hwg.png" /></a></figure><figure><a href="https://chatbotslife.com/chatbot-development-hire-top-ai-chatbot-developers-c8b45ef7f207"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*D0Jf3dI6ZThtqcfwDYY7mg.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3ff8d8df4cc3" width="1" height="1" alt=""><hr><p><a href="https://blog.chatbotslife.com/whatsapp-integration-with-rasa-chat-bot-3ff8d8df4cc3">WhatsApp Integration with Rasa Chat bot</a> was originally published in <a href="https://blog.chatbotslife.com/">Chatbots Life</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>