<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vishal Gorai on Medium]]></title>
        <description><![CDATA[Stories by Vishal Gorai on Medium]]></description>
        <link>https://medium.com/@greyhatlinux?source=rss-83eea1b79e2c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*9Eojo7EEIfJikcD_JD-0Rw.jpeg</url>
            <title>Stories by Vishal Gorai on Medium</title>
            <link>https://medium.com/@greyhatlinux?source=rss-83eea1b79e2c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 14:01:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@greyhatlinux/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Databases and Caches. If this, then why that?]]></title>
            <link>https://greyhatlinux.medium.com/databases-and-caches-if-this-then-why-that-e3fd7ed7dcc9?source=rss-83eea1b79e2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/e3fd7ed7dcc9</guid>
            <category><![CDATA[design-systems]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Vishal Gorai]]></dc:creator>
            <pubDate>Wed, 01 Oct 2025 18:16:04 GMT</pubDate>
            <atom:updated>2025-10-02T05:37:19.163Z</atom:updated>
            <content:encoded><![CDATA[<h3>Background</h3><p><em>Your friend wrote an exciting story about how Loki and Deadpool entered into a conflict related to a new girl in the town, but then Superman came to rescue, and finally got all the superheroes together.</em></p><p><em>Initially you kept it in your laptop, but then you wanted to share it with others.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/342/1*V8arjDolucx1ZedI4JqiVw.png" /><figcaption>File kept in your own computer</figcaption></figure><p><em>So then saved it in the institute Database with the name as ‘Story.pdf’. Now all your friends got interested in the story and wanted to read it. So they all rushed to download from the institute Database.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/342/1*3fE-aWU4Wew2GcBED9IAAQ.png" /><figcaption>File kept in the institute DB</figcaption></figure><p><em>Turns out, all your college friends loved it, and shared it with their friends, and now your Story is viral. Everybody is dying to read it, from all around the country.</em></p><p><em>But your institute database cannot handle such a huge traffic. Due to the massive outreach, now the website couldn’t download anything from the Database, not even the attendance sheet.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*ODwz_FITJcxmrzY9FmrY3Q.png" /><figcaption>DB Outage</figcaption></figure><p><em>It is a database outage.</em></p><p><em>The Dean picks you up for this issue, since it was your idea to share it to the public which has brought havoc to the institute Database.</em></p><p><em>How will you resolve this issue? Want to know more about scaling?</em></p><h4><em>I can help.</em></h4><p><em>Tip : Bonus at the end of the post.</em></p><p>Let’s begin with the basics:</p><h3>Database</h3><p>A database is a structured system for storing, managing, and retrieving large volumes of persistent data. It’s used for long-term data storage, such as user accounts, orders, or any information that must be reliably saved and queried.</p><h3><strong>Cache</strong></h3><p>A cache is a fast, temporary storage layer for frequently accessed data to reduce latency and load. It’s used to speed up data retrieval, like showing recent posts or product info without hitting the database every time.</p><h3>Let’s understand what we would be looking at in this post</h3><ol><li>How is cache different from a Database?</li><li>DB cache — What, How, Why?</li><li>Scaling Considerations</li><li>An additional cache</li><li>Trade Offs?</li></ol><h3>How is cache different from Database?</h3><p>Data, as we are aware of, must be stored in Databases, or filesystems. These serve as the primary storage location. Any request to view the object would fetch it from the DB. Data here is considered Source of Truth.</p><p>Cache is a mechanism to store frequently accessed data in a separate storage so as to reduce the latency to access the resource. I repeat, its just a strategy to retrieve data faster.</p><h3>DB cache — What, How, Why?</h3><p>Most production-grade databases include an internal caching mechanism to optimize performance, for optimising frequently queried records from the DB. These built in caches are very efficient, and customised as per the DB to keep the DB as stress free as possible.</p><p>MySQL -&gt; innodb_buffer_pool_size</p><p>PostgreSQL -&gt; shared_buffer</p><p>Oracle -&gt; DB_CACHE_SIZE</p><p>The optimal size of the buffer_pool depends on the available system memory and workload. But the general practice is to keep the buffer size as 70–80% of the available system memory.</p><p>The primary objective of these caches are to temporarily store the most accessed data and, and finally respond to the user queries, thereby eliminating a database call to fetch the data from DB.</p><p>In fact with performance and efficiency kicking in, these caches (buffers) are now optimizing even the write operations. In write-heavy systems, internal memory buffers (like the InnoDB redo log buffer) batch and delay disk writes to improve throughput.</p><h3>Scaling considerations</h3><p>Databases are one of the most robust systems ever built. All DBs comes with its own cache. And those are very efficient. But it has a limitation of being inside the DB. Well, scaling the Database, we also need to scale the DB cache and the performance is increased but as we know even vertical scaling has limitations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/342/1*bme4rlDtW7tHP5GcxC4kMQ.png" /><figcaption>DB Cache</figcaption></figure><p>We can scale the DB and its cache to serve from 10 requests per seconds to 1k-5k requests per second, but it would be a bottleneck for 100k requests per second.</p><p>The DBs inbuilt cache is very efficient, and designed well enough to handle high load, but when the application handles millions of users, it does succumb to high load. <strong>This threshold varies with Databases</strong>. But at some point, this does becomes the bottleneck.</p><p>These are not DB specific limitations, but a fundamental limitation of vertical scaling.</p><p>And as the requests-response time increases so does the users’ frustration.</p><h3>An Additional Cache</h3><p>Here comes the most intuitive strategy for all of the scaling related issues — <strong>Cache</strong>.</p><p>We can place an added cache layer before the queries make way to the actual DB. This has multiple benefits and does come with its own tradeoffs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/482/1*0rSrxk_mipKkOzZUJGKRYg.png" /></figure><p>This can be seen as a redundant layer, since we already have a DB cache in place for almost all databases nowadays. But the advantages it brings are worth noting for an application which is growing in size very rapidly.</p><h4>Benefits</h4><ol><li>Placing a caching layer would handle a read-heavy load very gracefully. Hence reduced latency.</li><li>This layer reduces our dependency on DBs cache to make our queries faster, and gives us the power to customize the cache as our use case be.</li><li>This also plays a role of separation of concern, between the choice of DB and choice of Cache layer. Now we can scale our DBs and Cache independently. This is particularly useful for microservice architectures.</li></ol><h4>Trade-offs</h4><p>We’ve already seen the advantages the cache layer provides : Speed, efficiency, performance.</p><p>But it is not a magic wand for all problems. It does come with its own limitations.</p><ol><li>Adding a cache layer in-between, adds another server to be maintained by the dev team. Hence more responsibility.</li><li>This increases the overall complexity of the system architecture. All services interacting with the DB now have to undergo slight modification to accommodate this layer.</li><li>Cache invalidation, and expiration is very crucial. With an extra layer of cache, these are to be very strategically implemented. Neglecting this would result in stale data.</li></ol><blockquote>Note : While adding another cache layer to your system architecture can provide significant performance and scalability benefits, it also introduces complexity, maintenance overhead, and potential risks.</blockquote><h4>Congrats, you’ve covered a crucial concept around Databases, and caches.</h4><p>Since you’ve come so far, exploring the technical details about the Database and caches, you deserve something more than just a pat on your back.</p><p><em>Your friend has finally decided to share the story with all the readers of this post as well. Enjoy the story.</em></p><h3>Story :</h3><p>In the bustling city of Metropolis, a new girl named Aria moved into town. She was mysterious, with an aura that drew attention from everyone, including the superheroes. Little did she know, her arrival would spark a conflict of epic proportions.</p><p>Loki, the God of Mischief, was the first to notice Aria. Intrigued by her enigmatic presence, he decided to investigate. Meanwhile, Deadpool, the Merc with a Mouth, also took an interest in Aria, but for entirely different reasons. He was convinced she was hiding something and wanted to uncover her secrets in his usual chaotic manner.</p><p>One evening, Loki and Deadpool crossed paths while following Aria. Loki, with his cunning smile, confronted Deadpool. “What brings you here, Wade? Surely, you’re not thinking of meddling in my affairs.”</p><p>Deadpool, unfazed, retorted, “Oh, Loki, always so dramatic. I’m just here for some fun. Besides, I think our new friend has a few secrets worth exploring.”</p><p>The tension between them escalated quickly, with Loki’s magic clashing against Deadpool’s unpredictable antics. The city trembled as their conflict grew, drawing the attention of Superman.</p><p>Superman, ever the peacemaker, swooped down and intervened. “Enough, both of you! This city has seen enough chaos. We need to resolve this peacefully.”</p><p>Loki and Deadpool paused, realizing the gravity of the situation. Aria, who had been watching from a distance, stepped forward. “I appreciate your concern, but there’s something you all need to know.”</p><p>Aria revealed that she was not just an ordinary girl but a guardian of an ancient artifact that could bring balance to the universe. She had come to Metropolis to protect it from falling into the wrong hands.</p><p>Superman, understanding the importance of unity, proposed a truce. “We must work together to ensure the safety of this artifact. Loki, Deadpool, can I count on you?”</p><p>Reluctantly, Loki and Deadpool agreed. They realized that their combined strengths could protect Aria and the artifact from any threat. With Superman leading the way, the trio formed an unlikely alliance.</p><p>As days passed, they faced numerous challenges, but their bond grew stronger. Aria’s presence brought out the best in them, and they learned to trust each other. Together, they safeguarded the artifact and ensured peace in Metropolis.</p><p>In the end, Aria’s arrival not only resolved the conflict but also united the superheroes in a way no one had ever imagined. They became a formidable team, ready to face any danger that came their way, proving that even the most unlikely alliances could lead to extraordinary outcomes.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e3fd7ed7dcc9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solving the CAPTCHAs]]></title>
            <link>https://greyhatlinux.medium.com/solving-the-captchas-8c52699e8644?source=rss-83eea1b79e2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/8c52699e8644</guid>
            <category><![CDATA[computers]]></category>
            <category><![CDATA[creativity]]></category>
            <category><![CDATA[captcha]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[apple]]></category>
            <dc:creator><![CDATA[Vishal Gorai]]></dc:creator>
            <pubDate>Mon, 21 Aug 2023 02:42:21 GMT</pubDate>
            <atom:updated>2023-08-21T02:42:21.694Z</atom:updated>
            <content:encoded><![CDATA[<p>Unravelling the CAPTCHA game from the start till date.</p><p><strong>The brief introduction to CAPTCHA</strong></p><p>Technically, CAPTCHA stands for “Completely Automated Public Turing Test to tell Computers and Humans Apart. The use case is pretty straightforward, to restrict unlimited consumption of internet resources by bots and spiders roaming in the public internet, we (users of internet) are presented with a real-time “turing test” while browsing. If users pass the test, they’re allowed forward, or else, they are thrown out.</p><p>With the mass adoption of internet is 2000s and simultaneously increasing computer bots, it became the need of the hour for protecting the resources from these 24x7 running scripts.</p><p>Just imagine a bot books all the movie tickets for a your favourite movie on its first show.The person running the bot, now sells the tickets in black for double or triple the price. I swear I will be pissed off.</p><p>Here comes the CAPTCHA for help. It was first developed by researchers in Carnegie Mellon University which uses the Turing test behind the scenes for it. Turing test in this context is “ a set of test that computers can grade, humans can pass, but paradoxiclly computers cannot.</p><blockquote>Fun fact : CAPTCHAs are intended to impede mechanized bots, but CAPTCHAs are themselves robotized.</blockquote><p>They’re customized to spring up in specific puts on websites. This feels annoying for sure. But this is so genuine in todays world. A report published on Wired.com in 2014 says “more than 50% of internet users are computer operates bots”. Think about 2023.</p><h3><strong>The traditional CAPTCHAs</strong></h3><p>Now that the CAPTCHA feels little justified, lets talk about their evolution in cyber space. Yes, its not only that internet has evolved over the time, all its components have shown significant progression. And the trajectory CAPTCHA has followed over the years caught my attention lately. Hence this blog, bringing out the length CAPTCHA has covered till date.</p><p>Going back about 10 years in time, we can recall encountering unwelcoming pop-ups at unexpected times. Those had distorted letters and numbers in them, and we had to identify them before we can proceed. This is a classic example of CAPTACHAs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7RWORSjJuwHqdLeB" /><figcaption>Image credit : <a href="https://www.okta.com/sg/identity-101/captcha/">https://www.okta.com/sg/identity-101/captcha/</a></figcaption></figure><p>Apart from the distortions in the characters, the fonts even had variety, some bold, some italics, some even user underlined for no obvious reason. For me personally, the biggest challenge was to distinguish from an “O for orange” and 0 (number zero). The only option which saved me from being declared a bot, in those places was the option to get next CAPTCHA</p><p>We don’t see them nowadays, but surely they must be in use even now. Another form of this traditional CAPTCHA method was the audio ones. I almost never used the audio CAPTCHAs.</p><blockquote>Do let me know your experiences with aud<strong>io CAPTCHAs .</strong></blockquote><h3><strong>Here comes the big tech giant — Google</strong></h3><p>The CAPTCHAs were doing all fine, and so was our frustration. No matter it served the purpose of CAPTCHAs, but ruined our momentum. Then came the tech super giant, Google and took over the CAPTCHA game and named it as reCAPTCHA.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/0*9W7GpkLYWfivmyvC" /><figcaption>Image Credit : <a href="https://developers.google.com/search/blog/2014/12/are-you-robot-introducing-no-captcha">https://developers.google.com/search/blog/2014/12/are-you-robot-introducing-no-captcha</a></figcaption></figure><p>It introduced the “<em>I’m not a robot” </em>button, which did a heck load of computation in the background to distinguish you from a bot. It utilizes all your past behaviour during your googling session, your button-clicks, mouse-movements to declare your behaviour human.</p><p>reCAPTCHA uses an advanced risk analysis engine and adaptive CAPTCHAs to keep automated software from engaging in abusive activities on your site</p><p>But incase, google doubted your humanly behaviour, it presented you with 4x4 boxes to identity fire hydrant and traffic lights. This process served purposes like training AI by giving them larger datasets. This in turn helps preserve books, improve maps, and solve hard AI problems by the bots itself.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/0*MGT89B8bHKjy4zHC" /><figcaption>Traffic Light CAPTCHA</figcaption></figure><p>But on a positive side, this was a manifold jump in the way we experienced CAPTCHAs during browsing. Lesser the CAPTACHs, better the user experience. Win-Win for all.</p><blockquote>More on Google’s reCAPTCHA : <a href="https://www.google.com/recaptcha/intro/invisible.html?ref=producthunt">https://www.google.com/recaptcha/intro/invisible.html?ref=producthunt</a></blockquote><h3><strong>The CAPTCHA game became interesting</strong></h3><p>Now that the U/X was improved, people started having fun with CAPTCHAs. Though the reCAPTCHAs reduced the pop-ups by 70%, creative people saw an opportunity even in the 30% of the times we had to prove ourselves human. Things started getting interesting</p><blockquote>Enter rotation CAPTCHA and slider CAPTCHA</blockquote><p>Slider CAPTCHA : We are now presented with a broken piece from a jigsaw puzzle and asked to slide the piece to correctly complete the puzzle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/377/0*HqQYJfxrBZ5HlEm2" /><figcaption>Slider CAPTCHA</figcaption></figure><p>Rotation CAPTCHA : In this, an object appears in odd position. We have to rotate it for few times, to bring it in upright position.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Jif1vQqC741b3zt8" /><figcaption>Rotation CAPTCHA</figcaption></figure><p>These variations of CAPTCHAs proves, people can turn anything to fun, given the creativity we have. This however didn’t bring any revolutionary change in the CAPTCHA space, but it appealed to the users. In fact I started enjoying solving CAPTCHAs at some point. But this fun was short lived. When work becomes more, our mind doesn’t really enjoy small fun sliders. After all who would welcome a CAPTCHA when you had almost booked a train ticket, but couldn’t pass the CAPTCHA on time. And by the time you start booking again, you get the wait listed ticket.</p><h3><strong>Enters the Big Daddy — Apple</strong></h3><p>Apple recently upped the CAPTCHA game by introducing their own “human auth” system. It recently launched a Youtube teaser to their latest upcoming feature where Apple has trolled other CAPTCHA methods, and indeed it feels so better to see the next level of CAPTCHA catching up. The perfect blend of Apple ecosystem and CAPTCHA.</p><blockquote>Youtube : <a href="https://www.youtube.com/watch?v=ykNWqUlLftE">https://www.youtube.com/watch?v=ykNWqUlLftE</a></blockquote><p>It is a must watch video. In fact this video intrigued me to dig into the CAPTCHA world from the start.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*DvEYSiudhywJNQt_" /><figcaption>Source : <a href="https://developer.squareup.com/docs/web-payments/apple-pay">https://developer.squareup.com/docs/web-payments/apple-pay</a></figcaption></figure><p>At the core it feels being such a basic idea, researchers didn’t use this before and Apple is doing it. Bingo! Fingerprint, and face-lock being used for real time human-auth tests, feels really seamless. Just like unlocking your phone, or your mac. This feature will by far increase the user experience at its best. Apple has integrate this feature with ApplePay which serves a great purpose. A significant jump in CAPTCHA systems used in Apple devices.</p><blockquote>The hack : Devices will use your fingerprints to classify you as humans, because bots do not have fingers right.</blockquote><blockquote><em>Thanks for reading! 😃</em></blockquote><p>If you’ve reached till here, I’m sure you’re into the CAPTCHAs as much as I’m. Cheers mate. I will appreciate you giving it a clap and share it with your peers. Consider getting connected on <a href="https://www.linkedin.com/in/greyhatlinux/">linkedin</a> :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8c52699e8644" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My six-months internship experience]]></title>
            <link>https://greyhatlinux.medium.com/my-six-months-internship-experience-39d49e54a2fc?source=rss-83eea1b79e2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/39d49e54a2fc</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[internship-experience]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[blog]]></category>
            <category><![CDATA[kafka]]></category>
            <dc:creator><![CDATA[Vishal Gorai]]></dc:creator>
            <pubDate>Sun, 24 Jul 2022 13:32:41 GMT</pubDate>
            <atom:updated>2022-07-24T13:52:24.039Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>at Fastenal India</blockquote><p>It has been more than 6 months working with <a href="https://www.fastenal.com/">Fastenal </a>in the infrastructure team. I had a chance to talk with many senior engineers, read lots of documentation and look into many aspects of industry level work. Along the journey I’ve come across many tools and technology used by the company to build and support the company’s infrastructure. With this post I would like to share my technical journey so far.</p><p>Talking about my day to day work, my team and I build data pipelines to stream data between endpoints seamlessly. It’s okay if you didn’t get a word of it. It is complicated. To put it in the simplest of words, we build the lines in the system design documents you regularly see.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/671/0*5UTwLGjGyQ-1kAJ3" /><figcaption>Basic System Architecture</figcaption></figure><p>For reference, the above represents a diagram of the most basic system. Our team is responsible for building, scaling and maintaining each of the lines which you see in the diagram. For my tenure at Fastenal India, I’ve contributed to build data pipelines to keep our databases in sync, ie, the dotted lines in the above picture.</p><h4>Demystifying real-time data streaming :</h4><p>Real-time data streaming isn’t that complex to understand as it is to read it. Let me assist you. A couple of decades back, we used to get news from newspapers. The news agency used to collect news all day, compile all of them and then finally print them in the newspapers which we all read with our morning coffee. Well, the scenario has completely changed now. We receive news on the go through google-news, news outlets, and even instagram as and when it happens. We do not wait for the entire day to get all the news at once. We get real-time news from around the world. This is an apt description of real-time data streaming.</p><figure><img alt="real time data streaming" src="https://cdn-images-1.medium.com/max/1024/0*GprwNq8D9_yIxx1N" /><figcaption>Source : Confluent page</figcaption></figure><p>There are ample mechanisms and great tools available to transport data between systems and processes, but we use the most talked about tool for data streaming — Kafka.</p><figure><img alt="A system without kafka" src="https://cdn-images-1.medium.com/max/691/0*Ro84iqnBlOEWauEN" /><figcaption>A system without kafka</figcaption></figure><h4>A little glimpse about Kafka :</h4><p>“Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications”</p><p>Few noteworthy points from the definition above :</p><ol><li>Open source Software</li><li>Distributed system</li><li>High performant system</li></ol><p>While <a href="https://kafka.apache.org/documentation/">Kafka documentation</a> is insanely comprehensive, the use cases of Kafka are indeed fascinating. The introduction and evolution of this data streaming mechanism has brought some significant changes in the way large companies ingest huge volumes of data and produce it to data consumers.</p><figure><img alt="A kafka-enabled system" src="https://cdn-images-1.medium.com/max/524/0*20sIX7KVRMZpxk22" /><figcaption>A kafka-enabled system</figcaption></figure><h4>What is Kafka :</h4><blockquote>Long answers short, it’s a Publish-Subscribe model</blockquote><p>Publishers produce data and subscribers consume data from respective publishers.</p><p>Publishers produce data in all shapes and sizes, in all imaginable formats. Subscribers consume data in specific format as they require for their needs.</p><h4>How kafka does it differently :</h4><blockquote>By decoupling data producers and consumers.</blockquote><p>Publishers publish data to a centralized system, and subscribers consume data from the centralized data store. Publishers produce data in all shapes and sizes, in all imaginable formats and simply push to the system. Subscribers consume data in specific format as they require for their needs. This format conversion is all handled by the centralized system in this case. This is called decoupling of producers and consumers. In this architecture, the producers and consumers are totally unaware of each other which has many advantages as proven in its use cases.</p><h4>How do we use it?</h4><p>As I have briefly mentioned above, we use the kafka framework to keep the databases in sync. More specifically, we work with sink connectors and k-streams for the purpose.</p><figure><img alt="kafka implementation" src="https://cdn-images-1.medium.com/max/592/0*ypJxboUJo951kS-I" /><figcaption>kafka implementation</figcaption></figure><p>That’s a lot to take in for a brief introduction about Kafka, and my job profile. Also kafka docs would be your best fr<a href="https://twitter.com/greyhatlinux">i</a>end for more <a href="https://twitter.com/greyhatlinux">i</a>n-depth techn<a href="https://twitter.com/greyhatlinux">i</a>cal<a href="https://twitter.com/greyhatlinux">i</a>t<a href="https://twitter.com/greyhatlinux">i</a>es. I would love to hear your use cases of Kafka if you use it in your work, or simply shoot me up with any questions you have about this.</p><p>I do intend to write more about the workings, discussing things in more detail. More technicalities for next time. Till then feel free to catch me up on <a href="https://www.linkedin.com/in/greyhatlinux/">Linkedin</a>, <a href="https://twitter.com/greyhatlinux">Twitter</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=39d49e54a2fc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Knock Knock!]]></title>
            <link>https://greyhatlinux.medium.com/knock-knock-6f0270b9d85d?source=rss-83eea1b79e2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/6f0270b9d85d</guid>
            <dc:creator><![CDATA[Vishal Gorai]]></dc:creator>
            <pubDate>Fri, 24 Sep 2021 18:01:06 GMT</pubDate>
            <atom:updated>2021-09-24T18:01:06.504Z</atom:updated>
            <content:encoded><![CDATA[<p>Checking off another one from my to-do list.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*IMy_kmkY-3jdQUmFWWZ5aw.png" /></figure><p>Launching my own weekly newsletter!</p><p>Creating a newsletter has been in my bucket list, but good things take time, and so I believe this was late. Not my issue though! <br> <br>I invite you to my weekly newsletter, at <a href="https://www.getrevue.co/profile/vishalgorai">Revue</a>. I’ve released a quick note, about me and my purpose for this newsletter on the very first issue.<br>A quick subscription will make my future releases reach you at the earliest. So go ahead, hit the subscription.</p><blockquote><a href="https://www.getrevue.co/profile/vishalgorai/issues/ready-steady-go-649381">My first letter</a></blockquote><p>Hit me up on <a href="http://linkedin.com/in/greyhatlinux">Linkedin</a>, <a href="http://instagram.com/_vishalgorai_">Instagram</a> or <a href="http://twitter.com/greyhatlinux">Twitter</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6f0270b9d85d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why DigiBoxx, Why?]]></title>
            <link>https://greyhatlinux.medium.com/why-digiboxx-why-7326c2f2fabc?source=rss-83eea1b79e2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/7326c2f2fabc</guid>
            <dc:creator><![CDATA[Vishal Gorai]]></dc:creator>
            <pubDate>Sat, 30 Jan 2021 03:31:05 GMT</pubDate>
            <atom:updated>2021-01-30T05:58:13.459Z</atom:updated>
            <content:encoded><![CDATA[<h4>Disclaimer : The views on this article are purely my personal opinions. It has nothing to do with endorsement, or brand defamation. I encourage the dev community to take a look at this, and correct me if I’ve misunderstood any aspect of it.</h4><h3>What is DigiBoxx?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*Bes022j6Lt5-N_98G_CP1Q.jpeg" /><figcaption>DigiBoxx</figcaption></figure><p>DigiBoxx is an Indian asset management and cloud storage provider, It was launched in December 2020 amid the huge wave of “Atmanirbhar Bharat’’ within the nation. The movement targets the ever growing dependence of Indians on foriegn service providers for almost all their needs. DigiBoxx came as an Indian tech startup in the cloud sector. It claims to be one of its kind indegenious SaaS providers with <em>data security</em> and <em>data localisation</em> as its main priorities. This company was brought out in public by Amitabh Kant, CEO NITI Aayog, with a very optimistic note for the future.</p><h3><strong>Is it a necessity?</strong></h3><p>In the cloud space, we have with us numerous service providers catering to all personal and business needs at the moment. But when we see from Indian landscape, it hurts to find not even a single Cloud Storage provider. For an emerging economy like India, and to handle the fastest happening digital transformation, an Indian Cloud Solution becomes a necessity. Also the recent data privacy threats coming in from China and other countries has escalated the need further more for data localisation and security focussed cloud service providers.</p><h3>Alternatives to digiboxx</h3><p>DigiBoxx is just in its nascent phase, and has to go miles. Obviously it has very tough competition from AWS S3, DropBox, Google Drive, etc. But it adds a sense of nationalism when using Indian Product. So yes, there are significant competitors out there, but it surely could give tight competition.</p><h3>My reviews 📑</h3><p>DigiBoxx is good for a start. Definitely not close to already established service providers in this sector, and has great room for improvement, but a fairly prominent stand for a long run.</p><h4>UI/UX : 3/5</h4><p>The User Interface is simple and sweet. But the User Experience is poor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sk5osTbT01p5YVNQ1s6WMg.png" /><figcaption>Digiboxx Mobile App login Screen</figcaption></figure><p>Hey its so obvious, one has to be the username and the other has to be the registered email.</p><blockquote>In contrast to the labels, the first textbox requires username and the second one requires your email 😆</blockquote><p>After uploading files, some pages, the options don’t work. You simply get tired of clicking the button, until noon.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/608/1*Qi9KKWNzG6rS-9yG22iRFg.gif" /><figcaption>Unresponsive buttons on DigiBoxx web</figcaption></figure><h4>File Sharing: 3/5</h4><p>Once you’ve your file in your boxx, it becomes fairly easy to share with people. You give it the required public-access permission and it is ready to be shared. The cool feature which I personally liked is, it allows you to share files to those who doesn’t even have DigiBoxx account. You simply put their email, and it will notify them about the file shared via their email. Though the other person has to ultimately create an account to view the file.</p><h4>Business Friendly : 2/5</h4><p>From a developer’s perspective, the very first thing you look for in a system is its APIs and its documentation. But DIgiBoxx has no such provision for this. Considering this is still evolving, what about the company claiming to be very feasible for small businesses? Does it expects business and its engineers to upload each file manually?</p><p>I hope it shows up with a great API package in the next versions.</p><p>Note : For now, DigiBoxx is definitely not ready to go beyond personal level. Neither DigiBoxx is ready, nor businesses would be flexible with it.</p><h4>Security: 1/5</h4><p>Being more into the engineering aspect, I doubt thesecurity aspects of the files I store in my box. And that wasn’t impressive. As far as I’ve analysed, upon uploading a file, there are two links available to you.</p><ol><li>For sharing with people. <em>(app.digiboxx.com/…)</em></li><li>In the Web content, that can be extracted very easily <em>(prod.store.digiboxx.com/…)</em></li></ol><p>Though the first link strictly follows the public-private access policies, the second link is above all these boundaries. No matter what access rules you’ve set for a file, if you have the second link, you can access it from anywhere anytime. I don’t know whether this design pattern is intentional or is a loophole, but this is very severe.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EawETb28abihG9ucxU_Lpw.gif" /><figcaption>Security Issue 1</figcaption></figure><p>Also, upon further recon, I found out that DigiBoxx internally uses “MinIO’’ for its file storage. <a href="https://min.io/">Link</a></p><blockquote>“MinIO’s high performance, Kubernetes-native object storage suite is built for the demands of the hybrid cloud. Software-defined, it delivers a consistent experience across every Kubernetes environment.”</blockquote><p>I learnt that minio was an open-source software solution, under Apache Licence v3. So DigiBoxx dev team can modify, and implement best security practices within the k8 cluster. That’s great!</p><p>But what came to me as a surprise was this :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qAnuX9AbFGPGDejHaNL4nw.gif" /><figcaption>Security Issue 2</figcaption></figure><p>This is a direct login console of one of their k8 nodes. Though this still requires valid access key and secret key, this provides a great attack surface for malicious actors. One could easily brute force these keys, and within hours (at max) the entire node along with its data will be compromised. A better approach would be to restrict the access to the login screen in the first place.</p><h3>My concerns 😯</h3><p>Apart from what issues I personally faced while using the service, what actually concerns me is:</p><h4>“Whether it truly is Indian?”</h4><p>The second link, which can be extracted from the web-page is :</p><p><a href="https://prod.store.digiboxx.com:9000/e32ebd3f4ded451c/Temp/Screenshot_20210126-141128_FAU-G.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=aazxcdklmnghfg%2F20210128%2Fus-east-1%2Fs3%2Faws4_request&amp;X-Amz-Date=20210128T144735Z&amp;X-Amz-Expires=604800&amp;X-Amz-SignedHeaders=host&amp;X-Amz-Signature=721ff4df0b3f688a862f5926bf1239daa3d55303a34d51704b2dbf0b40cd1a14">https://prod.store.digiboxx.com:9000/e32ebd3f4ded451c/Temp/Screenshot_20210126-141128_FAU-G.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=aazxcdklmnghfg%2F20210128%2F<strong>us-east-1</strong>%2Fs3%2F<strong>aws4_request</strong>&amp;X-Amz-Date=20210128T144735Z&amp;X-Amz-Expires=604800&amp;X-Amz-SignedHeaders=host&amp;X-Amz-Signature=721ff4df0b3f688a862f5926bf1239daa3d55303a34d51704b2dbf0b40cd1a14</a></p><p>(you can click on the link, it is a screenshot of FAUG game I tried recently)</p><p>If you look closely at this, you can very well find out well-known words, like “us-east-1”, “aws4_request”, etc. Infact on deep diving on this, I found out that this is an AWS service for signing API requests. More technically,</p><blockquote>“When you send HTTP requests to AWS, you sign the requests so that AWS can identify who sent them. You sign requests with your AWS access key, which consists of an access key ID and secret access key. Some requests do not need to be signed, such as anonymous <strong>requests to Amazon Simple Storage Service</strong> (Amazon S3) and some API operations in AWS Security Token Service (AWS STS).”</blockquote><p>Link to <a href="https://docs.aws.amazon.com/general/latest/gr/signing_aws_api_requests.html">AWS documentation</a>.</p><p>I may be completely wrong in this, but this appears to me like the assets with DigiBoxx use AWS services. It may be simply for authentication, or for entire storage. So, it is like our data is again in AWS data centers. More than the fact that it uses AWS, the data rests in “us-east-1” data center, which means data will comply to the data policies of that country. Back to square one? Honestly, I want to be proved wrong here. Because, if it turns out to be true, the existence of DigiBoxx hardly matters. And what about “Atmanirbhar Bharat”?</p><p>Is it another “Freedom 251” smartphone?</p><h3>Conclusion</h3><p>Though the necessity of an indeginous player is significant in this domain, companies should not rush to things just to exist. Companies should instead focus on security and on quality services and features.</p><p>I would reiterate that all the views and opinions are totally mine and it should be taken as a positive push by the tech company to further strengthen their services as well expand their scope in these domains. This was not to defame or harm the company motives by any means. I encourage the dev team at DigiBoxx to take this opportunity to improve, and If I’ve failed to understand things, I express my sincere apologies. Please find time correct me.</p><blockquote>Thanks for reading! 😃</blockquote><blockquote>Reach me out at <a href="https://twitter.com/greyhatlinux">twitter</a> ✉️</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7326c2f2fabc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes simplified with AWS EKS]]></title>
            <link>https://greyhatlinux.medium.com/kubernetes-simplified-with-aws-eks-32bccf61acc5?source=rss-83eea1b79e2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/32bccf61acc5</guid>
            <dc:creator><![CDATA[Vishal Gorai]]></dc:creator>
            <pubDate>Sat, 11 Jul 2020 19:18:24 GMT</pubDate>
            <atom:updated>2020-07-11T19:18:24.731Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Let’s get into container orchestration with aws.</blockquote><p>Hello readers! I hope you all are doing fine in your domains. Well, mine is cloud, and with this I’ll take you deep into this. Cloud is all over, and AWS dominates the cloud domain. Needless to say how it has revolutionarised the cloud space, and is moving further ahead with its continuously expanding services. In this article, we will be exploring one of its services ie, AWS Elastic Kuberneter Service (EKS) and simultaneously build a small Kubernetes (K8) application.</p><h3>EKS</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*FN7HQnHpNmwQbHDv5MbNAg.jpeg" /><figcaption>AWS EKS</figcaption></figure><p>EKS is a fully managed kubernetes service provided by AWS to provision and manage your K8 cluster. EKS efficiently manages the availability and scalability of your pods (containers), also it automatically detects and replaces unhealthy nodes for each cluster so your customer never gets a 404.</p><p>EKS has a huge fanbase. From huge investment banks like HSBC to the social media app Snapchat. EKS is the go to solution for all container implementation on cloud which you can think.</p><p>There are two ways to go ahead with EKS</p><ol><li>The EC2 way</li><li>The severless way</li></ol><h3>EC2 way :</h3><p>This gives you the complete control of your K8 cluster. From choosing EC2 instances with different configuration to monitoring it from inside the pods. It is a great option if your organisation has a very good DevOps team to manage so much granular control. Along with great control, you get better cost optimisation.</p><h3>The serverless way :</h3><p>This employs the AWS Fargate, so that you get rid of any server management thing and focus only on your pods, more specifically your business. This doesn’t give you the headache (control) of you node machines. Fargate provisions the right amount of compute, eliminating the need to choose instances and scale your cluster. In this, you pay for resources per application. This would be the go-to solution with orgs with a small or little expertise on server management stuffs. This obviously would be little expensive than EC2 way, since professional AWS engineers handle you master nodes.</p><p>Both this mechanisms have trade-offs, and it solely depends on your need to embrace any one.</p><h3>Overall</h3><p>The Master in EKS is configured in multiple availability Zones (AZs) by AWS. This makes your ecosystem fault tolerant, in case of natural disasters. You don’t have any access to Master Node, since it is fully managed and configured by AWS, this means you can’t <em>ssh</em> into it. I guess that itself reduces much of the stress ;)</p><p>EKS is tightly coupled with other AWS services like EFS, ELB, CloudWatch and many more. Here, we will be using a few in our very basic cluster.</p><h4>The internal EKS architecture :</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Mj2IAwvAQOnwW4I6105lLw.png" /><figcaption>eks architecture</figcaption></figure><h3>Jumping into the Cluster</h3><ol><li>Get your system ready :</li></ol><p>Install <a href="https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html">awscli</a> (for handling aws console from the terminal)</p><p>Install <a href="https://eksctl.io/introduction/#installation">eksctl</a> (for managing K8 on AWS)</p><p>Install <a href="https://kubernetes.io/docs/tasks/tools/install-kubectl/">kubectl</a> (for managing K8 cluster)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/921/1*eViRVHhCzcaMt4kCROG4Vw.png" /><figcaption>system setup</figcaption></figure><p><strong>2.</strong> Configure your awscli with the user with permission to access EKS and other services. I configured it with an admin. Also note that if you’re using AWS Fargate, decide the region appropriately. Fargate is not available in all AWS regions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*btoSeUOCr8iod7vGm9zeDw.png" /><figcaption>awscli configure</figcaption></figure><p><strong>3. </strong>Initially there won’t be any cluster with AWS. Check with following command</p><pre>$ eksctl get cluster</pre><p>4. We will provision our first cluster with very basic cluster configuration. We will use a <em>yaml</em> cluster configuration file and use <em>eksctl</em> to execute from cli.</p><pre>apiVersion: eksctl.io/v1alpha5<br>kind: ClusterConfig</pre><pre>metadata:<br> name: my-kube-config<br> region: ap-southeast-1<br> <br>nodeGroups:<br> — name: node-group-1<br>   desiredCapacity: 2<br>   instanceType: t2.micro<br>   ssh:<br>     - publicKeyName: &lt;your-keypair-name&gt; </pre><pre> — name: node-grop-2<br>   desiredCapacity: 1<br>   instanceType: t2.micro<br>   ssh:<br>     - publicKeyName: &lt;your-keypair-name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/761/1*ZmzOdrYsx8al--W1ktvfJw.png" /><figcaption>creating cluster with eksctl</figcaption></figure><blockquote>If you look into the logs closely, eksctl is using the CloudFormation service to launch the worker nodes</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*6ouODP--RukQfjSZxYV5EA.png" /><figcaption>using cloudformation in the backend</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*w7qZA8GPEn-IZf3WvJN4sw.png" /><figcaption>cluster up and running after 10–15 minutes</figcaption></figure><p>5. Configuring kubectl locally. Initially your kubectl won’t be configured, and your config file will be something like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/578/1*JrANJwf655w46bN2M-6m3A.png" /></figure><p>Run the command to configure the local kubeconfig file</p><pre>aws eks update-kubeconfig --name &lt;cluster-name&gt;</pre><blockquote>This makes you kubernetes ecosystem ready. Above this you can run you K8 apps.</blockquote><p>6. Without diving deep into K8, here we will deploy the most basic K8 application on our EKS</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WasdLHqfpYShjLUD_QV0KA.png" /><figcaption>k8 app deployment</figcaption></figure><blockquote>In K8 by default, the Pod is only accessible by its internal IP address within the Kubernetes cluster. To make the node accessible from outside the Kubernetes virtual network, you have to expose the Pod as a Kubernetes <em>Service</em>.</blockquote><p>7. Exposing the port to the outer world.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xmoo2HdM5EJBZ7oVGdTMQg.png" /><figcaption>EKS services</figcaption></figure><p>Here in EKS, the load balancer type is ELB Classic pre-defined by AWS. You could chaange it and use of from many available load balancers.</p><p>Managing the pod from inside.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*sk-ZhVDfXb7Ls6b-2LUc5g.png" /><figcaption>pod exec</figcaption></figure><p>8. Finally getting the mess cleaned up from the pods</p><pre>kubectl delete deployment --all</pre><p>9. Releasing the EKS services</p><pre>eksctl delete cluster --name &lt;cluster-name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*imFgUXgacKv7zCtswnEUTw.png" /><figcaption>eksctl delete cluster</figcaption></figure><h4>EKS Pricing</h4><p>You pay $0.10 per hour for each Amazon EKS cluster that you create. You can use a single Amazon EKS cluster to run multiple applications by taking advantage of Kubernetes namespaces and IAM security policies.</p><blockquote>Thank you all :)</blockquote><p>That’s it for stepping into the EKS. If you’ve come uptill here, I am sure you found this definitely into the dev and K8 stuff. Give it a clap and share it with your cluster. Consider getting connected with me on <a href="https://www.linkedin.com/in/greyhatlinux/">linkedin</a> :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=32bccf61acc5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS WorkSpace : The taste of cloud]]></title>
            <link>https://greyhatlinux.medium.com/aws-workspace-the-taste-of-cloud-957824b53b47?source=rss-83eea1b79e2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/957824b53b47</guid>
            <dc:creator><![CDATA[Vishal Gorai]]></dc:creator>
            <pubDate>Tue, 02 Jun 2020 15:35:58 GMT</pubDate>
            <atom:updated>2020-06-02T18:25:16.427Z</atom:updated>
            <content:encoded><![CDATA[<h3>AWS WorkSpace : The taste of cloud</h3><h4>Hello readers, hope you’re WFM.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/339/1*Bw2g-OCpxtS-NQ6YgWXflw.jpeg" /><figcaption>cloud</figcaption></figure><p>Cloud is no doubt the most in-trend technology. Even the Artificial Intelligent systems leverage the benefits cloud extensively, making it even more demanded technology we have currently.</p><p>We’ve all been using cloud space for more than a decade, but apart from google drive, one hardly experiences the first-hand potential of cloud. This is because the entry barrier to this technology has been set so high, that a non-tech person is easily turned off to get the first-hand experience.</p><p>Ask someone about cloud, and the very first few word he speaks is <strong>servers. </strong>Now before cloud, you now have to understand servers, then APIs, and then storage volumes, etc etc. This is the barrier I was talking about. It becomes really tough to actually feel what cloud has to offer you. But no worries, you’re not alone. You’re just a previous version of me. The only difference is that, I took the long and tough road, but you won’t have to.</p><blockquote>If you wish to learn more on cloud check <a href="https://medium.com/@greyhatlinux/an-edge-with-cloud-computing-de56f03cae2f">this</a>.</blockquote><p>With this blog I wish to provide you with the first-hand experience of cloud, without you knowing anything about servers and all. If you’ve ever used a computer, that’s more than sufficient for you to go ahead with this.</p><blockquote>Lets dive in :)</blockquote><p><strong>Virtual Desktop Infrastructure (VDI)</strong></p><p>From now on I’ll be talking about virtual desktops, virtual because it will be somewhere up in the clouds, and not with you. VDI is the technology to provide you with your personal desktop, but more mobile than you’ve ever thought. Almost all cloud service providers have this service, but my love for AWS is so pure, I simply couldn’t resist. So compiling eveything till now, we will be taking a look at <strong>AWS WorkSpace</strong> (VDI) in the following blog.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/154/1*AZ4JUIk8WZu7mPe0W-PaCA.png" /><figcaption>AWS WorkSpace</figcaption></figure><p>All you need for this is to have an AWS account, and set up your AWS WorkSpace from the AWS official <a href="https://docs.aws.amazon.com/workspaces/latest/adminguide/getting-started.html">docs</a>.</p><blockquote>In response of COVID-19, AWS WorkSpaces Free Tier provides one WorkSpace with Windows Performance bundle with 80 GB Root and 100 GB User Volumes, for up to 200 hours, and 400 hours of Linux.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/519/1*evW8MLevXTciNWAoT3b5tg.png" /><figcaption>Source : AWS Docs</figcaption></figure><p>This is the best time for you to get your hands dirty with the services, <strong>all for free</strong>. Its so infectious, that I fear that after this COVID, I will have no options than to buy the premium for this.</p><p><strong>AWS WorkSpace</strong> has a plethora of features for all. If you’re a student, it provides you an extremely mobile and an easily disposable desktop for all kinds of activities. If you’re a ML/AI enthusiast, this could be your heaven. If you’re more into graphics, I guess 16 vCPU, 122 GiB Ram would be enough. I won’t mention employees, because more than 70% of company provides their employees with a personal WorkSpace. Even a common man will find this extremely handy.</p><p>AWS WorkSpace feels like 99% you own PC, well that 1% depends on your internet connection. If you’ve unstable internet, probably this could be frustrating. Given that the only prerequisite to work in cloud is a stable internet connection, this becomes obvious.</p><p>Coming back to AWS, it gives you a number of specs to configure you want for your desktop. From 1 vCPU, 2GiB RAM upto 16 vCPUs, 122 GiB RAM, and trust me the 4CPU, 8GiB RAM it offers is significantly faster that your laptop with same config. Reason? Beacuse it provides you 4 vCPUs from Intel Xeon E processor family. Infact, it has options for different OSs. If you work on Windows, well and good, else you can also opt for Linux based desktops.</p><blockquote>Under Free tier it provides only CentOS Linux</blockquote><h3>Accessibility</h3><p>The best part about AWS WS is its accessibility. You must think that, if this is virtual desktop, it has to be used always on PCs only. Well, you underestimated it. AWS provides all kinds of options for accessing its WS. You name it, it has it for you.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/188/1*2vvMwd6oAxH0ACB-CUwovA.png" /><figcaption>access options</figcaption></figure><blockquote>The browser based connectivity provides you on the go, but AWS WS client application is recommended.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9OumfmmPgnZkRc8wfhZfZw.png" /><figcaption>AWS WS from client app on PC</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mxKYBU_-gYYjCzNgiNeiPg.png" /><figcaption>AWS WS from mobile</figcaption></figure><blockquote>To be honest, its feature to bring desktops to mobile has changed my perspective of bulky PCs.</blockquote><h3>Utility</h3><p>It has almost all kinds of feature you can expect from you own PC. You can browse internet, do development, designing, and much more. The kind of power and resources it provides you, is awesome.</p><p>The best part about this, is its internet speed. Its <strong>enormous</strong>. Well, the internet speed also depends on the AZ you select for you workspace. I personally prefer using WS if I have to do data intensive jobs, because it downloads GBs of data before you blink.</p><p>I used it to do my docker projects development.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VHXuPHTNKgrjm4VQERUaJQ.png" /><figcaption>In Singapore region, its close to 200 Mbps</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R6dX6fWc3OTwZV14JbX2jA.png" /><figcaption>In N. Virginia region, its close to about 800 Mbps.</figcaption></figure><blockquote>That sounds like all gold, but no. I’ll talk about the downsides further.</blockquote><h3>Hacks</h3><p>Number of workspaces are monitored region-wise. In singapore, you need permissions to provision more than 1 workspace, but in N. Virginia you can provision multiple WS without explicit premissions.</p><p>It provides extreme flexibility. Whenever you log in, it autostarts the desktop.It stops the WS when it is idle for more than 1 hr</p><p>You’re only billed for number of hours you use plus the start and stop time, (which is few seconds).</p><p>The system of snapshot also is again of great utility. When you log in, you get the exact same state of desktop as you left it.</p><p>In Linux WS, it comes pre-installed with AWS CLI. You can directly jump in to manage your system. In Windows, however, You explicitly need to install aws-cli.</p><p>Not all AZs have WorkSpace option. That doesn’t matter afterall. Opt for N. Virginia wherever you live, and enjoy your 800 Mbps net speed.</p><p>Before going for subscription, you must try out the free tier. I openly admit, this may not be for you.</p><h3>Few Downsides :</h3><blockquote>These are all my personal views, and depending on your needs and usage, you might never encounter this.</blockquote><p>In free version of Windows, it doesn’t provide MS Suite. I missed using PowerPoint in WS. Well, you can purchase the Suite along with a powerful system.</p><p>The Windows instance on 4GB RAM (free tier) feels a bit slow. Given the fact that Windows is extremely RAM comsuming, this is self-explanatory. But 8GB, 8vCPU (paid) system is blazingly fast. I used it without a lag.</p><p>Watching videos on WS costs you more of your data. I meant to say far more than what it would if you directly watch from PCs. I won’t advocate binge watching on WS atleast.</p><p>In Linux OS, It doesn’t allow direct root login. The user in WS always has to be a sudoer.</p><p>In Linux, you only get CentOS as an OS option. Well, Ubuntu would have been a good choice for a general option.</p><p>The CentOS comes with aws-cli v1, ie an obsolete package. You need to install the latest aws-cli v2 separately. Well, for windows, you dont even get aws-cli installed.</p><p>The internet connectivity is superfast, but you don’t get internet within your docker containers. I don’t know why.</p><blockquote>My hack : I figured it that the default nameservers in the <em>resolf.conf</em> file was an issue for containers to get internet connection. Simply overwrite the <em>resolf.conf</em> file with google’s nameserver, and you’re done!</blockquote><pre>$ sudo echo &quot;namesever 8.8.8.8&quot; &gt; /etc/resolf.conf</pre><h4>Thanks for being a patient reader.</h4><p>Well, that was my experience of WS, and I’m still exploring. If you have your own, I’ll be glad to hear it from you down below. Reach <a href="https://www.linkedin.com/in/greyhatlinux/">me</a> out if you face difficulty setting up your AWS WS.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=957824b53b47" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[E-mailing just got handy]]></title>
            <link>https://greyhatlinux.medium.com/e-mailing-just-got-handy-6b638bd3dcc3?source=rss-83eea1b79e2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b638bd3dcc3</guid>
            <dc:creator><![CDATA[Vishal Gorai]]></dc:creator>
            <pubDate>Thu, 14 May 2020 23:04:58 GMT</pubDate>
            <atom:updated>2020-05-15T07:14:29.470Z</atom:updated>
            <content:encoded><![CDATA[<p>The terminal is the most lethal weapon in the arsenal for a sysadmin. No GUI can ever replace the efficiency and power of Command Line Interface for a hard-core system engineer. All software applications make use of the power of the terminal behind the scenes. Interestingly, at the server level, there’s no GUI at all. The only two things you get is the a black screen, and a keyboard, and there it is, the entire organisation’s infrastructure is at your commands.</p><p>Well, the CLI isn’t loved initially. And there you got covered by the beautifully designed and smartly crafted UI for almost all utilities. Infact, the UI’s are superflexible and the go-to destination for all the new comers. But with time, one understands the capabilities of the terminal and its commands, and guess what, there’s no return! You can do anything from the terminal. There are utilities and packages for all functions, but the hard part is to find the best one for you and your needs.</p><h4>But, what about emailing from the server?</h4><blockquote>Good news hackers, I got you!</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*9D7GOC5nDvS3Df1HuwIgcw.png" /></figure><p>I’m here to increase your on-terminal screen time. In this, I’ll help you set-up your messaging service from terminal, so that you don’t have to migrate to Gmail, everytime you need to send an email.</p><p>In this I’ll be using the <strong>SSMTP package </strong>to configure email services. It is a linux utility and requires a quick initial setup before you’re good-to-go.</p><h4>Note :</h4><ol><li>This is only one way mailing system. You can only send emails, and not receive from the other end.</li><li>There are many such emailing services for systems, but I found this one the quickest way to get your set-up done.</li><li>It can be very handy to send short code-snippets to your colleagues or quick messages to your loved ones, from a server itself.</li><li>One of its primary use cases of <strong>ssmtp</strong> service is forwarding automated email (like system alerts) off your machine and to the admin email address for him to remain notified about system health.</li></ol><blockquote>Let’s Dive In</blockquote><p>Get you system up and running.</p><ol><li>Update and get youself the latest of all packages.</li></ol><pre>$ sudo apt-get update</pre><pre>$ sudo apt-get upgrade</pre><p>2. Install the ssmtp utility.</p><pre>$ sudo apt-get install ssmtp</pre><p>3. After this, you need to configure the ssmtp service from the ssmtp.conf file at the /etc/ssmtp/ folder. (I’ve used vim, but you can use you editor of your choice)</p><pre>$ cd /etc/ssmtp/<br>$ sudo vim ssmtp.conf</pre><p>Fill the following fields in the conf file.</p><pre>root=sender@gmail.com<br>mailhub=smtp.gmail.com<br>AuthUser=sender@gmail.com<br>AuthPass=password<br>UseSTRTTLS=YES<br>FromLineOverride=YES</pre><p>After editing the above fields, your conf file must look like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/677/1*vlGdMLAEjZGMhfWeNwxvgw.png" /><figcaption>ssmtp.conf</figcaption></figure><p>4. Final and the most important step. Go to the <a href="https://myaccount.google.com/lesssecureapps">settings</a> of your gmail account, and allow<strong><em> Less secure app access</em></strong> of the email.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/952/1*YmUfZFKlS8KLtmEXISo4Pg.png" /><figcaption>allow access from gmail</figcaption></figure><p>Da..da..done!</p><blockquote>Let’s send messages</blockquote><p>There are multiple ways to send messages with this. It all depends on you, and how you want it to be sent.</p><h4>1. The Quickest of all :</h4><pre>$ echo &quot;Greetings from terminal&quot; | sendmail receiver@gmail.com</pre><blockquote>(This doesn’t conatin any subject. Just the most relevant message)</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JNxADxmcnaT4phYNH5xL6g.png" /><figcaption>email received at the other end</figcaption></figure><h4>2. A bit descriptive :</h4><pre>$ sendmail receiver@gmail.com<br>Subject: Hello Boss</pre><pre>Wanted to tell you, the job&#39;s done!<br>Get the party ready.</pre><blockquote>This is self-descriptive email</blockquote><blockquote>It’s important to give double new-lines between subject body of email</blockquote><blockquote>After you’re done with the body, press <em>Enter</em> and then <em>Ctrl-d. </em>This takes you out of the interactive shell and sends the email.</blockquote><h4>3. Full explanatory email :</h4><p>You can make an email file, which you wish to send.</p><pre>$ touch message<br>$ vim message</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/1*CVPdrfGVNzCDhD7ECJ3X2Q.png" /><figcaption>detailed email</figcaption></figure><pre>$ sendmail -t &lt; message<br>or<br>$ ssmtp -t &lt; message</pre><blockquote>I’ve saved the file with name <strong><em>message</em></strong><em>.</em></blockquote><blockquote>The -t option takes the arguments like To, Cc, etc from the message body itself.</blockquote><h3>Troubleshooting</h3><ol><li>With <strong><em>UseSTARTTLS=yes</em></strong> you can only set port to 587, and not to 465.</li><li>Errors like <strong><em>authorization falied</em></strong><em> </em>could be due to denied access to unsecure apps from gmail account. Refer final but most important step-4.</li><li>Multi-Factor authentication doesn’t allow to to grant access to less secure apps. So remove the MFA from your account.</li><li>Errors might occur due to parsing error in its To, From sections.</li></ol><h3><strong>Hacks : my fav</strong></h3><ol><li>You can send emails with differThent <strong><em>usernames</em></strong>. The username in the email, is system username.</li></ol><blockquote>I sent the quickest mail with root account, hence the username is set to root.</blockquote><p>2. Your <strong><em>gmail account password</em></strong> stored in the conf file in <strong><em>plain text</em></strong>. This could be fatal, but keep calm because the conf file is only visible to root.</p><p>3. This <strong>ssmtp</strong> utility can be very well configured with docker containers to notify you about its health status.</p><p>4. Instead of <strong><em>sendmail</em></strong>,<strong><em> </em></strong>you can also use <strong><em>ssmtp</em></strong> for sending mails.</p><p>5. The final steps gives less secure apps, to acess you email account. This is most important, for messaging. Hence, I personally suggest using an entirely new account with no personal data like photos, contacts, drive in it.</p><blockquote>Well, I’m quite sure google handles this pretty well, but who knows when…</blockquote><h4>Kudos! Now you won’t have to switch back-and-forth the gmail for simple messaging. More importantly, you can configure your system to act smart, and alert you everytime it detects something fishy, like DoS attack, unauthorized login attempts, etc.</h4><p>That’s it for this blog. If you’ve come uptill here, I am sure you found this interesting. Give it a clap and share it with your nerdy friends. Consider getting connected with me on <a href="https://www.linkedin.com/in/greyhatlinux/">linkedin</a> :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b638bd3dcc3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[An edge with Cloud Computing]]></title>
            <link>https://greyhatlinux.medium.com/an-edge-with-cloud-computing-de56f03cae2f?source=rss-83eea1b79e2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/de56f03cae2f</guid>
            <category><![CDATA[servers]]></category>
            <category><![CDATA[iaas]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[paas]]></category>
            <dc:creator><![CDATA[Vishal Gorai]]></dc:creator>
            <pubDate>Fri, 03 Jan 2020 12:39:44 GMT</pubDate>
            <atom:updated>2020-01-03T12:39:44.964Z</atom:updated>
            <content:encoded><![CDATA[<h4>Cheers, your computer got wings!</h4><p>Well, I’m not here to explain what Cloud Computing is, but to unfold what cloud computing has to offer you and your (no matter of what size of ) organisation. Its so in-trends from the last few years that it is rare if not impossible to find someone unschooled of Cloud Computing.</p><p>Still to give you a start, and to wrap up the definition is the best possible layman’s terms :</p><h4>A cloud computer is your very own computer that you can use anywhere and everywhere with the only requirement, internet .</h4><figure><img alt="A very illustrative picture" src="https://cdn-images-1.medium.com/max/661/1*zcPnPa_C0KrV15odeTnIKw.png" /><figcaption>cloud computing</figcaption></figure><p>To dive deeper in cloud computing I would recommend some very descriptive blogs out there on medium.</p><h3><strong>But wait, where is the cloud?</strong></h3><p>Haha, do not reitirate the question. Even a toddler can show you the clouds. Its <em>everywhere</em>, and at <em>all times</em>, its omnipresent. That is where the entire technology revolves.</p><h3><strong>Any other advantages of Cloud Computer ?</strong></h3><p>You name a luxury, and it provides you with it. Availability, flexibility, proper user access management, most imporantly security, cost efficiency, user friendly, organisation friendly (no matter how big or small),horizontal as well as vertical scalability, speed, monitoring capability, remote accessibility, and in the worst case of failures, it offers quick recoverability… Well, the advantages didn’t over, I’m out of words. It has to render so much, that now we’re addicted to the technology, and there’s no way going back.</p><blockquote>Disclaimer : In the whole article I keep talking about AWS services because I’ve used it sizably. But since these basic features gives cloud computing its essence, these would be a primary feature with all cloud platforms like GCP, IBM and all others.</blockquote><h3>Avalability</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*rNmGP2iAT7gD-0IhA6gejQ.png" /><figcaption>Availability Zones</figcaption></figure><p>One of the primacy of Cloud Computer is that is globally available rather than your study table. No matter from which corner of the world you acess it, it always remains in your radar. Not any kind of advertisement,but the above picture shows the availability zones of AWS. You can evidently see how evenly is covers all regoins of the planet, to make your system available from everywhere.</p><h3><strong>Scalability</strong></h3><p>Cloud computing is always efficiently scalable,also it bills you on are billed on pay-as-you-use model doting your costs. The AWS bills you on per-second basis for various services.</p><p>Just imagine a case, where you host a website and suddenly your website gains popularity. From 200 views per day it suddenly escalates 20,000 views a day. There it requires horizontal as well as vertical scaling of your infrastructure, and that too in quick time. This simply implies huge investment to expand your database and processing power. But till you do so, your site continuously crashes, and your customers remain frustrated. But, let us consider within 10 days, you have set-up everything,and all works fine. Again your website suffers a downfall and visits per-day dwindles to 100. All your increased infra renders stagnant and useless.</p><p>This is very efficiently handled by cloud providers, as they have enormous resources oh hold. Once your system requirement heighten, it provisions more resources automatically, in no time. Again, in off-days it swiftly pulls out extra resources provisioned during high-time. Thereby, it saves your investment, time, man-power and you get no excuse than to focus on productivity. And for all this you’re billed on per-second basis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/376/1*_Gjzv9SySs1SfRewsB9mZw.png" /><figcaption>auto scalaing</figcaption></figure><h3>Network speed</h3><p>The large Servers of AWS, GCP and others are connected through wires with huge gigabit bandwidth (yes gigabit and not megabit). This makes them lightening fast with data transmission. I won’t talk much about it, the visuals will themselves.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*MzngBXKUolidxrSMomWJpQ.png" /><figcaption>comparing home and cloud network speed.</figcaption></figure><p>The above one is free Linux Server rented from AWS with minimum capabilities (which is why the speed didin’t reach upto gigabits), and the lower one is my local machine’s terminal. Provisioning powerful instances can even reach upto a whooping 100 gigibit speed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*EbXk_pwD4bVitLCHEpMNgA.png" /></figure><p>Running the traceroute explains it all. Its evident that with cloud computers the latency hardly attains double digit, while my local system never reaches single digit. Well, I’m not at all exaggerating, I was on my best high-speed-network.</p><h3>Monitoring Capabilities</h3><p>Not only services power your systems, there are also provisions to monitor your machines in terms of so many factors. These have monitoring and management service that provides data and actionable insights for your machines, hybrid, and on-premises applications and infrastructure resources.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gwYIYvi3UNW3Xzn3t51mBA.png" /><figcaption>AWS cloudwatch</figcaption></figure><p>AWS CloudWatch enables you to monitor your complete stack (applications, infrastructure, and services) and leverage alarms, logs, and events data to take automated actions and reduce Mean Time to Resolution (MTTR).</p><h3>Other Benifits</h3><p>There are a variety of other advantages that cloud computing offers, but I found these the most basic ones that one should completely understand to migrate his system from an on-premise to the cloud.</p><h3>References :</h3><p>For deep knowledge on what is cloud computing and more of its core operations, here are my favs</p><ol><li><a href="https://medium.com/@vickynimbalkar/cloud-computing-for-beginners-ceb417658912">Vicky Nimbalkar</a></li><li><a href="https://medium.com/tech-batoora/introduction-to-cloud-computing-8017e884ee5e">Liya Roy</a></li></ol><h4>End Note</h4><p>Well, thanks for reading and cheers, you now have a headstart with cloud computing. Do like and share with your buddies over the cloud.</p><p>Have a great day!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=de56f03cae2f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Phases of an NMAP scan]]></title>
            <link>https://greyhatlinux.medium.com/phases-of-an-nmap-scan-e6491e081707?source=rss-83eea1b79e2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6491e081707</guid>
            <category><![CDATA[scan]]></category>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[reconnaissance]]></category>
            <category><![CDATA[nmap]]></category>
            <category><![CDATA[kali-linux]]></category>
            <dc:creator><![CDATA[Vishal Gorai]]></dc:creator>
            <pubDate>Fri, 20 Dec 2019 20:19:07 GMT</pubDate>
            <atom:updated>2019-12-20T20:32:11.576Z</atom:updated>
            <content:encoded><![CDATA[<h4>Step-by-step towards the network</h4><p><em>Hello readers, hope you all doing well.</em></p><p>For security researchers and hackers nmap must be their breakfast to start their day with. Well, given the diverse functionality and extreme flexibility, it certainly deserves its dominance.</p><figure><img alt="nmap logo" src="https://cdn-images-1.medium.com/max/225/1*e0PvOyJqlUEGd8h3WHWEnA.jpeg" /><figcaption>nmap logo</figcaption></figure><blockquote>Nmap (“Network Mapper”) is a free and open source utility for network discovery and security auditing. Many systems and network administrators also find it useful for tasks such as network inventory, managing service upgrade schedules, and monitoring host or service uptime.</blockquote><p>There are a good number of blogs enumerating nmap usage, scan types and other utilities. I’ll personally list them at the end. But I didn’t find any reagrding the phases an NMAP scan goes through. This blog will just take you through nmap scan phases with its outcome on the terminal. For futher research and contribution, I recommend you to go through the official website of <a href="https://nmap.org/">nmap</a>.</p><h3>1. Target Enumeration</h3><p>You must have been always scanning a particular IP, or single IP at a time. No offence, as 80% of the nmap users do so. But nmap is more to this. It can take combination of host DNS names, IP addresses, CIDR network notations. This is an essential step and cannot be skipped. But you make nmap’s task easy by simply providing an IP, so no further resolution is required.</p><p>Here, -sL switch comes very very handy to actually identify you targets. The stealth here is : It discovers its host without sending any packets to the target system, so this scan escapes IDS.</p><p>Supposing you need to pentest on Jharkhand government webiste(assuming you’ve all required written auth), mapping jharkhand.gov.in with -sL gives you a proper list of IPs under your jurisdiction.</p><pre>nmap -sL jharkhand.gov.in/29</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/884/1*dBD-RuMh0Zd8_dsX_Ka3mw.png" /><figcaption>-sL is very useful for planning targets and playing safe.</figcaption></figure><p><em>From the output, it is absolutely clear that you can only test on 112.133.209.139, because only that IP comes under jharkhand.gov.in with CIDR 29 (rest IPs are under some other organisations). Stay away from the other IP addresses as it may land you a handsome civil and criminal charge.</em></p><blockquote>Always perform -sL with CIDR 24 as it gives complete flexibility, though it may take very long time</blockquote><p>On further recon, I evaluated that 112.133.209.139 IP itself gives you acess to these sites :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/263/1*bND9dFz9JJbrZj2RERnN4Q.png" /><figcaption>subdomains with jharkhand.gov.in</figcaption></figure><h3>2. Host Discovery</h3><p>Scans begin by discovering which hosts are actually up (active) and thus are worth deeper investigaton. Nmap offers various host discovery techniques. Nmap offers many host discovery techniques, ranging from quick ARP requests to elaborate combinations of TCP, ICMP, and other types of probes This phase is run by default, though you can skip it (simply assume all target IPs are online) using the -Pn (no ping) option. To quit after host discovery, specify -sn</p><blockquote>The -sn switch is very useful, it only specifies whether the host is active or not. It allows light reconnaissance of a target network without attracting much attention.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*DbHbe_0LtnnmK5hCGKwptw.png" /><figcaption>-sn for one IP address</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/1*VNtn1_w5Dlb2xlrwJoz0PA.png" /><figcaption>-sn for host discovery from a total of 64 IP addresses.</figcaption></figure><blockquote>Knowing how many hosts are up is more valuable to attackers than the list provided by list scan of every single IP and host name . As shown above form the given CIDR a total of 64 address are there, but only 1 is active rest hosts are down, so it saves time os scanning the inactive IPs for the particular network range.</blockquote><p>Not only can nmap scan IPs it can also do with the domain names. It is capable of reverse DNS lookups for the input provided. This feature makes it more human friendly, as we tend to remember domain names more than their IP addresses. This runs by default, but it may be skipped with the -n (no DNS resolution), DNS can be slow even with Nmap’s built-in parallel stub resolver, this option reduces scanning times.</p><blockquote>Prefer using -n option for speedy results.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*XLww-Z5AdEF8H2aFNUakcg.png" /><figcaption>fast scan by removing reverse DNS lookup</figcaption></figure><h3>3. Port Scanning</h3><p>This is Nmap’s core operation. Probes are sent, and the responses (or non-responses) to those probes are used to classify remote ports into states such as open, closed, or filtered.</p><p>There are many mechanisms by which you can scan and confirm wich ports are open or closed. Sometimes, one scan technique can yield different results that the other due to various firewalls restrictions and filtering.</p><p><em>-sS</em>SYN Stealth port scan</p><p><em>-sT </em>TCP Connect port scan</p><p><em>-sT </em>UDP port scan</p><p><em>-sT </em>TCP ACK port scan</p><p><em>-p </em>Port or port range (eg. -p1–10000)</p><p><em>-p- For scanning all 65535 ports</em></p><p><em>-F scanning first 100 ports</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/820/1*OsCy8bP-BKlGuzYW5W3LAA.png" /><figcaption>an example of -sS SYN scan against 10.10.10.170</figcaption></figure><blockquote>Its always better to perform successive scans with multiple ways. Scanning all 65535 ports but it takes very very long, instead break it down into batches of thousands and run all simultaneoulsy.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/727/1*zroCXIRF7aX_j8sX-TAqQA.png" /><figcaption>time consumption increases upon scanning all 65635 ports. For a single IP scanning all 65535 ports would take about 40 minutes.</figcaption></figure><h3>4. Service and Version Detection</h3><p>If any ports are found to be open, Nmap may be able to determine what server software is running on the remote system. It does this by sending a variety of probes to the open ports and matching any responses against a database of thousands of more than 6,500 known service signatures. It is not a by-default service and is enabled with the -sV option.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/834/1*sBibiDtthr87ZW22g1iUIQ.png" /><figcaption>Output with and without service versions</figcaption></figure><h3>5. OS Detection</h3><p>If requested with the -O option, Nmap proceeds to Operating System (OS)detection. Different operating systems implement network standards in subtly different ways. By measuring these differences it is often possible to determine the operating system running on a remote host. Nmap matches responses to a standard set of probes against a database of more than a thousand known operating system responses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/1*mBMdzuv2tr_wP7hxYmme2A.png" /><figcaption>Probable OS of the host machine.</figcaption></figure><blockquote>Assigning -sV also does the job of OS Detection. Personally, I prefer <em>-sV more than </em><em>-O .</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/917/1*yl0dDW9n6LIEXSenD8F1tA.png" /><figcaption>-sV and -O shows almost equal results</figcaption></figure><h3>6. Traceroute</h3><p>Nmap contains an optimized traceroute implementation, enabled by the --traceroute option. It can find the network routes to many hosts in parallel, using the best available probe packets as determined by Nmap&#39;s previous discovery phases</p><blockquote>I work in virtual box, so my — traceroute doesn’t show up many nodes. But in main machine it should.</blockquote><h3>7. Script Scanning</h3><p>Most Nmap Scripting Engine (NSE) scripts run during this main script scanning phase, rather than the prescan and postscan phases. NSE is powered by the Lua programming language and a standard library designed for network information gathering. Scripts running during this phase generally run once for each target host and port number that they interact with. They commonly perform tasks such as detecting service vulnerabilities, malware discovery, collecting more information from databases and other network services, and advanced version detection. NSE is not an option by default. You request it by options switches such as --script or -sC.</p><h3>8. Output</h3><p>Nmap collects all the information it has gathered and writes it to the screen or to a file. Nmap can write output in several formats. Its default, human-readable format (interactive format) is usually presented in this book. Nmap also offers an XML-based output format, among others.</p><p>-O &lt;filename&gt; Produces output in normal format. No need of example, this is the format we’ve been following from the</p><p>-oX &lt;filename&gt;.xml This produces output in XML format.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/873/1*tNdZCRRGngp0lCy-Ks1VHg.png" /><figcaption>to get output in XML format</figcaption></figure><blockquote>The output in XML can be coverted to html by <em>xsltproc report.xml -o myreport.html</em> and then opening the new report in any browser.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/739/1*rgV_ABHClB1iv06mvXFEPw.png" /><figcaption>nmap output in XML format</figcaption></figure><blockquote>Disclaimer : Using Nmap against a target or network without explicit permission can be illegal under you jurisdiction and hence should therefore not be attempted. Please get authorisation before scanning any organisations network. Please read the following excerpt from nmap before hitting your terminal with nmap.</blockquote><h4>Thanking</h4><p>Thanks for your time. I hope you now know to recon an a target better than you friends. Well, I promised you to give you refernce to other very elaborative nmap blogs. Those are :</p><ol><li>by <a href="https://medium.com/@PrakhashS/concepts-behind-network-scanning-using-nmap-598df2a72ab9">Prakash Sivakumar</a></li></ol><p>2. by <a href="https://medium.com/@infosecsanyam/nmap-cheat-sheet-nmap-scanning-types-scanning-commands-nse-scripts-868a7bd7f692">Sanyam Chnawla</a></p><p>If you gained knowledge, and found it explanatory enough, do like and share among your hacker friends.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6491e081707" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>