<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ketan Pradhan on Medium]]></title>
        <description><![CDATA[Stories by Ketan Pradhan on Medium]]></description>
        <link>https://medium.com/@ketanpradhan?source=rss-1f75c30f3c2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*Xc4ZM5LdLOKdzUG7.</url>
            <title>Stories by Ketan Pradhan on Medium</title>
            <link>https://medium.com/@ketanpradhan?source=rss-1f75c30f3c2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 19:20:14 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ketanpradhan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Understanding Podman Volumes: Named Volumes vs Bind Mounts (And Why Rootless Users on RHEL Should…]]></title>
            <link>https://medium.com/@ketanpradhan/understanding-podman-volumes-named-volumes-vs-bind-mounts-and-why-rootless-users-on-rhel-should-277ee89ad498?source=rss-1f75c30f3c2------2</link>
            <guid isPermaLink="false">https://medium.com/p/277ee89ad498</guid>
            <category><![CDATA[podman-volumes]]></category>
            <category><![CDATA[bind-mount]]></category>
            <category><![CDATA[named-volumes]]></category>
            <category><![CDATA[rhel]]></category>
            <category><![CDATA[podman]]></category>
            <dc:creator><![CDATA[Ketan Pradhan]]></dc:creator>
            <pubDate>Fri, 30 May 2025 08:38:22 GMT</pubDate>
            <atom:updated>2025-05-30T08:38:22.962Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>podUnderstanding Podman Volumes: Named Volumes vs Bind Mounts (And Why Rootless Users on RHEL Should Care)</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CBOsGU1diw8fsYLjN1MOiw.png" /></figure><p>When working with <strong>Podman</strong> on <strong>RHEL-based systems</strong>, especially in <strong>rootless mode</strong>, volume mounting can lead to confusing permission issues if you’re not clear on the difference between <strong>named volumes</strong> and <strong>bind mounts</strong>.</p><p>After personally hitting these roadblocks while deploying an app as a non-root user, I decided to document this so others don’t waste hours debugging `Permission denied` errors.</p><p><strong>The Two Ways to Mount Data in Podman Compose</strong></p><p>Let’s break down the two most common methods for mounting data:</p><p><strong>1. Named Volumes (Managed by Podman)</strong></p><p>Create the volume:</p><pre>podman volume create app-logs</pre><p>Use it in your `podman-compose.yaml`:</p><pre>services:<br>  backend:<br>    image: myapp:latest<br>    volumes:<br>      - app-logs:/usr/src/app/logs:z<br><br>volumes:<br>  app-logs:</pre><p><strong>Pros</strong></p><p>* Podman manages the location (`~/.local/share/containers/storage/volumes/`).</p><p>* Data persists independently of the container lifecycle.</p><p>* Easy to inspect and backup (`podman volume inspect` or `export`).</p><p>* Works well with SELinux labels when using `:z`.</p><p><strong>Pitfall Avoided</strong></p><p>As a rootless user, Podman ensures correct permissions and ownership within the user namespace.</p><p><strong>2. Bind Mounts (Direct Host Path)</strong></p><pre>services:<br>  backend:<br>    image: myapp:latest<br>    volumes:<br>      - ./application.properties:/app/application.properties:z</pre><p><strong>Use Case</strong></p><p>Ideal for mounting configuration files or source code in development.</p><p><strong>Common Problem</strong></p><pre>Error: permission denied while trying to mount...</pre><p>This is <strong>especially common in rootless mode</strong> on RHEL where <strong>SELinux</strong> is enforcing.</p><p><strong>What Happens in Rootless Mode (And RHEL SELinux)</strong></p><p>RHEL enforces strict file labelling policies via SELinux. Even if your container runs as the same UID, <strong>the SELinux context must match</strong>.</p><p><strong>How to Fix It:</strong></p><p>Use the `:z` option to relabel the host path:</p><pre>volumes:<br>    - ./logs:/app/logs:z</pre><p>or</p><pre>bash<br>  sudo chcon -Rt svirt_sandbox_file_t ./logs</pre><p>If that doesn’t work in rootless mode, you’re safer using <strong>named volumes</strong>:</p><pre>podman volume create app-logs<br>podman unshare chown -R 1001:0 $(podman volume inspect app-logs -f &#39;{{.Mountpoint}}&#39;)</pre><p><strong>Real Example: How I Fixed It</strong></p><p>I had this setup:</p><pre>volumes:<br>  - ./logs:/usr/src/app/logs</pre><p>And got this error:</p><pre>Error: permission denied while trying to mount...</pre><p>After switching to:</p><pre>volumes:<br>  - app-logs:/usr/src/app/logs:z<br><br>volumes:<br>  app-logs:</pre><p>And running:</p><pre>podman volume create app-logs<br>podman unshare chown -R 1001:0 ~/.local/share/containers/storage/volumes/app-logs/_data</pre><p>✅ The container started smoothly. No SELinux errors. Data persisted.</p><p>If you’re running Podman <strong>as a non-root user</strong> on <strong>RHEL or CentOS</strong>, always prefer <strong>named volumes</strong> for persistent data and <strong>bind mounts</strong> only for controlled config files — and don’t forget the `:z` label!</p><p>Got questions or want a working example repo? Drop them in the comments!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=277ee89ad498" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Verifying Microsoft Azure AD JWT Tokens in Node.js]]></title>
            <link>https://medium.com/@ketanpradhan/verifying-microsoft-azure-ad-jwt-tokens-in-node-js-d38f54cbb791?source=rss-1f75c30f3c2------2</link>
            <guid isPermaLink="false">https://medium.com/p/d38f54cbb791</guid>
            <category><![CDATA[validate-microsfot-token]]></category>
            <category><![CDATA[verify-azure-token]]></category>
            <category><![CDATA[jwt]]></category>
            <dc:creator><![CDATA[Ketan Pradhan]]></dc:creator>
            <pubDate>Sun, 03 Nov 2024 06:18:10 GMT</pubDate>
            <atom:updated>2024-11-03T06:23:25.669Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ykdKK6ImTNG-_QpoUmU8GA.jpeg" /></figure><p>In this article, I will walk through how to verify JSON Web Tokens (JWT) issued by Microsoft Azure Active Directory (AD) in a Node.js environment. This approach is widely used for authenticating and authorizing users in applications, ensuring that tokens are valid and trustworthy.</p><h3>Why Verify JWT Tokens?</h3><p>When a client authenticates through Azure AD, it receives a signed JWT token containing claims about the user and the session. Verifying the token ensures that:</p><ul><li>The token was issued by a trusted source (Azure AD).</li><li>The token hasn’t been tampered with.</li><li>The token hasn’t expired.</li></ul><h3>Setting Up</h3><p>We’ll use two popular packages:</p><ol><li><strong>jsonwebtoken</strong>: A library for handling JWTs in Node.js.</li><li><strong>jwks-rsa</strong>: A package that retrieves and parses the public keys used by Azure AD to sign tokens.</li></ol><p>To get started, install the packages:</p><pre>npm install jsonwebtoken jwks-rsa</pre><h3>Retrieving Public Keys from Microsoft</h3><p>Azure AD signs JWT tokens with RSA private keys, and these tokens can be verified using the corresponding public keys. Microsoft provides these public keys in JSON Web Key Sets (JWKS) at a specific endpoint, which is usually in this format:</p><pre>https://login.microsoftonline.com/&lt;tenant_id&gt;/discovery/keys</pre><h3>Code Walkthrough</h3><p>Here’s the code for setting up our JWT verification.</p><pre>const jwt = require(&#39;jsonwebtoken&#39;);<br>const jwksClient = require(&#39;jwks-rsa&#39;);<br><br>// Configure the client to retrieve public keys from Microsoft<br>const client = jwksClient({<br>  jwksUri: &#39;https://login.microsoftonline.com/&lt;tenant-id&gt;/discovery/keys&#39; // Update with your JWKS URL<br>});<br><br>// Helper function to retrieve signing key from kid (key ID)<br>function getKey(header, callback) {<br>  client.getSigningKey(header.kid, (err, key) =&gt; {<br>    if (err) {<br>      return callback(err);<br>    }<br>    const signingKey = key.getPublicKey();<br>    callback(null, signingKey);<br>  });<br>}<br><br>// Function to verify token<br>function verifyToken(token) {<br>  return new Promise((resolve, reject) =&gt; {<br>    jwt.verify(token, getKey, { algorithms: [&#39;RS256&#39;] }, (err, decoded) =&gt; {<br>      if (err) {<br>        reject(`Token verification failed: ${err.message}`);<br>      } else {<br>        resolve(decoded); // token is valid<br>      }<br>    });<br>  });<br>}<br><br>// Usage example<br>const token = &#39;&lt;your-id_token-here&gt;&#39;; // use id_token<br>verifyToken(token)<br>  .then((decoded) =&gt; {<br>    console.log(&#39;Token is valid:&#39;, decoded);<br>  })<br>  .catch((error) =&gt; {<br>    console.error(error);<br>  });</pre><h3>Breaking Down the Code</h3><h4>Step 1: Configuring the JWKS Client</h4><p>We configure the jwksClient to use the jwksUri, which points to Azure AD&#39;s endpoint for public keys:</p><pre>const client = jwksClient({<br>  jwksUri: &#39;https://login.microsoftonline.com/&lt;tenant_id&gt;/discovery/keys&#39;<br>});</pre><h4>Step 2: Retrieving the Signing Key</h4><p>Each JWT header has a unique key ID (kid), allowing Azure AD to specify the key used to sign the token. We create a helper function, getKey, to retrieve this key from Azure AD:</p><pre>function getKey(header, callback) {<br>  client.getSigningKey(header.kid, (err, key) =&gt; {<br>    if (err) {<br>      return callback(err);<br>    }<br>    const signingKey = key.getPublicKey();<br>    callback(null, signingKey);<br>  });<br>}</pre><h4>Step 3: Verifying the JWT Token</h4><p>The verifyToken function takes a token as input, verifies it using the jsonwebtoken library, and resolves or rejects based on the outcome</p><pre>function verifyToken(token) {<br>  return new Promise((resolve, reject) =&gt; {<br>    jwt.verify(token, getKey, { algorithms: [&#39;RS256&#39;] }, (err, decoded) =&gt; {<br>      if (err) {<br>        reject(`Token verification failed: ${err.message}`);<br>      } else {<br>        resolve(decoded); // token is valid<br>      }<br>    });<br>  });<br>}</pre><h3>Testing the Code</h3><p>You can test the function by passing a valid JWT token (as shown in the usage example). If the token is valid, it will print the decoded token payload, otherwise it will log an error.</p><h3>Conclusion</h3><p>By following the steps above, you can reliably verify JWT tokens from Azure AD in your Node.js application. This setup provides an essential layer of security, ensuring that only legitimate tokens can access protected resources.</p><h3>Troubleshooting Tips</h3><ul><li>Ensure the jwksUri is correct and matches the tenant ID for the Azure AD issuing the token.</li><li>Confirm that the token is in RS256 format, as this is the algorithm used by Azure AD for signing tokens.</li><li>If you encounter issues with decoding, check that the token is not expired and was issued to the correct audience.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d38f54cbb791" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Practical Guide to Apache Kafka with Node.js: Code Examples]]></title>
            <link>https://medium.com/@ketanpradhan/a-practical-guide-to-apache-kafka-with-node-js-code-examples-329cc65be502?source=rss-1f75c30f3c2------2</link>
            <guid isPermaLink="false">https://medium.com/p/329cc65be502</guid>
            <category><![CDATA[zookeeper]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[kafka]]></category>
            <dc:creator><![CDATA[Ketan Pradhan]]></dc:creator>
            <pubDate>Tue, 17 Oct 2023 16:08:29 GMT</pubDate>
            <atom:updated>2023-10-17T16:08:59.856Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i6ppi4kfJGgTjLoQo2Z51A.jpeg" /></figure><p>In this article, we’ll dive into code examples that demonstrate how to work with Apache Kafka using Node.js. Apache Kafka is a powerful distributed streaming platform that’s widely used for building real-time data pipelines and streaming applications. If you’re already familiar with the basics of Kafka and want to see practical code examples, you’re in the right place. Let’s get started.</p><h3>Starting a ZooKeeper Container</h3><p><a href="https://zookeeper.apache.org/">ZooKeeper</a> is an essential component for managing distributed systems, including Apache Kafka. To start a ZooKeeper container, execute the following command:</p><pre>docker run -p 2181:2181 zookeeper</pre><p>Here’s what each part of the command does:</p><ul><li>docker run: This command is used to run Docker containers.</li><li>-p 2181:2181: This part of the command maps the host machine&#39;s port 2181 to the container&#39;s port 2181. ZooKeeper typically uses port 2181 for client connections.</li><li>zookeeper: This specifies the name of the Docker image to be used. When this command is executed, Docker pulls the ZooKeeper image from the Docker Hub if it&#39;s not already available locally.</li></ul><p>Starting the ZooKeeper container is the first step in creating a functional Kafka environment. ZooKeeper is used for coordination and management tasks within a Kafka cluster.</p><h3>Starting a Kafka Container</h3><p>Once you have ZooKeeper up and running, it’s time to start a Kafka container. The Kafka container is started with specific environment variables to configure its connection to ZooKeeper. Here’s the command:</p><pre>docker run -p 9092:9092 \<br>-e KAFKA_ZOOKEEPER_CONNECT=192.168.29.176:2181 \<br>-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://192.168.29.176:9092 \<br>-e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \<br>confluentinc/cp-kafka</pre><p>Let’s break down this command:</p><ul><li>docker run: This part of the command is the same as in the previous example; it&#39;s used to run Docker containers.</li><li>-p 9092:9092: This maps port 9092 on the host machine to port 9092 in the Kafka container. Port 9092 is the default port for Kafka brokers.</li><li>-e KAFKA_ZOOKEEPER_CONNECT=192.168.29.176:2181: Here, we specify the ZooKeeper connection details. Kafka relies on ZooKeeper for distributed coordination. You should replace 192.168.29.176 with the actual IP address of your ZooKeeper container.</li><li>-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://192.168.29.176:9092: This environment variable sets the advertised listeners for Kafka. The IP address here should match your Docker host&#39;s IP. It ensures that Kafka clients can connect to the broker correctly.</li><li>-e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1: This sets the replication factor for Kafka&#39;s internal offsets topic to 1. You can adjust this based on your replication needs.</li><li>confluentinc/cp-kafka: This specifies the Kafka Docker image to use. The image is pulled from the Docker Hub and is provided by Confluent Inc.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i6ppi4kfJGgTjLoQo2Z51A.jpeg" /></figure><p>Starting the Kafka container with these configurations allows you to have a Kafka broker up and running, ready to receive and manage topics and messages.</p><h3>Code Examples</h3><p>We’ll cover several key components and operations when working with Apache Kafka and Node.js. You can use these code examples as a reference for building your own Kafka-based applications.</p><h3>client.js: Configuring Kafka Client</h3><p>In this code example, we’ll configure a Kafka client using the kafkajs library. The client configuration is essential for connecting to the Kafka cluster and interacting with topics.</p><pre>const { Kafka } = require(&quot;kafkajs&quot;);<br><br>exports.kafka = new Kafka({<br>  clientId: &quot;my-app&quot;,<br>  brokers: [&quot;&lt;PRIVATE_IP&gt;:9092&quot;],<br>});</pre><h3>admin.js: Creating Kafka Topics</h3><p>Apache Kafka uses topics to organize and categorize messages. This code example shows how to create a Kafka topic using the Kafka Admin module.</p><pre>const { kafka } = require(&quot;./client&quot;);<br><br>async function init() {<br>  const admin = kafka.admin();<br>  console.log(&quot;Admin connecting...&quot;);<br>  admin.connect();<br>  console.log(&quot;Admin Connection Success...&quot;);<br><br>  console.log(&quot;Creating Topic [rider-updates]&quot;);<br>  await admin.createTopics({<br>    topics: [<br>      {<br>        topic: &quot;rider-updates&quot;,<br>        numPartitions: 2,<br>      },<br>    ],<br>  });<br>  console.log(&quot;Topic Created Success [rider-updates]&quot;);<br><br>  console.log(&quot;Disconnecting Admin..&quot;);<br>  await admin.disconnect();<br>}<br><br>init();</pre><h3>producer.js: Building a Kafka Producer</h3><p>Producers are responsible for sending messages to Kafka topics. This code example demonstrates how to create a Kafka producer that sends rider location updates.</p><pre>const { kafka } = require(&quot;./client&quot;);<br>const readline = require(&quot;readline&quot;);<br><br>const rl = readline.createInterface({<br>  input: process.stdin,<br>  output: process.stdout,<br>});<br><br>async function init() {<br>  const producer = kafka.producer();<br><br>  console.log(&quot;Connecting Producer&quot;);<br>  await producer.connect();<br>  console.log(&quot;Producer Connected Successfully&quot;);<br><br>  rl.setPrompt(&quot;&gt; &quot;);<br>  rl.prompt();<br><br>  rl.on(&quot;line&quot;, async function (line) {<br>    const [riderName, location] = line.split(&quot; &quot;);<br>    await producer.send({<br>      topic: &quot;rider-updates&quot;,<br>      messages: [<br>        {<br>          partition: location.toLowerCase() === &quot;north&quot; ? 0 : 1,<br>          key: &quot;location-update&quot;,<br>          value: JSON.stringify({ name: riderName, location }),<br>        },<br>      ],<br>    });<br>  }).on(&quot;close&quot;, async () =&gt; {<br>    await producer.disconnect();<br>  });<br>}<br><br>init();</pre><h3>consumer.js: Developing Kafka Consumers</h3><p>Consumers subscribe to Kafka topics and process the messages. This code example shows how to create a Kafka consumer with a specified group ID.</p><pre>const { kafka } = require(&quot;./client&quot;);<br>const group = process.argv[2];<br><br>async function init() {<br>  const consumer = kafka.consumer({ groupId: group });<br>  await consumer.connect();<br><br>  await consumer.subscribe({ topics: [&quot;rider-updates&quot;], fromBeginning: true });<br><br>  await consumer.run({<br>    eachMessage: async ({ topic, partition, message, heartbeat, pause }) =&gt; {<br>      console.log(<br>        `${group}: [${topic}]: PART:${partition}:`,<br>        message.value.toString()<br>      );<br>    },<br>  });<br>}<br><br>init();</pre><p>These code examples provide a practical foundation for working with Apache Kafka and Node.js. You can use them as building blocks for your Kafka-based projects. Whether you’re building real-time analytics, log processing, or any other streaming application, Apache Kafka is a robust solution, and Node.js makes it accessible and easy to work with.</p><h3>Running Locally</h3><p>Run Multiple Consumers</p><pre>node consumer.js &lt;GROUP_NAME&gt;</pre><p>Create Producer</p><pre>node producer.js<br>&gt; tony south<br>&gt; tony north</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=329cc65be502" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Secure Your Git Repository with Gitleaks and Pre-commit Hooks]]></title>
            <link>https://medium.com/@ketanpradhan/secure-your-git-repository-with-gitleaks-and-pre-commit-hooks-5f37bb03429b?source=rss-1f75c30f3c2------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f37bb03429b</guid>
            <category><![CDATA[git]]></category>
            <category><![CDATA[gitlab]]></category>
            <category><![CDATA[gitleaks]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[pre-commit]]></category>
            <dc:creator><![CDATA[Ketan Pradhan]]></dc:creator>
            <pubDate>Sat, 26 Aug 2023 12:40:26 GMT</pubDate>
            <atom:updated>2023-08-26T12:47:07.613Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/668/1*Ben-0Jp8xsszbJR1JS9q8g.png" /></figure><p>Introduction:<br>Version control systems like Git have become an integral part of modern software development, enabling teams to collaborate efficiently and track changes to their codebase. However, ensuring that sensitive information, such as passwords or API keys, doesn’t end up in your repository is crucial for maintaining the security of your projects. In this article, we’ll explore how to use the `gitleaks` tool along with `pre-commit` hooks to automatically detect and prevent sensitive information from being committed to your Git repository.</p><p>### Prerequisites:<br>- Basic understanding of Git and version control<br>- Familiarity with command-line tools</p><p><strong>What is Gitleaks?</strong></p><p>Gitleaks is an open-source tool designed to scan your Git repository for sensitive information, such as passwords, API keys, and other confidential data. By integrating Gitleaks into your development workflow, you can proactively prevent accidental exposure of sensitive information within your codebase.</p><p><strong>Setting Up Pre-commit Hooks:</strong></p><p>Pre-commit is a framework for managing and maintaining pre-commit hooks in your Git repository. These hooks are small scripts that run before each commit to ensure that certain conditions are met before the commit is allowed. In our case, we’ll use a pre-commit hook to run Gitleaks and identify potential security issues.</p><p>Here’s how you can set up a pre-commit hook for Gitleaks:</p><p>1. Create a `.pre-commit-config.yml` file in the root directory of your Git repository.</p><p>2. Add the following content to the file:</p><pre>repos:<br>  - repo: https://github.com/gitleaks/gitleaks<br>    rev: v8.18.0  # Specify the desired version of Gitleaks<br>    hooks:<br>      - id: gitleaks</pre><p>3. Install the `pre-commit` tool if you haven’t already:</p><pre>pip install pre-commit</pre><p>4. Install the pre-commit hook in your repository:</p><pre>pre-commit install</pre><p><strong>How the Setup Works:</strong></p><p>When you run `git commit`, the pre-commit hook configured in the `.pre-commit-config.yml` file will execute. In this case, the hook will invoke Gitleaks to scan the changes you’re about to commit for any sensitive information. If Gitleaks detects any potential issues, the commit will be rejected, and you’ll be notified of the problem.</p><p><strong>Benefits of Using Gitleaks and Pre-commit:</strong></p><p>- **Early Detection of Sensitive Information:** Gitleaks scans the changes before they are committed, allowing you to catch sensitive information before it enters your repository.</p><p>- **Automated Security Checks:** Pre-commit hooks automate the process of running security checks, reducing the risk of human error.</p><p>- **Team Collaboration:** By enforcing security checks at the commit stage, your entire development team benefits from a consistent security workflow.</p><p><strong>Conclusion:</strong></p><p>Integrating Gitleaks with pre-commit hooks is a simple yet effective way to enhance the security of your Git repository. By detecting and preventing the accidental inclusion of sensitive information, you can maintain the confidentiality of your codebase and minimize the potential impact of security breaches. Remember that security is an ongoing process, and tools like Gitleaks and pre-commit hooks can play a crucial role in bolstering your development practices.</p><p>Give it a try in your projects and start enjoying the benefits of automated security checks!</p><p>Happy coding securely!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f37bb03429b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reading a large files in nodejs]]></title>
            <link>https://medium.com/@ketanpradhan/reading-a-large-files-in-nodejs-39284ef797f4?source=rss-1f75c30f3c2------2</link>
            <guid isPermaLink="false">https://medium.com/p/39284ef797f4</guid>
            <category><![CDATA[chunk]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[filereader]]></category>
            <category><![CDATA[stream]]></category>
            <dc:creator><![CDATA[Ketan Pradhan]]></dc:creator>
            <pubDate>Sat, 28 Jan 2023 10:47:31 GMT</pubDate>
            <atom:updated>2023-01-28T12:44:17.924Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/680/1*VxmblrSlwC4txU-onz2mvA.jpeg" /></figure><p>Node.js is a powerful platform for building scalable and performant applications. However, when it comes to handling large files, it can be a bit of a challenge. The traditional approach of reading an entire file into memory can quickly lead to memory issues, especially when dealing with files that are several gigabytes in size. In this article, we will explore a few strategies for handling large files in Node.js using streams, chunking, and asynchronous operations.</p><p>Streams</p><p>One of the most efficient ways to handle large files in Node.js is to use streams. Streams allow you to read and write data in small chunks, rather than reading the entire file into memory at once. The built-in fs module provides a createReadStream() method that can be used to create a read stream for a large file. Here is an example of how to use a read stream to process a large file:</p><pre>const fs = require(&#39;fs&#39;);<br><br>const readStream = fs.createReadStream(&#39;largeFile.txt&#39;);<br><br>readStream.on(&#39;data&#39;, chunk =&gt; {<br>    // process the data chunk<br>    console.log(chunk.toString());<br>});<br><br>readStream.on(&#39;end&#39;, () =&gt; {<br>    console.log(&#39;file has been read completely&#39;);<br>});</pre><p>In this example, we created a read stream for a large file named largeFile.txt. The data event is emitted whenever a new chunk of data is available. We can process this chunk of data as it comes in, rather than waiting for the entire file to be read into memory. The end event is emitted when the stream has finished reading the file.</p><p>Chunking</p><p>Another strategy for handling large files is to split the file into smaller chunks and process them one at a time. This approach is particularly useful when the file is too large to be processed in a single pass. For example, you can use the split-file package to split a large file into smaller chunks and process each chunk separately.</p><pre>const splitFile = require(&#39;split-file&#39;);<br><br>splitFile.splitFileBySize(&#39;10mb.txt&#39;, 1024 * 1024 * 1) // 1MB<br>    .then((chunks) =&gt; {<br>    chunks.forEach((chunk, idx) =&gt; {<br>      console.log(`Chunk ${idx + 1}:`);<br>      console.log(chunk);<br>      //perforn the read operation<br>      fs.unlink(chunk, (err) =&gt; {<br>        if (err) throw err;<br>        console.log(`Chunk ${idx + 1} deleted`);<br>      });<br>    });<br>  })</pre><p>In this example, we used the splitFileBySize() method from the split-file package to split the large file into smaller chunks of 1MB each, logs the contents of each chunk to the console, and then deletes each chunk using the fs.unlink() function.</p><p>Asynchronous Operations</p><p>Asynchronous operations are another way to handle large files in Node.js. If your file processing logic is I/O-bound, it’s better to use async/await or callbacks to avoid blocking the event loop.</p><pre>const fs = require(&#39;fs&#39;);<br><br>const readLargeFile = async (file) =&gt; {<br>    try {<br>        const data = await fs.promises.readFile(file);<br>        console.log(data.toString());<br>    } catch (err) {<br>        console.error(err);<br>    }<br>};<br><br>readLargeFile(&#39;largeFile.txt&#39;);</pre><p>In this example, we used the fs.promises.readFile() method to read the file asynchronously. This method returns a promise that is resolved with the contents of the file. By using the await keyword, we can wait for the data to get processed.</p><p>Github : <a href="https://github.com/ketanbgm">https://github.com/ketanbgm</a></p><p>Sample code : <a href="https://github.com/ketanbgm/read-large-files-in-nodejs">https://github.com/ketanbgm/read-large-files-in-nodejs</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=39284ef797f4" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>