<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Tech with Harry on Medium]]></title>
        <description><![CDATA[Stories by Tech with Harry on Medium]]></description>
        <link>https://medium.com/@techwithharry?source=rss-2046c51afa0b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*JC7X_MbelAHQsAwyVzUGwA.png</url>
            <title>Stories by Tech with Harry on Medium</title>
            <link>https://medium.com/@techwithharry?source=rss-2046c51afa0b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 21:32:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@techwithharry/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Are open-source NPM packages always secure to use?]]></title>
            <link>https://systemweakness.com/are-open-source-npm-packages-always-secure-and-safe-to-use-215e9d3902be?source=rss-2046c51afa0b------2</link>
            <guid isPermaLink="false">https://medium.com/p/215e9d3902be</guid>
            <category><![CDATA[npm]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[dependency-management]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Tech with Harry]]></dc:creator>
            <pubDate>Sat, 01 Apr 2023 20:55:20 GMT</pubDate>
            <atom:updated>2023-04-03T15:32:19.895Z</atom:updated>
            <content:encoded><![CDATA[<h4>Node.js has a rich ecosystem of open-source packages available through the npm registry. These packages provide developers with pre-built functionalities that can be easily incorporated into their applications. However, many of these packages may contain security vulnerabilities that can be exploited by attackers</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GdCM-pLCJxLjd2eIHDw32Q.png" /></figure><h3>Introduction</h3><p>Node.js is a widely used platform for building web applications, but like any software platform, it is not immune to security vulnerabilities. In this blog post, we will discuss more about the vulnerabilities of open-source packages of Node.js on NPM.</p><h3>What is NPM?</h3><blockquote><strong>npm</strong> is the world’s largest software registry. Open source developers from every continent use <strong>npm</strong> to share and borrow packages, and many organizations use <strong>npm</strong> to manage private development as well. — <a href="https://docs.npmjs.com/about-npm">About npm | npm Docs (npmjs.com)</a></blockquote><p>Software Developers can install the dependency packages via npm command in terminal. All the required packages will also be downloaded as they are needed for the main packages. Developers can also search for a package on NPM registry public link <a href="https://www.npmjs.com/">https://www.npmjs.com/</a></p><h3>The security risk with NPM packages</h3><p>Due to the fact that NPM is an open-source platform that anyone can easily publish their own packages, it brings concerns about insecure dependencies. Developers must be very careful when installing any packages from NPM or other Nodejs package management tools.</p><p>Attackers can easily exploit the vulnerabilities within the packages to execute arbitrary code to steal important information.</p><h4>“node-forge” package — Improper Verification of Cryptographic Signature</h4><p>One recent example of a risky npm package is the <a href="https://www.npmjs.com/package/node-forge">node-forge — npm (npmjs.com)</a> package. This package provides a set of cryptographic utilities for Node.js applications, including functions for generating and verifying digital signatures.</p><p>Early in 2021, a flaw in a package was found that made it possible for attackers to get around signature verification procedures. This was caused by an issue with the way the package handled signature encoding. Attackers might therefore provide a signature that was particularly created and would be recognised as authentic by the application.</p><h4>“event-stream” package — The event-stream incident</h4><p>Another recent example of a risky npm package is the <a href="https://www.npmjs.com/package/event-stream">event-stream — npm (npmjs.com)</a> package. This package is a popular utility for working with Node.js streams and is used by many other packages as a dependency.</p><p>A malicious attacker had access to the “event-stream” package in 2018 and was able to introduce harmful code into it. The software was created with the intention of stealing users’ installed Bitcoin wallets. Because the original package maintainer had relinquished control of the package, a new maintainer was able to acquire access and insert the malicious code, making the attack effective. More details can be found in this blog from <a href="https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident">npm Blog Archive: Details about the event-stream incident (npmjs.org)</a></p><h3>How to mitigate the risk</h3><p>To mitigate the risk of insecure dependencies, developers should take the following steps</p><ol><li><strong>Verify the source code of dependencies</strong>: Before using any packages, developers should thoroughly check the source code to make sure it doesn’t contain any harmful code or known vulnerabilities by reading the official package’s documentation, version history, and the source code itself.</li><li><strong>Use well-known and trusted packages</strong>: Only install packages from trusted sources or created by authors who have a good reputation in the community. We should also check the package’s download statistics and community feedback to ensure that it is widely used and has a good track record of being security vulnerability free.</li><li><strong>Keep dependencies up-to-date</strong>: Developers should update their dependencies often to make sure that any known vulnerabilities are addressed. They should properly test their application before releasing any updates and be mindful of any breaking changes that might happen while updating dependencies list.</li><li><strong>Use security tools:</strong> Developers can use security tools like npm audit to scan their dependencies for vulnerabilities.</li></ol><ul><li><strong>npm audit</strong>: “<em>The </em><a href="https://docs.npmjs.com/cli/audit"><em>npm audit command</em></a><em> submits a description of the dependencies configured in your package to your default registry and asks for a report of known vulnerabilities. </em><em>npm audit checks direct dependencies, </em><em>devDependencies, </em><em>bundledDependencies, and </em><em>optionalDependencies, but </em><strong><em>does not check</em></strong><em> </em><em>peerDependencies.” — </em>Auditing package dependencies for security vulnerabilities</li><li><strong>Snyk</strong>: Snyk is a cloud solution that offers for Nodejs applications automated security testing and monitoring tools. It also helps to find and address vulnerabilities in Kubernetes applications, containers, and dependencies. More details can be found here <a href="https://snyk.io/">Snyk | Developer security | Develop fast. Stay secure. | Snyk</a>.</li><li><strong>NSP</strong>: Node Security Project (NSP) provides a command-line tool that can scan an application’s dependencies for known security issues. It can also integrate with build systems like Jenkins and Travis CI. More details can be found here <a href="https://docs.codeclimate.com/docs/nodesecurity">Node Security Project (nsp) (codeclimate.com)</a>.</li><li><strong>Dependabot</strong>: Dependabot is a tool of GitHub that automatically updates an application’s dependencies and alerts developers to any security issues. It can be integrated with GitHub and other source code management systems. More details can be found here <a href="https://github.com/dependabot">Dependabot (github.com)</a>.</li><li><strong>Retire.js</strong>: A CLI tool that can scan all application’s JavaScript dependencies for known vulnerabilities and output it to the users. The GitHub source link is here <a href="https://github.com/retirejs/retire.js/">RetireJS/retire.js: scanner detecting the use of JavaScript libraries with known vulnerabilities. Can also generate an SBOM of the libraries it finds. (github.com)</a>.</li></ul><h3>Conclusion</h3><p><strong>NPM</strong> packages can offer a variety of advantages, including reduced costs and development time. It is crucial to remember that not all packages are secure to use and that some could have serious security flaws or be vulnerable to malicious attacks. Developers must therefore do extensive research and take all essential safety steps before integrating any third-party code into their projects.</p><h3>Resources</h3><p><em>Auditing package dependencies for security vulnerabilities | npm Docs</em>. (n.d.). Auditing Package Dependencies for Security Vulnerabilities | Npm Docs. <a href="https://docs.npmjs.com/auditing-package-dependencies-for-security-vulnerabilities">https://docs.npmjs.com/auditing-package-dependencies-for-security-vulnerabilities</a></p><p><em>node-forge vulnerabilities | Snyk</em>. (n.d.). Find Detailed Information and Remediation Guidance for Vulnerabilities. <a href="https://security.snyk.io/">https://security.snyk.io/</a></p><p><em>A post-mortem of the malicious event-stream backdoor | Snyk</em>. (2018, December 6). Snyk. <a href="https://snyk.io/blog/a-post-mortem-of-the-malicious-event-stream-backdoor/">https://snyk.io/blog/a-post-mortem-of-the-malicious-event-stream-backdoor/</a></p><p><em>npm Blog Archive: Details about the event-stream incident</em>. (n.d.). Npm Blog Archive: Details About the Event-stream Incident. <a href="https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident">https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident</a></p><p>If you find my blog interesting, please consider following it for more updates and insights contents at <a href="https://medium.com/u/2046c51afa0b">Tech with Harry</a>. Thank you!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=215e9d3902be" width="1" height="1" alt=""><hr><p><a href="https://systemweakness.com/are-open-source-npm-packages-always-secure-and-safe-to-use-215e9d3902be">Are open-source NPM packages always secure to use?</a> was originally published in <a href="https://systemweakness.com/">System Weakness</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Files Uploader CLI with Nodejs, Typescript, and AWS S3 [Part 2/2]]]></title>
            <link>https://medium.com/@techwithharry/files-uploader-cli-with-nodejs-typescript-and-aws-s3-part-2-2-bf0518c51888?source=rss-2046c51afa0b------2</link>
            <guid isPermaLink="false">https://medium.com/p/bf0518c51888</guid>
            <category><![CDATA[cli]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[aws-s3]]></category>
            <dc:creator><![CDATA[Tech with Harry]]></dc:creator>
            <pubDate>Thu, 30 Mar 2023 20:30:01 GMT</pubDate>
            <atom:updated>2023-03-31T14:52:37.904Z</atom:updated>
            <content:encoded><![CDATA[<h4>An interactive CLI application to help uploading file or folder to AWS S3</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YpLHxOAWoShUo0WGaDe0Ug.jpeg" /></figure><h3>Introduction</h3><p>Hi folks, in the first part (<a href="https://medium.com/@techwithharry/files-uploader-cli-with-nodejs-typescript-and-aws-s3-part-1-2-86fc29e919ad">https://medium.com/@techwithharry/files-uploader-cli-with-nodejs-typescript-and-aws-s3-part-1-2-86fc29e919ad</a>), we already finished more than 70% of the application where we can start uploading a file to a specific folder. However, there are still some challenges that we need to solve in this part 2 such as:</p><ul><li>Upload a folder instead of just a file</li><li>Create a new folder while uploading</li><li>Creating a new bucket and uploading a file or folder to that newly created folder as well.</li></ul><p>So let’s get started!</p><h3>Getting Started</h3><h4>1. Upload a folder instead of just a file</h4><p>We need to add one more helper function in the file aws.ts that handles the general logic to upload a folder of files. In addition, sometimes if we also want to upload a folder to S3, we want to be able to choose if we want to keep the root/parent folder or not. Thus, it’s better to add another prompt question to ask if we want to keep the parent folder or not.</p><p>Add to then end of file aws.ts with a new helper function uploadToS3 as below</p><pre>// aws.ts<br><br><br>// Upload a folder to S3<br>export const uploadToS3 = (<br>  bucketName: string,<br>  uploadingPath: string,<br>  s3UploadingPath: string,<br>  keepParentFolder: boolean = false<br>) =&gt; {<br>  return new Promise(async (resolve) =&gt; {<br>    const stats = fs.statSync(uploadingPath);<br><br>    // Check if this is a directory or not<br>    if (stats.isDirectory()) {<br>      if (keepParentFolder) {<br>        const currentFolder = path.basename(uploadingPath);<br>      }<br><br>      const files = fs.readdirSync(uploadingPath);<br>      const result: Record&lt;string, {}&gt; = {};<br>      for (const file of files) {<br>        const filePath = path.join(uploadingPath, file);<br>        const uploadingResult = await putObject(<br>          bucketName,<br>          filePath,<br>          keepParentFolder<br>            ? path.join(s3UploadingPath, path.basename(uploadingPath))<br>            : s3UploadingPath<br>        );<br>        result[filePath] = uploadingResult;<br>      }<br>      resolve(result);<br>    } else {<br>      resolve(putObject(bucketName, uploadingPath, s3UploadingPath));<br>    }<br>  });<br>};</pre><p>In inquirer.ts add a basic prompt function to ask for a confirm question about keeping the parent folder and export it.</p><pre>// inquirer.ts<br><br>// Prompt asking for flatten folder choice<br>export const promptFlattenFolderChoiceQuestion = (): Promise&lt;{<br>  keepParentFolder: boolean;<br>}&gt; =&gt; {<br>  return new Promise&lt;{ keepParentFolder: boolean }&gt;((resolve) =&gt; {<br>    inquirer<br>      .prompt([<br>        {<br>          type: &quot;confirm&quot;,<br>          name: &quot;keepParentFolder&quot;,<br>          message: chalk.green(&quot;Do you want to keep parent folder?&quot;),<br>        },<br>      ])<br>      .then(resolve);<br>  });<br>};</pre><p>In index.ts add this below block of code at the end of the main function, outside the while loop to ask user to confirm keeping parent folder.</p><pre>// index.ts<br><br>...<br>  while (true) {<br>    ....<br>  }<br><br>  // NOTE: Add these two lines below<br>  const { keepParentFolder } = await promptFlattenFolderChoiceQuestion();<br>  await uploadToS3(selectedBucket, filepath, selectedFolder, keepParentFolder);<br>}<br><br>main();</pre><p>Try to run npm start &lt;path-to-a-folder&gt; and here is an example of the result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/937/1*px3yATgQ6vAiZsNMsVLiTA.png" /></figure><p>As you can see from the example above, the files are uploaded correctly to our selecting folder on S3 and we are able to select whether to keep the parent folder or not. Sweet!</p><h4>2. Add the ability to create a new folder on the fly</h4><p>The next target is to give the user the ability to create a new folder while uploading the file/folder. A folder in AWS S3 is just a <strong>prefix</strong>, so this can be done easily by prepending the folder name to the filename, this logic is the same as what we already did in the above section to keep the parent folder.</p><p>Add a new prompt helper function to ask for folder name under file inquirer.ts . We also prepare ahead for the type argument with both folder and bucket type so that we can reuse this for the next section.</p><pre>// inquirer.ts<br><br>// Prompt asking for a name<br>export const promptNameQuestion = (<br>  type: &quot;folder&quot; | &quot;bucket&quot;<br>): Promise&lt;{<br>  name: string;<br>}&gt; =&gt; {<br>  return new Promise&lt;{ name: string }&gt;((resolve) =&gt; {<br>    inquirer<br>      .prompt([<br>        {<br>          type: &quot;text&quot;,<br>          name: &quot;name&quot;,<br>          message: chalk.green(`What is the new ${type} name?`),<br>        },<br>      ])<br>      .then(resolve);<br>  });<br>};</pre><p>Add this block of code in file index.ts under the if statement for NEW_FOLDER_CHOICE . This block of code asks for the user prompt to enter a new folder name (no blank folder allowed) and configure the folder path on AWS S3.</p><pre>// index.ts<br>...<br>    // NOTE: Add the below code<br>    } else if (folder === NEW_FOLDER_CHOICE) {<br>      let folderName: string = &quot;&quot;;<br><br>      // Keep asking till user input the folder name<br>      while (!folderName) {<br>        folderName = (await promptNameQuestion(&quot;folder&quot;)).name;<br>      }<br>      selectedFolder = prefix + folderName + &quot;/&quot;;<br>      break;<br>    } else {<br>...</pre><p>Here is an example of creating a new folder on the fly</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*3XxDa16UXn73962vsTgl5w.png" /><figcaption>Example of creating a new folder on the fly</figcaption></figure><p>As we already know folder on AWS S3 is just a prefix, so we can create subfolders by having the separator / . For example, the below screenshot displays the newly created folder name study/coding/nodejs . AWS S3 will understand to create 3 subfolders study , coding , and nodejs respectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*EjD0s7XPpm-KwSAvqTxBDw.png" /><figcaption>Example of creating subfolders on the fly</figcaption></figure><h4>3. Create a new bucket</h4><p>Firstly, let’s add the helper function on aws.ts to help create a new bucket.</p><pre>// aws.ts<br><br>// List all buckets in S3<br>export const createBucket = (bucketName: string) =&gt; {<br>  return new Promise&lt;{ error: number | string }&gt;((resolve) =&gt; {<br>    s3.createBucket({ Bucket: bucketName }, (err: AWSError) =&gt; {<br>      if (err &amp;&amp; err.message) {<br>        console.log(chalk.red(err.message));<br>        return resolve({ error: err.message });<br>      } else {<br>        console.log(<br>          chalk.green(`Successfully created new bucket s3://${bucketName}`)<br>        );<br>        return resolve({ error: 0 });<br>      }<br>    });<br>  });<br>};</pre><p>Then in the index.ts file, add the code inside the if statement that selects NEW_BUCKET_CHOICE . This code just basically keeps asking for a new bucket name and try to create that bucket until successful. Because we may enter an existing name that is not globally unique or not follows S3 bucket name policies such as the name length, including / or containing uppercase characters. You can read more here (<a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html">https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html</a>).</p><pre>// index.ts<br><br>...<br>  // NOTE: Add the below code<br>  if (bucket === NEW_BUCKET_CHOICE) {<br>    while (!selectedBucket) {<br>      let newBucketName = &quot;&quot;;<br>      while (!newBucketName) {<br>        newBucketName = (await promptNameQuestion(&quot;bucket&quot;)).name;<br>      }<br><br>      // Create new bucket<br>      const { error } = await createBucket(newBucketName);<br>      if (!error) {<br>        selectedBucket = newBucketName;<br>      }<br>    }<br>  } else {<br>...</pre><p>Try to run the application and here is the example result. As you can see from the screenshot below, I intentionally try to create a bucket name abc123 and the program outputs this bucket already existed, and therefore an error is logged to the console. Therefore, it asks again for the bucket name you want to create again until it can successfully create it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*__TkXxXOueHNgBgibBZTAA.png" /><figcaption>An example of creating a new bucket on the fly</figcaption></figure><p>If you follow up till now, congratulation, you did a great job. There are still many rooms to extend this CLI application to serve more purposes such as:</p><ul><li>Ability to multi-upload, if the file size is large (this is recommended by AWS).</li><li>Ability to display files as well in case the user wants to override</li><li>Ability to download files/folders, we sometimes just do not want to upload only.</li><li>Ability to go back on selection because we may select the wrong folder.</li><li>Ability to ignore a set of files that could be predefined in .gitignore</li></ul><p>That’s it, hope you can have a good time building, learning, and having fun with this simple project. You can find the GitHub repository of this project here <a href="https://github.com/hnngo/upload-tool-s3">https://github.com/hnngo/upload-tool-s3</a>.</p><p>If you find my blog interesting, please consider following it for more updates and insights at <a href="https://medium.com/u/2046c51afa0b">Tech with Harry</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bf0518c51888" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Files Uploader CLI with Nodejs, Typescript, and AWS S3 [Part 1/2]]]></title>
            <link>https://medium.com/@techwithharry/files-uploader-cli-with-nodejs-typescript-and-aws-s3-part-1-2-86fc29e919ad?source=rss-2046c51afa0b------2</link>
            <guid isPermaLink="false">https://medium.com/p/86fc29e919ad</guid>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[aws-s3]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[cli]]></category>
            <category><![CDATA[nodejs]]></category>
            <dc:creator><![CDATA[Tech with Harry]]></dc:creator>
            <pubDate>Thu, 30 Mar 2023 20:28:16 GMT</pubDate>
            <atom:updated>2023-03-31T01:48:42.291Z</atom:updated>
            <content:encoded><![CDATA[<h4>An interactive CLI application to help uploading file or folder to AWS S3</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YpLHxOAWoShUo0WGaDe0Ug.jpeg" /></figure><h3>Introduction</h3><p>Hi folks, let&#39;s write some code today!</p><p>Sometimes, at the end of a working day, we want to upload our working documents to cloud storage such as Google Drive, or OneDrive, and it is very easy to do so by just drag and drop files/folders to those platforms.</p><p>How about AWS S3? AWS has already provided a very great CLI package aws-cli to support uploading files to AWS S3, then why do we still need this application?</p><pre>aws s3 sync &lt;local_folder&gt; s3://&lt;bucket_name&gt;/&lt;remote_folder&gt; [--options]</pre><p>The above command requires the mentioned bucket and folder to exist before we upload, or else we have to create it beforehand. This is sometimes considered to be annoying for a late-to-dinner critical time.</p><p>And more than that, we have to remember the AWS CLI command to create the bucket and upload the bucket. Developers do not like to remember everything, so a CLI application with full <strong>INTERACTIVE</strong> actions would be a nicer solution for this situation.</p><p>Here is a sneak peek of the finishing application, if you found this interesting, then let’s build it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*QJEKMTacKe7mRA4NGowmOg.gif" /><figcaption>Finishing application</figcaption></figure><h3>Getting Started</h3><h4>1. Project initialization</h4><p>Firstly, let&#39;s initialize the nodejs application with Typescript</p><ul><li>Create a new directory for your project and navigate to it in your terminal. Then running npm init -y (feel free to add additional information if you want to)</li><li>Install all the dependency packages that we need with the command</li></ul><pre>npm install aws-sdk dotenv inquirer chalk commander<br><br>npm install --save-dev typescript ts-node @types/inquirer @types/node</pre><ul><li>Add a start script to package.json as well as type to module as we will use the import statement inside nodejs</li></ul><pre>// package.json<br>{<br>  ...<br>  &quot;scripts&quot;: {<br>    &quot;start&quot;: &quot;ts-node --esm src/index.ts&quot;<br>  },<br>  &quot;type&quot;: &quot;module&quot;,<br>  ...<br>}</pre><ul><li>Create a tscofig.json file with basic settings.</li></ul><pre>{<br>  &quot;compilerOptions&quot;: {<br>    &quot;target&quot;: &quot;es6&quot;,<br>    &quot;module&quot;: &quot;NodeNext&quot;,<br>    &quot;outDir&quot;: &quot;dist&quot;,<br>    &quot;strict&quot;: true,<br>    &quot;esModuleInterop&quot;: true<br>  },<br>  &quot;include&quot;: [&quot;src/**/*&quot;]<br>}</pre><ul><li>Create a src directory in the root of your project and add a TypeScript file src/index.ts to it.</li></ul><pre>// index.ts<br>const message: string = &quot;Hello world!&quot;;<br><br>console.log(message);</pre><ul><li>Try to run the project by running in the command line npm start and if the console output Hello world! without any errors then you are good to go to the next step.</li></ul><h4>2. Prepare AWS IAM</h4><p>It can be very lengthy to include how to create an AWS IAM user here, I will attach the link to AWS official document on how to create an IAM user. Make sure you grant enough permissions to list buckets, create bucket, list objects, and put objects.</p><p><a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html">Create an IAM user in your AWS account</a></p><p>After creating the IAM user, create a new env file in the code repository named .env. Please remember to add this file to .gitignore the file if you intend to upload this to any public source code management tool such as Github, Gitlab, etc…</p><pre>AWS_ACCESS_KEY=&lt;Your-Access-Key&gt;<br>AWS_SECRET_KEY=&lt;Your-Secret-Key&gt;<br>AWS_REGION=&lt;Your-Intended-Region&gt;</pre><h4>3. Setup AWS interface and helper functions</h4><p>Create a separate file named aws.ts inside src folder as below to config AWS with the credentials and export all the functions that we need.</p><pre>import fs from &quot;fs&quot;;<br>import dotenv from &quot;dotenv&quot;;<br>import AWS, { AWSError } from &quot;aws-sdk&quot;;<br>import chalk from &quot;chalk&quot;;<br><br>dotenv.config();<br><br>AWS.config.update({<br>  accessKeyId: process.env.AWS_ACCESS_KEY,<br>  secretAccessKey: process.env.AWS_SECRET_KEY,<br>  region: process.env.AWS_REGION,<br>});<br><br>const s3 = new AWS.S3();<br><br>// List all buckets in S3<br>export const listBucket = () =&gt; {<br>  return new Promise&lt;string[]&gt;((resolve, reject) =&gt; {<br>    s3.listBuckets((err: AWSError, data: AWS.S3.ListBucketsOutput) =&gt; {<br>      if (err || !data.Buckets) {<br>        return resolve([]);<br>      }<br>      const buckets: string[] = [];<br>      data.Buckets.forEach((bucket) =&gt; {<br>        if (bucket.Name) {<br>          buckets.push(bucket.Name);<br>        }<br>      });<br>      resolve(buckets);<br>    });<br>  });<br>};<br><br>// List all folders of a bucket in S3<br>export const listFoldersOfBucket = (<br>  bucketName: string,<br>  prefix: string<br>): Promise&lt;string[]&gt; =&gt; {<br>  return new Promise&lt;string[]&gt;((resolve, reject) =&gt; {<br>    s3.listObjectsV2(<br>      { Bucket: bucketName, Prefix: prefix, Delimiter: &quot;/&quot; },<br>      (err: AWSError, data: AWS.S3.ListObjectsV2Output) =&gt; {<br>        if (err || !data.CommonPrefixes) {<br>          return resolve([]);<br>        }<br>        const folders: string[] = [];<br>        data.CommonPrefixes.forEach((folder) =&gt; {<br>          if (folder.Prefix) {<br>            folders.push(folder.Prefix);<br>          }<br>        });<br>        resolve(folders);<br>      }<br>    );<br>  });<br>};<br><br>// Upload a file to S3<br>export const putObject = (<br>  bucketName: string,<br>  fileName: string,<br>  s3UploadingPath: string<br>): Promise&lt;{ error: number | string }&gt; =&gt; {<br>  let awsFilePath = s3UploadingPath + fileName;<br>  if (fileName.includes(&quot;/&quot;)) {<br>    awsFilePath =<br>      s3UploadingPath + fileName.slice(fileName.lastIndexOf(&quot;/&quot;) + 1);<br>  }<br><br>  return new Promise&lt;{ error: number | string }&gt;((resolve) =&gt; {<br>    // Prepare the s3 uploading path<br>    let fileContent;<br>    try {<br>      fileContent = fs.readFileSync(fileName);<br>    } catch (error) {<br>      console.log(chalk.red(&quot;Error reading file: &quot; + fileName));<br>      return;<br>    }<br><br>    // Uploading logic<br>    s3.putObject(<br>      { Bucket: bucketName, Key: awsFilePath, Body: fileContent },<br>      (err: AWSError) =&gt; {<br>        if (err) {<br>          console.log(chalk.red(`Unsuccessfully uploaded ${fileName}`));<br>          return resolve({ error: err.message });<br>        } else {<br>          console.log(<br>            chalk.green(`Successfully uploaded ${fileName} to ${awsFilePath}`)<br>          );<br>          return resolve({ error: 0 });<br>        }<br>      }<br>    );<br>  });<br>};</pre><p>The exported functions:</p><ul><li>listBucket: list all the buckets on S3</li><li>listFoldersOfBucket: List all folders of a bucket in S3</li><li>putObject: Upload a file to S3</li></ul><p>Replace all content in the file index.ts with this block code below and run npm start to check if the program works up till now.</p><pre>import { listBucket } from &quot;./aws.js&quot;;<br><br>listBucket().then(console.log);</pre><p>Troubleshoot:</p><ul><li>The type error form Typescript may prevent you from using import statement, make sure you follow section 1 where we setup package.json and tsconfig.json to use module import.</li><li>You may face the error of insufficient permission with the IAM user that you just created, make sure you give enough permissions to that user.</li></ul><h4>4. Prepare the prompt utility with inquirer and chalk to make the CLI more interactive</h4><p>In the src directory, create another file named inquirer.ts that contains all the logic to prompt asking about buckets and folders to select.</p><pre>import inquirer from &quot;inquirer&quot;;<br>import chalk from &quot;chalk&quot;;<br><br>export const NEW_BUCKET_CHOICE = &quot;New bucket&quot;;<br>export const generateBucketQuestion = (choices: string[]) =&gt; [<br>  {<br>    type: &quot;list&quot;,<br>    name: &quot;bucket&quot;,<br>    message: chalk.green(&quot;Please select which bucket?&quot;),<br>    choices: [...choices, NEW_BUCKET_CHOICE],<br>  },<br>];<br><br>export const CURRENT_FOLDER_CHOICE = &quot;Current folder&quot;;<br>export const NEW_FOLDER_CHOICE = &quot;New folder&quot;;<br>export const generateFolderQuestion = (choices: string[]) =&gt; [<br>  {<br>    type: &quot;list&quot;,<br>    name: &quot;folder&quot;,<br>    message: chalk.green(&quot;Please select which folder?&quot;),<br>    choices: [CURRENT_FOLDER_CHOICE, ...choices, NEW_FOLDER_CHOICE],<br>  },<br>];<br><br>// Prompt asking for bucket choice<br>export const promptBucketChoiceQuestion = &lt;T&gt;(<br>  choices: string[]<br>): Promise&lt;T&gt; =&gt; {<br>  return new Promise&lt;T&gt;((resolve) =&gt; {<br>    inquirer.prompt(generateBucketQuestion(choices)).then(resolve);<br>  });<br>};<br><br>// Prompt asking for folder choice<br>export const promptFolderChoiceQuestion = (<br>  choices: string[]<br>): Promise&lt;{ folder: string }&gt; =&gt; {<br>  return new Promise&lt;{ folder: string }&gt;((resolve) =&gt; {<br>    inquirer.prompt(generateFolderQuestion(choices)).then(resolve);<br>  });<br>};</pre><ul><li>promptBucketChoiceQuestion: function to promptly ask for the selection of which <strong>bucket</strong> will be uploaded to.</li><li>promptFolderChoiceQuestion: function to promptly ask for the selection of which <strong>folder</strong> will be uploaded to.</li></ul><h4>5. Prepare the logic for command line argument parsing</h4><p>This part will help us to pass the arguments from the command line to indicate which file we want to upload to S3 with the help of the npm package commander (<a href="https://www.npmjs.com/package/commander">https://www.npmjs.com/package/commander</a>)</p><p>Replace all content in index.ts file with the below content</p><pre>// index.ts<br>import { program } from &quot;commander&quot;;<br><br>function parseArgumentsCLI() {<br>  program<br>    .argument(&quot;&lt;filepath&gt;&quot;, &quot;Path to the file to process&quot;)<br>    .parse(process.argv);<br><br>  const filepath = program.args[0];<br>  return { filepath };<br>}<br><br>async function main() {<br>  const { filepath } = parseArgumentsCLI();<br><br>  console.log(&quot;File path &quot; + filepath);<br>}<br><br>main();</pre><p>Try to run the CLI application with npm start ./hello-word.txt .The console will output File path ./example-files/helloword.txt</p><h4>6. Integrate the core logic to upload a file to AWS S3</h4><p>Modify the content of index.ts as below.</p><pre>import chalk from &quot;chalk&quot;;<br>import { listBucket, listFoldersOfBucket, putObject } from &quot;./aws.js&quot;;<br>import {<br>  promptBucketChoiceQuestion,<br>  promptFolderChoiceQuestion,<br>  NEW_BUCKET_CHOICE,<br>  CURRENT_FOLDER_CHOICE,<br>  NEW_FOLDER_CHOICE,<br>} from &quot;./inquirer.js&quot;;<br>import { program } from &quot;commander&quot;;<br><br>function parseArgumentsCLI() {<br>  program<br>    .argument(&quot;&lt;filepath&gt;&quot;, &quot;Path to the file to process&quot;)<br>    .parse(process.argv);<br><br>  const filepath = program.args[0];<br>  return { filepath };<br>}<br><br>async function main() {<br>  let selectedBucket: string = &quot;&quot;;<br>  let selectedFolder: string = &quot;&quot;;<br>  let prefix = &quot;&quot;;<br><br>  const { filepath } = parseArgumentsCLI();<br>  const allBuckets = await listBucket();<br>  const { bucket } = await promptBucketChoiceQuestion&lt;{ bucket: string }&gt;(<br>    allBuckets<br>  );<br><br>  if (bucket === NEW_BUCKET_CHOICE) {<br>    // TODO: Create new bucket<br>  } else {<br>    selectedBucket = bucket;<br>  }<br><br>  // Because user might choose sub-folder, so it should be in a while loop<br>  while (true) {<br>    const subFolderList: string[] = await listFoldersOfBucket(selectedBucket, prefix);<br>    const { folder } = await promptFolderChoiceQuestion(<br>      subFolderList<br>        .map((folder) =&gt; (!!prefix ? folder.replace(prefix, &quot;&quot;) : folder))<br>        .map((folder) =&gt; folder.replace(&quot;/&quot;, &quot;&quot;))<br>    );<br><br>    if (folder === CURRENT_FOLDER_CHOICE) {<br>      selectedFolder = prefix;<br>      break;<br>    } else if (folder === NEW_FOLDER_CHOICE) {<br>      // TODO: Prompt to ask new folder name<br>      break;<br>    } else {<br>      prefix += folder + &quot;/&quot;;<br>    } <br>  }<br><br>  let awsFilePath = selectedFolder + filepath;<br>  if (filepath.includes(&quot;/&quot;)) {<br>    awsFilePath =<br>      selectedFolder + filepath.slice(filepath.lastIndexOf(&quot;/&quot;) + 1);<br>  }<br><br>  const result = await putObject(selectedBucket, filepath, awsFilePath);<br>  if (!result.error) {<br>    console.log(<br>      chalk.green(`Successfully uploaded ${filepath} to ${awsFilePath}`)<br>    );<br>  }<br>}<br><br>main();</pre><p>Here is the walkthrough logic inside main function:</p><ul><li>First, it will get the filepath from command liner parser commander .</li><li>Then, it fetches all the buckets that existed in the AWS S3 and passes it as the choices of the bucket for the user to select.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/439/1*u3uTQ_JjLmmlRkSWbeAFEQ.png" /><figcaption><em>Example of how the bucket list are displayed to be selected</em></figcaption></figure><ul><li>After selecting the bucket (we will work on creating a new bucket in Part 2), it will fetch all the folders inside the bucket and display them to be selected</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/519/1*wo-OPLZ_YC3lEvuedzmUsg.png" /><figcaption>Example of how the folders are displayed to be selected</figcaption></figure><ul><li>We can either select this current folder to upload to, or we can also select the sub-folder as well, or we can even create a new folder on the fly while uploading (we will work on this in Part 2).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/519/1*Tm6iaBSI0V3gV9R-IXiQmw.png" /><figcaption>Example of if we keep selecting the subfolder</figcaption></figure><ul><li>After selecting Current folder to indicate that we finally choose this is the place we want to upload to, it will perform the uploading to logic to upload it to S3.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/703/1*dPuq87geQ6ZeayO3W49nFA.png" /><figcaption>Example of selecting `Current Folder` to upload to</figcaption></figure><p>Wow, if you have reached this part, congratulation you have done a great job today. It’s a long way until we can finally test a part of this application.</p><p>We still have a few things to work on such as uploading a folder instead of a file or creating a new folder while uploading, or even creating a new bucket and uploading files/folder to the newly created folder as well.</p><p><strong>PART 2: </strong><a href="https://medium.com/@techwithharry/files-uploader-cli-with-nodejs-typescript-and-aws-s3-part-2-2-bf0518c51888">https://medium.com/@techwithharry/files-uploader-cli-with-nodejs-typescript-and-aws-s3-part-2-2-bf0518c51888</a></p><p>If you find my blog interesting, please consider following it for more updates and insights at <a href="https://medium.com/u/2046c51afa0b">Tech with Harry</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=86fc29e919ad" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[IoT Data Streaming with Nodejs]]></title>
            <link>https://medium.com/@techwithharry/iot-data-streaming-with-nodejs-4ea188e077d8?source=rss-2046c51afa0b------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ea188e077d8</guid>
            <category><![CDATA[influxdb]]></category>
            <category><![CDATA[iot]]></category>
            <category><![CDATA[streaming]]></category>
            <category><![CDATA[mqtt]]></category>
            <category><![CDATA[nodejs]]></category>
            <dc:creator><![CDATA[Tech with Harry]]></dc:creator>
            <pubDate>Tue, 28 Mar 2023 05:19:52 GMT</pubDate>
            <atom:updated>2023-03-28T05:19:52.162Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qo87Kj0Zhh_LfS8-niEhyA.jpeg" /></figure><p>Data streaming is a technique used to process and transfer large amounts of data in a continuous and efficient manner. In Node.js, data streaming can be achieved using the built-in stream module. In this blog, we will explore the basics of data streaming with Node.js and provide some sample codes.</p><h3>What is Node.js Data Streaming?</h3><p>In Node.js, data streaming is a way of handling large amounts of data by breaking it into smaller, more manageable chunks called streams. Streams are a sequence of data that can be read or written in a continuous and efficient manner. This makes it possible to process data in real time without the need for loading it all into memory at once.</p><h3>Types of Streams</h3><p>Node.js provides four types of streams: Readable, Writable, Duplex, and Transform. Each type of stream is used for a different purpose.</p><ul><li><strong>Readable stream</strong>: Used for reading data from a source.</li><li><strong>Writable stream</strong>: Used for writing data to a destination.</li><li><strong>Duplex stream</strong>: Used for both reading and writing data.</li><li><strong>Transform stream</strong>: Used for modifying data as it passes through the stream.</li></ul><h3>Simple Example with <strong>Readable stream and Writable stream</strong></h3><p>Let&#39;s look at some sample code for creating a simple data stream in Node.js.</p><pre>const fs = require(&#39;fs&#39;);<br><br>// Create a readable stream to read data from a file<br>const readableStream = fs.createReadStream(&#39;input.txt&#39;);<br><br>// Create a writable stream to write data to a file<br>const writableStream = fs.createWriteStream(&#39;output.txt&#39;);<br><br>// Pipe the readable stream to the writable stream to read and write data simultaneously<br>readableStream.pipe(writableStream);<br><br>// Listen for the &#39;data&#39; event to read data from the readable stream<br>readableStream.on(&#39;data&#39;, (chunk) =&gt; {<br>  console.log(`Received ${chunk.length} bytes of data.`);<br>});<br><br>// Listen for the &#39;end&#39; event to indicate the end of the readable stream<br>readableStream.on(&#39;end&#39;, () =&gt; {<br>  console.log(&#39;Finished reading data.&#39;);<br>});<br><br>// Listen for the &#39;finish&#39; event to indicate the end of the writable stream<br>writableStream.on(&#39;finish&#39;, () =&gt; {<br>  console.log(&#39;Finished writing data.&#39;);<br>});</pre><p>In this example, we use the built-in fs module to create a readable stream from a file called input.txt and a writable stream to a file called output.txt. We then pipe the readable stream to the writable stream to read and write data simultaneously.</p><p>We also listen for the &#39;data&#39; event to read data from the readable stream and print a message to the console indicating the number of bytes received. We listen for the &#39;end&#39; event to indicate the end of the readable stream, and the &#39;finish&#39; event to indicate the end of the writable stream. When the streams have finished reading and writing data, we print messages to the console to indicate that the process is complete.</p><p>This is just a simple example, but data streaming with Node.js can be used for much more complex applications such as real-time data processing, data analysis, and more.</p><h3>Example processing real-time IoT with MQTT and InfluxDB</h3><pre>const mqtt = require(&quot;mqtt&quot;);<br>const Influx = require(&quot;influx&quot;);<br><br>// Connect to MQTT broker<br>const client = mqtt.connect(&quot;mqtt://localhost&quot;);<br><br>// Connect to InfluxDB<br>const influx = new Influx.InfluxDB({<br>  host: &quot;localhost&quot;,<br>  database: &quot;iot_data&quot;,<br>});<br><br>// Listen for incoming MQTT messages<br>client.on(&quot;connect&quot;, () =&gt; {<br>  console.log(&quot;Connected to MQTT broker.&quot;);<br><br>  client.subscribe(&quot;iot/sensor1&quot;);<br>});<br><br>client.on(&quot;message&quot;, (topic, message) =&gt; {<br>  console.log(`Received message on topic &#39;${topic}&#39;: ${message.toString()}`);<br><br>  // Parse the message payload as JSON<br>  const data = JSON.parse(message.toString());<br><br>  // Write the data to InfluxDB<br>  influx<br>    .writePoints([<br>      {<br>        measurement: &quot;sensor_data&quot;,<br>        tags: {<br>          sensor_id: data.sensor_id,<br>        },<br>        fields: {<br>          temperature: data.temperature,<br>          humidity: data.humidity,<br>        },<br>        timestamp: data.timestamp,<br>      },<br>    ])<br>    .then(() =&gt; {<br>      console.log(&quot;Data written to InfluxDB.&quot;);<br>    })<br>    .catch((err) =&gt; {<br>      console.error(`Error writing data to InfluxDB: ${err}`);<br>    });<br>});</pre><p>In this example, we use the mqtt library to connect to an MQTT broker and listen for incoming messages on the iot/sensor1 topic. MQTT broker is a messaging middleware that acts as an intermediary between devices (publishers) and applications (subscribers) that communicate using the MQTT protocol.</p><p>When a message is received, we parse the JSON payload and write the data to an InfluxDB database using the influx library. We use the measurement option to specify the name of the InfluxDB measurement, the tags option to specify any metadata associated with the data (such as the sensor ID), and the fields option to specify the actual data values (such as temperature and humidity). We also include a timestamp option to specify the timestamp associated with the data.</p><p>This code can be used to process real-time IoT data streams and store the data in a time-series database for later analysis and visualization. By using Node.js and the mqtt and influx libraries, we can easily connect to MQTT brokers and InfluxDB databases, and process large amounts of real-time data in a scalable and efficient way.</p><h3>Conclusion</h3><p>Data streaming is a powerful technique for handling large amounts of data in Node.js. With the built-in stream module, you can easily create data streams for reading, writing, and modifying data. In this blog, we have covered the basics of the data streaming and provided some sample codes for creating a simple data stream. Try implementing data streaming in your Node.js projects to improve performance and scalability.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ea188e077d8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Secure database connection in Nodejs application]]></title>
            <link>https://systemweakness.com/secure-database-connection-in-nodejs-application-3c3979566741?source=rss-2046c51afa0b------2</link>
            <guid isPermaLink="false">https://medium.com/p/3c3979566741</guid>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[expressjs]]></category>
            <dc:creator><![CDATA[Tech with Harry]]></dc:creator>
            <pubDate>Sun, 26 Mar 2023 22:06:09 GMT</pubDate>
            <atom:updated>2023-03-29T11:18:58.618Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rdpA-D-4vtUFcXZS6bpFcA.png" /><figcaption>Secure database connection in Nodejs application</figcaption></figure><p>Securing database connections is an essential part of building secure Node.js applications. Here are some best practices for handling secure database connections in Node.js</p><h3>1. Use environment variables</h3><p>Store sensitive database connection information (such as usernames, passwords, and connection strings) in environment variables rather than hard-coding them in your application code. This helps protect your application from accidental exposure of sensitive information as well as giving flexibility in changing the credentials depending on the environment that we are using (dev, staging, or production).</p><pre>require(&quot;dotenv&quot;).config();<br><br>const dbHost = process.env.DB_HOST;<br>const dbPort = process.env.DB_PORT;<br>const dbUser = process.env.DB_USER;<br>const dbPassword = process.env.DB_PASSWORD;<br>const dbName = process.env.DB_NAME;</pre><h3><strong><em>2. Use encrypted connections</em></strong></h3><p>When you transmit data over an unencrypted connection, the data can be <strong><em>intercepted</em></strong> by attackers who can then read, modify, or steal it. This is especially important when sensitive data, such as usernames and passwords, financial data, or personal information, is being transmitted.</p><p>Using encrypted connections can help protect your data from these threats. Encrypted connections use <strong>Secure Sockets Layer (SSL)</strong> or <strong>Transport Layer Security (TLS)</strong> protocols to encrypt the data in transit, making it more difficult for attackers to intercept and access the data.</p><p>Encrypting your database connection can also help you comply with various data protection regulations, such as the <em>General Data Protection Regulation (GDPR)</em> or the <em>Health Insurance Portability and Accountability Act (HIPAA)</em>.</p><p>For example, when using MongoDB, use the mongodb+srv connection string to connect over SSL.</p><pre>const { MongoClient } = require(&#39;mongodb&#39;);<br>require(&#39;dotenv&#39;).config();<br><br>const dbUser = process.env.DB_USER;<br>const dbPassword = process.env.DB_PASSWORD;<br>const uri = `mongodb+srv://${dbUser}:${dbPassword}@&lt;cluster&gt;.mongodb.net/test?retryWrites=true&amp;w=majority`;<br><br>const client = new MongoClient(uri, { useNewUrlParser: true, useUnifiedTopology: true });<br>async function connect() {<br>  try {<br>    // Connect to the MongoDB server<br>    await client.connect();<br><br>    console.log(&#39;Connected successfully to server&#39;);<br>  } catch (err) {<br>    console.error(err);<br>  }<br>}<br><br>connect();</pre><p>With PostgreSQL, here is the example with the help of tls module. The rejectUnauthorized option is checked to verify the server&#39;s SSL certificate.</p><pre>const { Client } = require(&quot;pg&quot;);<br>const tls = require(&quot;tls&quot;);<br>const dotenv = require(&quot;dotenv&quot;);<br><br>dotenv.config();<br><br>const sslOptions = {<br>  rejectUnauthorized: true,<br>};<br>const tlsClient = tls.connect(5432, process.env.DB_HOST, sslOptions);<br><br>const client = new Client({<br>  host: process.env.DB_HOST,<br>  port: process.env.DB_PORT,<br>  user: process.env.DB_USER,<br>  password: process.env.DB_PASSWORD,<br>  database: process.env.DB_NAME,<br>  ssl: {<br>    sslmode: &quot;require&quot;,<br>    sslfactory: tlsClient,<br>  },<br>});<br><br>client.connect()<br>  .then(() =&gt; console.log(&quot;Connected to PostgreSQL server&quot;))<br>  .catch((error) =&gt; console.error(&quot;Error connecting to PostgreSQL server&quot;, error));</pre><h3>3. Use connection pooling</h3><p>Connection pooling is a technique used to optimize the performance of applications that use a database. When an application needs to connect to a database, it creates a new connection to the database server.</p><p>Creating a new connection is a relatively expensive operation in terms of time and resources, so connection pooling aims to minimize the number of connections that are created by reusing existing connections whenever possible. Libraries like pg and mysql2 for PostgreSQL and MySQL, respectively, provide built-in connection pooling.</p><p>In Express.js application, we can split the database pool establishment into a separate file</p><pre>// database.js<br>const { Pool } = require(&#39;pg&#39;);<br><br>// Create a new connection pool with a maximum of 20 connections<br>const pool = new Pool({<br>  host: process.env.DB_HOST,<br>  port: process.env.DB_PORT,<br>  user: process.env.DB_USER,<br>  password: process.env.DB_PASSWORD,<br>  database: process.env.DB_NAME,<br>  max: 20,<br>});<br><br>module.exports = pool;</pre><p>And import into our main app.js like this. Passing the pool connection through the help of middleware so that every route handler can access to the pool connection to query the database without the overhead of open and close database connection.</p><pre>const express = require(&quot;express&quot;);<br>const pool = require(&quot;./database&quot;);<br><br>const app = express();<br><br>// Middleware function that helps to carry over the pool<br>app.use((req, res, next) =&gt; {<br>  req.db = { pool };<br>  next();<br>});<br><br>// Route that uses the connection pool<br>app.get(&quot;/users/:id&quot;, () =&gt; {<br>  async function getUser(req, res, next) {<br>    const { pool } = req.db;<br>    try {<br>      const result = await pool.query(&quot;SELECT * FROM users WHERE id = $1&quot;, [<br>        req.params.id,<br>      ]);<br>      res.json(result.rows[0]);<br>    } catch (err) {<br>      console.error(err);<br>      res.status(500).send(&quot;Error querying database&quot;);<br>    }<br>  }<br>});<br><br>app.listen(3000, () =&gt; {<br>  console.log(&quot;Server running on port 3000&quot;);<br>});</pre><h3>4. Limit access to your database server</h3><p>Configure your database server to only accept connections from authorized IP addresses, and use firewalls and other security measures to restrict access to your database server</p><ul><li><strong>Use a firewall</strong>: Configure your database server to only allow connections from trusted IP addresses or ranges. This situation can be applied to the system where our Nodejs application sits behind an API Gateway or a Reverse Proxy server like Nginx.<br>This can be done by setting up firewall rules on your server. By limiting the IP addresses that are allowed to connect to your database server, you can reduce the risk of unauthorized access.</li><li>An example shown as below. Note that the req.ip property may not always contain the correct IP address if the request has passed through one or more proxies. If you need to get the original IP address of the client, you can use a library like request-ip to retrieve it from the X-Forwarded-For header, which contains a list of IP addresses in reverse proxy chains.</li></ul><pre>const ipCheck = require(&#39;ip-range-check&#39;);<br>const allowedRange = &#39;192.168.1.0/24&#39;;<br><br>// Use firewall to limit access<br>app.use((req, res, next) =&gt; {<br>  const ip = req.ip;<br>  const ipRange = ipCheck(ip, allowedRange);<br>  if (ipRange) {<br>    // If the IP address is within the allowed range, proceed with the request<br>    next();<br>  } else {<br>    // If the IP address is not within the allowed range, send a 403 Forbidden response<br>    res.sendStatus(403);<br>  }<br>});</pre><ul><li><strong>Use authentication</strong>: Ensure that your database server requires strong passwords and uses secure authentication protocols, such as SSL/TLS. Use different usernames and passwords for different users, and limit the permissions of each user to only what they need to do their job. In Node.js, you can use various libraries to connect to your database server and pass in the necessary authentication details.</li><li>Use <strong>connection pooling</strong> and <strong>SSL</strong> are already discussed above.</li></ul><p>By implementing these best practices, you can help ensure that your Node.js application is secure and that your database server is protected from unauthorized access.</p><p>If you find my blog interesting, please consider following it for more updates and insights at <a href="https://medium.com/u/2046c51afa0b">Tech with Harry</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3c3979566741" width="1" height="1" alt=""><hr><p><a href="https://systemweakness.com/secure-database-connection-in-nodejs-application-3c3979566741">Secure database connection in Nodejs application</a> was originally published in <a href="https://systemweakness.com/">System Weakness</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Node.js good practices for security]]></title>
            <link>https://medium.com/@techwithharry/node-js-security-good-practices-for-security-56a5f56d4f6d?source=rss-2046c51afa0b------2</link>
            <guid isPermaLink="false">https://medium.com/p/56a5f56d4f6d</guid>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Tech with Harry]]></dc:creator>
            <pubDate>Sun, 26 Mar 2023 07:33:50 GMT</pubDate>
            <atom:updated>2023-03-26T18:25:17.957Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8KnGUEx-K60yXua6UjWczA.jpeg" /></figure><p>Security is an essential aspect of any application, and Node.js applications are no exception. Node.js is a powerful tool that can be used to build secure and robust web applications, but it’s important to understand the security risks that come with building applications using Node.js. Here are some potential topics you could cover in an article on Node.js security best practices:</p><p><strong><em>1. SQL Injection</em></strong></p><p>SQL injection is a type of attack where an attacker injects malicious SQL code into an application to gain access to sensitive data or execute unauthorized actions. To prevent SQL injection in a Node.js application, you can use parameterized queries or prepared statements, which separate the query logic from the user input. Parameterized queries and prepared statements ensure that the user input is treated as a value rather than as part of the query syntax, which prevents attackers from injecting malicious code into the query. A simple example is shown below.</p><pre>const mysql = require(&#39;mysql&#39;);<br><br>// Set up a connection to the database<br>const connection = mysql.createConnection({<br>  host: &#39;localhost&#39;,<br>  user: &#39;root&#39;,<br>  password: &#39;password&#39;,<br>  database: &#39;mydatabase&#39;<br>});<br><br>// Define a function to get user data from the database<br>function getUser(username, password, callback) {<br>  const query = &#39;SELECT * FROM users WHERE username = ? AND password = ?&#39;;<br>  const values = [username, password];<br><br>  // Use a parameterized query to prevent SQL injection<br>  connection.query(query, values, (error, results, fields) =&gt; {<br>    if (error) {<br>      callback(error, null);<br>    } else {<br>      callback(null, results);<br>    }<br>  });<br>}</pre><p>In this example, the getUser() function takes a username and password as arguments and returns user data from the database. To prevent SQL injection, the function uses a parameterized query by replacing the values in the SQL statement with placeholders (?) and passing the actual values as an array (values) to the connection.query() method. This ensures that the user input is treated as a value rather than as part of the SQL syntax, which prevents SQL injection attacks.</p><p>Additionally, using an ORM (object-relational mapping) library like Sequelize or TypeORM can provide an additional layer of protection against SQL injection by automatically escaping user input.</p><p><strong><em>2. Cross-site scripting (XSS)</em></strong></p><p>Cross-site scripting is a type of attack where an attacker injects malicious scripts into a web page viewed by other users. This can be used to steal sensitive data or execute unauthorized actions. To prevent XSS in a Node.js application, you can use input validation and sanitization to ensure that user input is free of malicious scripts. You can also use a content security policy (CSP) to control which external resources (such as scripts and stylesheets) are loaded by your application. A simple example is shown below.</p><pre>const express = require(&#39;express&#39;);<br>const bodyParser = require(&#39;body-parser&#39;);<br>const xss = require(&#39;xss&#39;);<br><br>const app = express();<br><br>// Use body-parser middleware to parse incoming request bodies<br>app.use(bodyParser.json());<br><br>// Define a route to handle a POST request<br>app.post(&#39;/submit&#39;, (req, res) =&gt; {<br>  // Get the user input from the request body<br>  const userInput = req.body.input;<br><br>  // Validate and sanitize the user input using the xss library<br>  const sanitizedInput = xss(userInput);<br><br>  // Save the sanitized input to the database or perform other actions<br>  // ...<br><br>  res.send(&#39;Input saved successfully!&#39;);<br>});<br><br>// Start the server<br>app.listen(3000, () =&gt; {<br>  console.log(&#39;Server started on port 3000&#39;);<br>});</pre><p>In this example, the server uses the xss library to validate and sanitize user input received in a POST request. The xss library is specifically designed to prevent Cross-site scripting (XSS) attacks by escaping any HTML, JavaScript, or other potentially malicious code in the user input.</p><p><strong><em>3. Cross-site request forgery (CSRF)</em></strong></p><p>Cross-site request forgery is a type of attack where an attacker tricks a user into performing an action on a website without their knowledge or consent. To prevent CSRF in a Node.js application, you can use tokens to ensure that requests are coming from a trusted source. Tokens are typically generated on the server and included in each request. The server then verifies the token before executing the requested action. A simple example is shown below.</p><pre>const express = require(&#39;express&#39;);<br>const bodyParser = require(&#39;body-parser&#39;);<br>const csrf = require(&#39;csurf&#39;);<br><br>const app = express();<br><br>// Use body-parser middleware to parse incoming request bodies<br>app.use(bodyParser.json());<br><br>// Set up CSRF protection using the csurf middleware<br>const csrfProtection = csrf({ cookie: true });<br><br>// Define a route to render a form with a CSRF token<br>app.get(&#39;/form&#39;, csrfProtection, (req, res) =&gt; {<br>  res.render(&#39;form&#39;, { csrfToken: req.csrfToken() });<br>});<br><br>// Define a route to handle a POST request<br>app.post(&#39;/submit&#39;, csrfProtection, (req, res) =&gt; {<br>  // Verify the CSRF token<br>  if (req.csrfToken() !== req.body._csrf) {<br>    res.status(403).send(&#39;Invalid CSRF token!&#39;);<br>    return;<br>  }<br><br>  // Save the form data to the database or perform other actions<br>  // ...<br><br>  res.send(&#39;Form data saved successfully!&#39;);<br>});<br><br>// Start the server<br>app.listen(3000, () =&gt; {<br>  console.log(&#39;Server started on port 3000&#39;);<br>});</pre><p>In this example, the server uses the csurf middleware to generate and validate CSRF tokens. When the user visits the /form route, the server generates a CSRF token using the req.csrfToken() method and renders a form that includes the token as a hidden input field. When the user submits the form, the server verifies that the CSRF token in the request body matches the token generated earlier using the req.csrfToken() method. If the tokens don&#39;t match, the server returns a 403 Forbidden response.</p><p>By using CSRF tokens like this, you can help prevent Cross-site request forgery (CSRF) attacks in your Node.js application. It’s also important to set the SameSite attribute on any cookies used by your application to ensure that they&#39;re not sent with cross-site requests. Additionally, you should avoid using HTTP methods with side effects (such as POST and DELETE) for GET requests, and avoid relying solely on cookies for authentication and authorization.</p><p><strong><em>4. Denial-of-service (DoS) attacks</em></strong></p><p>DoS attacks are a type of attack where an attacker floods a server with requests in an attempt to overwhelm it and make it unavailable to users. To prevent DoS attacks in a Node.js application, you can use <strong><em>rate limiting</em></strong> to limit the number of requests that can be made in a certain period of time. You can also use load balancers and server clusters to distribute the workload across multiple servers. A simple example is shown below.</p><pre>const express = require(&#39;express&#39;);<br>const rateLimit = require(&#39;express-rate-limit&#39;);<br><br>const app = express();<br><br>// Set up rate limiting using the express-rate-limit middleware<br>const limiter = rateLimit({<br>  windowMs: 15 * 60 * 1000, // 15 minutes<br>  max: 100, // limit each IP to 100 requests per windowMs<br>  message: &#39;Too many requests from this IP, please try again later&#39;<br>});<br><br>// Apply rate limiting to all routes<br>app.use(limiter);<br><br>// Define a route to handle a GET request<br>app.get(&#39;/&#39;, (req, res) =&gt; {<br>  res.send(&#39;Hello, world!&#39;);<br>});<br><br>// Start the server<br>app.listen(3000, () =&gt; {<br>  console.log(&#39;Server started on port 3000&#39;);<br>});</pre><p>In this example, the server uses the express-rate-limit middleware to set limits on incoming requests from each IP address. The limiter object specifies that each IP address can make a maximum of 100 requests within a 15-minute window. If a client exceeds this limit, the server returns a 429 Too Many Requests response with the message &quot;Too many requests from this IP, please try again later&quot;.</p><p>By setting limits on incoming requests like this, you can help prevent Denial-of-service (DoS) attacks in your Node.js application. It’s also a good practice to use a load balancer or proxy server to distribute incoming requests across multiple servers and to implement connection timeouts and request timeouts to prevent long-running requests from tying up server resources. Additionally, you should monitor your server logs for unusual or suspicious activity and implement DDoS protection solutions if your application is particularly vulnerable to DoS attacks.</p><p>In conclusion, the above sections are the simple but efficient ways to secure your nodejs application. There are more things to consider while developing the application such as using secure coding practices like avoiding eval or implementing secure data storage with proper user authentication and authorization process.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=56a5f56d4f6d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>