<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Syed Muhammed Hassan Ali on Medium]]></title>
        <description><![CDATA[Stories by Syed Muhammed Hassan Ali on Medium]]></description>
        <link>https://medium.com/@syed007hassan?source=rss-72ea816b21a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*eSnPwRip9gsuyN1nFCGJqQ.jpeg</url>
            <title>Stories by Syed Muhammed Hassan Ali on Medium</title>
            <link>https://medium.com/@syed007hassan?source=rss-72ea816b21a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 21:57:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@syed007hassan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Use Google OAuth In NextJS And NestJS]]></title>
            <link>https://blog.stackademic.com/use-google-oauth-in-nextjs-and-nestjs-e70d98b0f898?source=rss-72ea816b21a------2</link>
            <guid isPermaLink="false">https://medium.com/p/e70d98b0f898</guid>
            <category><![CDATA[nextjs]]></category>
            <category><![CDATA[google-oauth]]></category>
            <category><![CDATA[nestjs]]></category>
            <category><![CDATA[jwt]]></category>
            <category><![CDATA[middleware]]></category>
            <dc:creator><![CDATA[Syed Muhammed Hassan Ali]]></dc:creator>
            <pubDate>Mon, 01 Jan 2024 13:44:30 GMT</pubDate>
            <atom:updated>2024-01-01T16:28:12.618Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A Step-by-Step Guide to Integrate OAuth2.0 Authentication</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_vhpPGZtSa8TpAPHKJVy_Q.png" /></figure><h3>Introduction</h3><p>Welcome, developers! User authentication is a key aspect of your web application, and Google Login can make it both user-friendly and secure. In this guide, I’ll walk you through integrating Google Login into your NestJS application. Instead of NextAuth, I will be implementing the OAuth in the backend (Nest) and the frontend will consume this service.</p><p><strong>The flow is as follows:</strong></p><ol><li>The user is directed to an endpoint on the Backend API and is automatically redirected to the Google login screen.</li><li>After placing your credentials, you will be redirected to another endpoint of our server where we will receive all the user information.</li><li>We will look for the user in our database<br><strong>If the user exists:</strong> We return a JWT that contains the user’s ID and email as payload.<br><strong>If the user does not exist:</strong> We register the user in our database and then return a JWT.</li><li>We use middleware on NextJs to get the token from params of the redirected URL and save it to a cookie.</li></ol><p>Before diving into the code, let’s quickly set up your Google Developer Console to obtain the required credentials.</p><h3>Google Developer Console Configuration</h3><ul><li><strong>Create a Project:</strong> Go to the <a href="https://console.cloud.google.com/apis/dashboard">Google Developer Console</a> and create a new project.</li><li><strong>API &amp; Services:</strong> In your project, navigate to “API &amp; Services.”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*65EP6DaeKlkUrQjdubjo9w.png" /></figure><ul><li><strong>Credentials:</strong> Click on “Create credentials” and select “OAuth client ID.”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/711/1*_XXsfFzy43ETQnmgGqpShw.png" /></figure><ul><li><strong>Configure Consent Screen:</strong> Configure the consent screen by specifying user types, app names, support email, and developer contact info. You can skip the test user section or provide an email for testing.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*Ngxv3dXZvzMBugWB.jpeg" /></figure><ul><li><strong>Authorized Redirect URL:</strong> In the authorized redirect URL, enter <a href="http://localhost:5000/auth/google/callback"><strong>http://localhost:5000/auth/google/callback</strong></a><a href="http://localhost:3000/auth/google/callback."><strong>.</strong></a> On localhost:5000, I have my backend running therefore the authorized origin is also set the same. On production, you can change both of the URLs.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*CHXCWvZVKuQzdRwHXyNPcA.png" /></figure><ul><li><strong>Copy Credentials:</strong> After configuration, you’ll receive the <strong>Client ID</strong> and <strong>Client Secret</strong>. Make sure to keep these credentials safe in the <strong>.env</strong> file.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/270/1*9ErJIEqEDAu0lVvQ1-bkpA.png" /></figure><h3>NestJs Code</h3><p>I created a folder called auth, where I have stored all the authentication logic of my app, inside, we will have several things, 2 Authentication Guardians, one for authentication with Google, and the other for requests that require a JWT.</p><p>We will have 2 Passport strategies, a strategy for each authentication provider, in this case, we only have Google, and another strategy to extract the JWT from the cookies, in this way we will verify the identity of the user in each request, to give or not access to the resources, finally a basic controller and service of any Nest module.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/281/1*5f4s1zuaRjHNeU0ZlV94_Q.png" /></figure><p>Before starting, we must install the following packages, with NPM/Yarn/PNPM depending on your case</p><blockquote>npm install @nestjs/jwt @nestjs/passport passport passport-google-oauth2 passport-jwt passport-local</blockquote><blockquote>npm install -D @types/passport-google-oauth2 @types/passport-jwt @types/passport-jwt @types/passport-local</blockquote><p><strong>auth.module.ts</strong></p><pre>import { Module } from &#39;@nestjs/common&#39;;<br>import { AuthService } from &#39;./auth.service&#39;;<br>import { AuthController } from &#39;./auth.controller&#39;;<br>import { JwtModule } from &#39;@nestjs/jwt&#39;;<br>import { JwtGuardStrategy } from &#39;./guards/jwt-auth.strategy&#39;;<br>import { JwtGuard } from &#39;./guards/jwt-auth.guard&#39;;<br>import { GoogleOauthGuard } from &#39;./guards/google-oauth.guard&#39;;<br>import { GoogleStrategy } from &#39;./guards/google-oauth.strategy&#39;;<br>@Module({<br>  imports: [<br>    JwtModule.registerAsync({<br>      useFactory: () =&gt; ({<br>        secret: process.env.JWT_SECRET,<br>        signOptions: {<br>          expiresIn: &#39;3d&#39;,<br>        },<br>        global: true,<br>      }),<br>    }),<br>  ],<br>  controllers: [AuthController],<br>  providers: [AuthService, JwtGuardStrategy, JwtGuard, GoogleStrategy],<br>})<br>export class AuthModule {}</pre><p>In the auth module, import the JwtModule and make it global. Also, add the guards and strategies in the provider.</p><p><strong>jwt-auth.guard.ts</strong></p><pre>import { Injectable } from &#39;@nestjs/common&#39;;<br><br>import { AuthGuard } from &#39;@nestjs/passport&#39;;<br><br>@Injectable()<br>export class JwtGuard extends AuthGuard(&#39;jwt&#39;) {}</pre><p><strong>jwt-auth.strategy.ts</strong></p><pre>import { ExecutionContext, Injectable } from &#39;@nestjs/common&#39;;<br>import { ExtractJwt, Strategy } from &#39;passport-jwt&#39;;<br>import { PassportStrategy } from &#39;@nestjs/passport&#39;;<br><br>@Injectable()<br>export class JwtGuardStrategy extends PassportStrategy(Strategy) {<br>  constructor() {<br>    super({<br>      jwtFromRequest: ExtractJwt.fromAuthHeaderAsBearerToken(),<br>      ignoreExpiration: false,<br>      secretOrKey: process.env.JWT_SECRET,<br>    });<br>  }<br><br>  async validate(payload: any) {<br>    console.log(&#39;payload&#39;, payload);<br>    return payload;<br>  }<br>}</pre><p><strong>google-oauth.guard.ts</strong></p><pre>import { Injectable } from &#39;@nestjs/common&#39;;<br>import { AuthGuard } from &#39;@nestjs/passport&#39;;<br><br>@Injectable()<br>export class GoogleOauthGuard extends AuthGuard(&#39;google&#39;) {}</pre><p>Google’s guardian is simple, we create a GoogleOauthGuard class that extends from <a href="http://twitter.com/nest/passport">@nest/passport</a>’s AuthGuard class with the identifier google.</p><p><strong>google-oauth.strategy.ts</strong></p><pre>import { Inject, Injectable } from &#39;@nestjs/common&#39;;<br>import { PassportStrategy } from &#39;@nestjs/passport&#39;;  <br>import { configDotenv } from &#39;dotenv&#39;;<br>import { Strategy, VerifyCallback } from &#39;passport-google-oauth2&#39;;<br><br>configDotenv();<br><br>@Injectable()<br>export class GoogleStrategy extends PassportStrategy(Strategy, &#39;google&#39;) {<br>  constructor() {<br>    super({<br>      clientID: process.env.GOOGLE_CLIENT_ID,<br>      clientSecret: process.env.GOOGLE_CLIENT_SECRET,<br>      callbackURL: process.env.CALLBACK_URL,<br>      scope: [&#39;profile&#39;, &#39;email&#39;],<br>    });<br>  }<br><br>  async validate(<br>    _accessToken: string,<br>    _refreshToken: string,<br>    profile: any,<br>    done: VerifyCallback,<br>  ): Promise&lt;any&gt; {<br>    const { id, name, emails, photos } = profile;<br><br>    const user = {<br>      provider: &#39;google&#39;,<br>      providerId: id,<br>      email: emails[0].value,<br>      name: `${name.givenName} ${name.familyName}`,<br>      picture: photos[0].value,<br>    };<br><br>    done(null, user);<br>  }<br>}</pre><p>In the constructor, we pass a super with the following parameters:<br><strong>clientID</strong>: Our Client ID provided by Google<br><strong>clientSecret</strong>: Our Client secret provided by Google<br><strong>callbackURL</strong>: Our redirect URL that we previously configured in the Google console<br><strong>scopes</strong>: an array with the information we want to obtain from the user, the most common is profile and email</p><p>You can log the <strong>profile object</strong> to see what it brings, each provider brings this object in different ways, then you create a user object with the information you need according to your business model and your user schema, and finally, you return the user with the done method.</p><p><strong>auth.controller.ts</strong></p><pre>  @Get(&#39;callback/google&#39;)<br>  @UseGuards(GoogleOauthGuard)<br>  async googleAuthCallback(@Req() req, @Res() res: Response) {<br>    try {<br>      const token = await this.authService.oAuthLogin(req.user);<br>      res.redirect(`${FRONTEND_URL}/oauth?token=${token.jwt}`);<br>    } catch (err) {<br>      res.status(500).send({ success: false, message: err.message });<br>    }<br>  }</pre><p>In the controller, I created a GET method with the ‘callback/google’ endpoint that uses the previously created Google guard, this endpoint will redirect the user to the Google login page.</p><p><strong>auth.service.ts</strong></p><pre>  async oAuthLogin(user) {<br>    if (!user) {<br>      throw new Error(&#39;User not found!!!&#39;);<br>    }<br><br>//    .... your business logic<br><br>//   .... add whatever payload you want to have<br>    const payload = {<br>      email: user.email,<br>      name: user.name,<br>    };<br><br>    const jwt = await this.jwtService.sign(payload);<br><br>    return { jwt };<br>  }</pre><p>The oAuthLogin method receives a user object. Your business logic could be, to look for a user by email, if it does not find it, it means that the user is not registered in the database, you must register it according to the user model that you have defined, then register the user in our database and return a JWT with the user’s ID and email. If the user exists, we simply return the JWT by signing it with the payload.</p><h3>NextJs Code</h3><p>In Nextjs, I have a Login page in app directory that uses both the normal login method using form and other directly using Google OAuth.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S9-LAP4jCDk6czvTsJ_6Fw.png" /></figure><p>Both of these methods would eventually return a JWT token that can be saved in the browser, for security purpose you can further do encryption of those tokens.</p><p>To call the backend API that directs to Google server, I have simply used a function that handles the click for Sign In With Google button.</p><pre>  const handleOauth = async (event: any) =&gt; {<br>    event.preventDefault();<br>    window.location.href = `${Backend_URL}/auth/google/callback`;<br>  };<br><br></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y2pApqK_nRQN8bybDqcWPw.png" /></figure><p>To get the token from the API redirect URL, you can use middleware in NextJS. I will save the token from the params to the cookie. On each request middleware could easily check the existing token. Down below is the middleware logic that I have used.</p><p><strong>middleware.ts</strong></p><pre>import { NextRequest, NextResponse } from &quot;next/server&quot;;<br>import cookie from &quot;cookie&quot;;<br>import { parseJwt } from &quot;@/lib/Constants&quot;;<br><br>export function middleware(req: NextRequest) {<br>  const cookies = cookie.parse(req.headers.get(&quot;Cookie&quot;) || &quot;&quot;);<br>  const token = cookies.token;<br>  const tokenFromOauth = req.cookies.get(&quot;token&quot;);<br><br>  let tokenData;<br><br>  if (token) {<br>    tokenData = parseJwt(token.toString());<br>  }<br><br>  if (tokenFromOauth) {<br>    tokenData = parseJwt(tokenFromOauth.value.toString());<br>  }<br><br>  if (<br>    !token &amp;&amp;<br>    !tokenFromOauth &amp;&amp;<br>    req.nextUrl.pathname.startsWith(&quot;/recruiter&quot;)<br>  ) {<br>    console.log(&quot;cant enter recruiter &quot;);<br>    return NextResponse.redirect(new URL(&quot;/login&quot;, req.url));<br>  }<br><br>  if (req.nextUrl.pathname.startsWith(&quot;/oauth&quot;)) {<br>    const oAuthToken = req.nextUrl.searchParams.get(&quot;token&quot;) || &quot;&quot;;<br><br>    if (oAuthToken.length &gt; 0) {<br>   <br>      const response = NextResponse.next();<br><br>      response.cookies.set({<br>        name: &quot;token&quot;,<br>        value: oAuthToken.toString(),<br>        maxAge: 60 * 60 * 24 * 7,<br>      });<br><br>      return NextResponse.redirect(new URL(&quot;/recruiter&quot;, req.url));<br>    }<br>  }<br><br><br>  return NextResponse.next();<br>}</pre><h3>Conclusion</h3><p>Congratulations on seamlessly integrating Google Login into your NestJS and NextJS application! This enhancement not only simplifies the authentication process but also elevates the overall user experience, while simultaneously fortifying the security measures of your application.</p><h3>Stackademic</h3><p><em>Thank you for reading until the end. Before you go:</em></p><ul><li><em>Please consider </em><strong><em>clapping</em></strong><em> and </em><strong><em>following</em></strong><em> the writer! 👏</em></li><li><em>Follow us on </em><a href="https://twitter.com/stackademichq"><strong><em>Twitter(X)</em></strong></a><em>, </em><a href="https://www.linkedin.com/company/stackademic"><strong><em>LinkedIn</em></strong></a><em>, and </em><a href="https://www.youtube.com/c/stackademic"><strong><em>YouTube</em></strong></a><strong><em>.</em></strong></li><li><em>Visit </em><a href="http://stackademic.com/"><strong><em>Stackademic.com</em></strong></a><em> to find out more about how we are democratizing free programming education around the world.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e70d98b0f898" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/use-google-oauth-in-nextjs-and-nestjs-e70d98b0f898">Use Google OAuth In NextJS And NestJS</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Effortless SQL Database Restoration with Dockerode in Node.js/Nest.js]]></title>
            <link>https://blog.stackademic.com/effortless-sql-database-restoration-with-dockerode-in-node-js-nest-js-546aacdbdcdb?source=rss-72ea816b21a------2</link>
            <guid isPermaLink="false">https://medium.com/p/546aacdbdcdb</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[nestjs]]></category>
            <category><![CDATA[docker-remote-api]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[postgres]]></category>
            <dc:creator><![CDATA[Syed Muhammed Hassan Ali]]></dc:creator>
            <pubDate>Thu, 23 Nov 2023 06:16:31 GMT</pubDate>
            <atom:updated>2023-11-23T06:16:31.973Z</atom:updated>
            <content:encoded><![CDATA[<h4>Utilizing Remote Docker API to Seamlessly Restore (Import) Postgres Dump File Within a Container</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9vSeA0TsOCJ0hu2J.png" /><figcaption><strong>The diagram depicts the container states accessible through the API</strong></figcaption></figure><blockquote>This article assumes a basic understanding of docker and its ecosystem.</blockquote><h4>Introduction</h4><p>I found the concept of containers fascinating and delved into their possibilities. In one experiment, I faced the challenge of efficiently importing a database from the client side while leveraging the power of Docker containers. Initially, I resorted to manually uploading the database dump using PgAdmin within a PostgreSQL container. However, realizing the need for a more automated solution, I decided to implement a script that could seamlessly execute the database restoration process. Through the Docker Remote API and <a href="https://www.npmjs.com/package/dockerode"><strong>Dockerode</strong></a>, I managed to create a script that not only handled the complexities of proper mounting but also executed the necessary commands within the PostgreSQL container. This automated approach proved invaluable, especially in a multi-container environment, eliminating the manual effort of uploading database dump files.</p><p>Initially, I had some trouble because this part in the Remote API is a little bit tricky but after some time I finally achieved success and that’s why I wish to contribute my findings on this topic.</p><p>Later, in the implementation section, I will cover the execution details of one-off commands running inside Docker containers.</p><h4>Background</h4><p>Docker has a daemon server that keeps running on your machine and it exposes a Remote API for managing your containers. The most popular way to interact with containers is by using the Docker CLI tool.</p><p>If you ever used docker and its CLI, it’s for sure you’ve run into this command:</p><p>docker run &lt;options&gt; &lt;image&gt;.</p><p>The interesting part is that the Docker CLI uses the <a href="https://docs.docker.com/engine/api/v1.43/">Docker Remote API</a> under the hood. Additionally, the <strong>run</strong> command does not have a direct corresponding API endpoint to run containers, and it is composed of three main steps through the API that <a href="https://docs.docker.com/engine/api/v1.43/#tag/Container/operation/ContainerCreate"><strong>create</strong></a>, <a href="https://docs.docker.com/engine/api/v1.43/#tag/Container/operation/ContainerAttach"><strong>attach</strong></a> and <a href="https://docs.docker.com/engine/api/v1.43/#tag/Container/operation/ContainerStart"><strong>start</strong></a><strong> </strong>your containers.</p><h4>How `docker run` CLI command is composed under the hood</h4><blockquote>As I stated before, the Docker CLI abstracts the feature of “running a container”. That way we can have a better and easier experience while not having to deal with each of the required steps separately.</blockquote><p>When you execute docker run, it undergoes a specific sequence. Initially, it invokes the create command, generating a container with a unique ID, yet preserving it in a stopped state. Following this, the process seamlessly transitions to the attach command, establishing a connection to the container&#39;s input/output for interactive communication. After successful attachment, the start command is employed, setting the container in motion, executing designated tasks, and showcasing the resulting response. By default, the container remains stopped post-execution. However, incorporating the --rm option ensures the automatic destruction of the container upon task completion.</p><h3>Implementation</h3><h4>Description</h4><p>The goal of this implementation is first to test a basic API, that is to run containers. Then move towards the main implementation of restoring a SQL database by executing commands inside a container using remote API. I am using <a href="https://github.com/apocas/dockerode"><strong>Dockerode</strong></a>, which abstracts direct access to the Docker API and provides some utilities like streaming stdin/stdout through the <strong>TCP</strong> connection. We also have other <a href="https://docs.docker.com/engine/api/sdk/"><strong>SDKs</strong></a> for different languages.</p><p><strong>docker-compose.yaml</strong></p><pre>version: &#39;3&#39;<br><br>services:<br>  nest-backend:<br>    container_name: nest-app<br>    image: nest-api<br>    build:<br>      dockerfile: Dockerfile<br>      context: .<br>    ports:<br>      - 5000:5000<br>    environment:<br>      - DB_TYPE=postgres<br>      - PG_HOST=postgres<br>      - PG_USER=postgres<br>      - PG_PASSWORD=postgres<br>      - PG_DB=postgres<br>      - PG_PORT=5432<br>    networks:<br>      - backend_network<br>    volumes:<br>      - .:/usr/src/app<br>      - /usr/src/app/node_modules<br><br>    depends_on:<br>      - postgres<br><br>  postgres:<br>    container_name: postgres-db<br>    image: postgres:12<br>    ports:<br>      - &#39;5432:5432&#39;<br>    environment:<br>      - POSTGRES_USER=postgres<br>      - POSTGRES_PASSWORD=postgres<br>      - POSTGRES_DB=postgres<br>    networks:<br>      - backend_network<br>    volumes:<br>      - postgres_data:/var/lib/postgresql/data<br>      - ./src/dbDumpFile:/dbDumpFile<br>      - /var/run/docker.sock:/var/run/docker.sock<br><br>  pgadmin:<br>    container_name: pgadmin<br>    image: dpage/pgadmin4<br>    restart: always<br>    environment:<br>      PGADMIN_DEFAULT_EMAIL: admin@pgadmin.com<br>      PGADMIN_DEFAULT_PASSWORD: admin<br>    networks:<br>      - backend_network<br>    ports:<br>      - &#39;5050:80&#39;<br>    depends_on:<br>      - postgres<br><br>networks:<br>  backend_network:<br>    driver: bridge<br><br>volumes:<br>  postgres_data: {}</pre><blockquote>The <strong>`/var/run/docker.sock`</strong> is the Docker daemon socket and is the main entry point for the Docker remote API. By mounting this socket into services with `- /var/run/docker.sock:/var/run/docker.sock`, we’re allowing the Docker client in our services to communicate with the Docker daemon running on the host machine.</blockquote><blockquote>In our case, Dockerode, which is a Docker client for Node.js, uses this socket to send commands to the Docker daemon. So, if your application running in the `nest-backend` or `postgres` service needs to manage Docker containers (like starting or stopping containers), it needs access to this Docker socket.</blockquote><p>Install the required package, then use <strong>docker-compose up</strong> to run the containers.</p><pre>npm i dockerode @types/dockerode</pre><p>Now import the package.</p><pre>import * as Docker from &#39;dockerode&#39;;</pre><p><strong>Code</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0028f13f5e37d728b014ca3b1e61ede2/href">https://medium.com/media/0028f13f5e37d728b014ca3b1e61ede2/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f4008eba21307d0f8d3724c483a8e998/href">https://medium.com/media/f4008eba21307d0f8d3724c483a8e998/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_cunYq0dMyk2ORWVYuK4bw.jpeg" /><figcaption><strong>API returned a response after running a hello-world image in a container</strong></figcaption></figure><p>Before moving towards the API that restores the Postgres database dump file that I already added in <strong>src/dbDumpFile/postgres.tar </strong>directory, have a look at the initial way of restoring/importing the database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y9pPJpMs3Fr--KgwTXywkA.jpeg" /><figcaption><strong>Manually uploading the dump file.</strong></figcaption></figure><p>Let&#39;s automate the above process.</p><blockquote>Since Postgres container has to access this dump file, I have mounted that folder in the container by adding in volumes.</blockquote><pre>postgres:<br># ... other postgres container configs<br>   volumes:<br>    - postgres_data:/var/lib/postgresql/data<br>    - ./src/dbDumpFile:/dbDumpFile<br>    - /var/run/docker.sock:/var/run/docker.sock</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/708/1*AzpVmPWSx7Nqj7UX-VPUYw.png" /><figcaption><strong>dbDumpFile folder that has been successfully mounted in the container.</strong></figcaption></figure><p><strong>Code</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/479bded6c8c366dc04cdbbbaadc64666/href">https://medium.com/media/479bded6c8c366dc04cdbbbaadc64666/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4238fed724b6c7498b26420f858f0a7d/href">https://medium.com/media/4238fed724b6c7498b26420f858f0a7d/href</a></iframe><blockquote>The <strong>`demuxStream`</strong> function is used to separate the output stream of the Docker exec command into stdout and stderr.</blockquote><blockquote>In the code, `demuxStream` is used to separate the output of the `pg_restore` command into stdout and stderr, and then log them to the console.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PhxrmV37YFTyfjB7-JHIMA.jpeg" /><figcaption><strong>Run the API to execute the commands.</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/806/1*VSjRMeZ9ANE1rKL03GgQeA.png" /><figcaption><strong>Logs that show successful restoration of the database</strong>.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*HqiTXT67DARePwKETMbG4g.png" /><figcaption><strong>PgAdmin now shows the restored tables.</strong></figcaption></figure><h3>Wrapping up</h3><p>Docker containers enable lots of cool possibilities like the example of this article which is really simple, but we could create an application that would act upon receiving data and perform complex operations with it like code compilation and runtime execution. We could spawn containers on demand to deal with some tasks, and if we think about it, it is what serverless actually is. In the serverless world, containers can be scaled on demand. However, the process of spawning containers should be managed by container orchestrators like Kubernetes or Docker Swarm.</p><p>That is how I developed an API using Dockerode to restore a PostgreSQL database dump file within the PostgreSQL container. The API utilizes Remote API calls for efficient interaction, automating the process of running pg_restore commands for seamless database importing.</p><p>Access the complete source code at <a href="https://github.com/Syed007Hassan/Restoring-Database-Using-Dockerode">https://github.com/Syed007Hassan/Restoring-Database-Using-Dockerode</a>.</p><h3>Stackademic</h3><p><em>Thank you for reading until the end. Before you go:</em></p><ul><li><em>Please consider </em><strong><em>clapping</em></strong><em> and </em><strong><em>following</em></strong><em> the writer! 👏</em></li><li><em>Follow us on </em><a href="https://twitter.com/stackademichq"><strong><em>Twitter(X)</em></strong></a><em>, </em><a href="https://www.linkedin.com/company/stackademic"><strong><em>LinkedIn</em></strong></a><em>, and </em><a href="https://www.youtube.com/c/stackademic"><strong><em>YouTube</em></strong></a><strong><em>.</em></strong></li><li><em>Visit </em><a href="http://stackademic.com/"><strong><em>Stackademic.com</em></strong></a><em> to find out more about how we are democratizing free programming education around the world.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=546aacdbdcdb" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/effortless-sql-database-restoration-with-dockerode-in-node-js-nest-js-546aacdbdcdb">Effortless SQL Database Restoration with Dockerode in Node.js/Nest.js</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Docker Compose to Run NestJS Applications with Redis and Postgres]]></title>
            <link>https://blog.stackademic.com/using-docker-compose-to-run-nestjs-applications-with-redis-and-postgres-586ab132b60c?source=rss-72ea816b21a------2</link>
            <guid isPermaLink="false">https://medium.com/p/586ab132b60c</guid>
            <category><![CDATA[nestjs]]></category>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[redis]]></category>
            <category><![CDATA[docker-compose]]></category>
            <category><![CDATA[typescript]]></category>
            <dc:creator><![CDATA[Syed Muhammed Hassan Ali]]></dc:creator>
            <pubDate>Sun, 15 Oct 2023 20:19:48 GMT</pubDate>
            <atom:updated>2024-09-25T15:47:30.286Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iBbJLQkkmTwd2Io8" /></figure><h3>Introduction</h3><p>Developing web applications, especially open-source ones, can be a complex task, particularly when configuring a local development environment that closely resembles the production environment. Coordinating various technologies and databases can be quite challenging. However, Docker Compose simplifies this process, enabling the creation of a consistent and easily maintainable local development setup. This blog delves into the use of Docker Compose to establish a local development environment for a web application constructed with NestJS, Redis, and Postgres, showcasing the crucial role Docker plays in setting up open-source applications</p><h3>Why Docker Compose?</h3><p>Before diving into the details, let’s briefly discuss why Docker Compose is a valuable tool for local development. Docker Compose is a container orchestration tool that allows you to define and run multi-container Docker applications. It lets you define the services, networks, and volumes needed for your application in a single, declarative configuration file. This means you can encapsulate your entire development environment in code, making it easy to share and reproduce across different machines.</p><blockquote>If you are looking for a more powerful and scalable container orchestration tool, I recommend using Kubernetes. However, if you are just getting started with containerization or you need a simple and easy-to-use tool, Docker Compose is a good option.</blockquote><p><a href="https://differ.blog/differ-ai">Introducing DifferAI</a></p><h3>The Technology Stack</h3><p>For our local development environment, we’ll be working with the following technologies:</p><ul><li><strong>NestJS:</strong> A powerful Node.js framework for building server-side applications, using TypeScript for strong typing and maintainability.</li><li><strong>Redis:</strong> An in-memory data store often used for caching and session management in web applications.</li><li><strong>PostgreSQL:</strong> A relational database management system that’s excellent for storing structured data.</li></ul><h3>Getting started</h3><p>For this example, I will be using a <a href="https://github.com/Syed007Hassan/NestJs-Template/tree/cba8dc7b120130e210930152c9dd71a081a90503/Template-1">template codebase</a> that includes a pre-configured setup with TypeORM and PostgreSQL, an Authentication module with JWT, and a Swagger API client.</p><h3>Prerequisites</h3><p><a href="https://www.docker.com/products/docker-desktop/">Docker for Desktop</a> must be installed on supported operating systems.</p><h4>Step 1: Create a Dockerfile</h4><p>A Dockerfile is essential for creating self-contained, consistent, and reproducible environments for applications within Docker containers. It specifies the exact steps to set up an application’s runtime environment, ensuring that it runs consistently across different systems and platforms. This level of control and predictability is crucial for efficient development, testing, and deployment in containerized environments.</p><p><strong>Dockerfile</strong></p><pre># Use a Node.js Alpine-based image for the development stage<br>FROM node:18-alpine<br><br># Set the working directory in the container<br>WORKDIR /usr/src/app<br><br># Copy application dependency manifests to the container image<br>COPY package*.json ./<br><br># Install application dependencies using `npm install`<br>RUN npm install<br><br># Copy the rest of the application code to the container<br>COPY . .<br><br># Build the application (if needed)<br>RUN npm run build<br><br># Define the command to start your application in development mode<br>ENTRYPOINT [&quot;/bin/sh&quot;, &quot;-c&quot;, &quot;npm run start:dev&quot;]</pre><h4>Step 2: Create a Docker-Compose file</h4><p>The <strong>docker-compose.yml</strong> file lays out the instructions on how to run image(s) into containers - exactly what we need to spin up a local development environment.</p><p>Let’s add the instructions in the <strong>docker-compose.yml</strong> file to spin up a local development environment (starting with just the NestJS app).</p><p><strong>docker-compose.yml</strong></p><pre>services:<br>  nest-backend:<br>    container_name: nest-app<br>    image: nest-api<br>    build:<br>      dockerfile: Dockerfile<br>      context: .<br>    ports:<br>      - 5000:5000<br>    networks:<br>      - backend_network<br>    volumes:<br>      - .:/usr/src/app<br>      - /usr/src/app/node_modules<br>    restart: unless-stopped</pre><p>Let’s highlight a few important parts of this file:</p><ul><li>Defines a service named <strong>nest-backend</strong> to run a Docker container for a Nest.js application.</li><li>Specifies the container name as <strong>nest-app</strong> and the image as <strong>nest-api</strong>for the container.</li><li>Uses a Dockerfile in the <strong>current context</strong> to build the container.</li><li><strong>Maps port</strong> 5000 from the host to port 5000 within the container for accessing the application.</li><li>Connects the container to a network named <strong>backend_network</strong> for communication.</li><li>Sets up <strong>volumes</strong> to mount the current directory and node_modules into the container, allowing code changes and sharing dependencies.</li><li>Configures the container to <strong>restart automatically</strong>unless explicitly stopped.</li></ul><h4>Step 3: Add Postgres and PgAdmin service in docker-compose.yaml</h4><pre>services:<br>  nest-backend:<br>    container_name: nest-app<br>    image: nest-api<br>    build:<br>      dockerfile: Dockerfile<br>      context: .<br>    ports:<br>      - 5000:5000<br>    environment:<br>      - DB_TYPE=postgres<br>      - DB_SCHEMA=public<br>      - PG_HOST=postgres<br>      - PG_USER=postgres<br>      - PG_PASSWORD=postgres<br>      - PG_DB=postgres<br>      - PG_PORT=5432<br>    networks:<br>      - backend_network<br>    volumes:<br>      - .:/usr/src/app<br>      - /usr/src/app/node_modules<br>    restart: unless-stopped<br>    depends_on:<br>      - postgres<br>   <br>  postgres:<br>    container_name: postgres-db<br>    image: postgres:12<br>    ports:<br>      - 5432:5432<br>    environment:<br>      - POSTGRES_USER=postgres<br>      - POSTGRES_PASSWORD=postgres<br>      - POSTGRES_DB=postgres<br>    networks:<br>      - backend_network<br>    restart: unless-stopped<br>    volumes:<br>      - postgres_data:/var/lib/postgresql/data<br><br>  pgadmin:<br>    container_name: pgadmin<br>    image: dpage/pgadmin4<br>    restart: always<br>    environment:<br>      PGADMIN_DEFAULT_EMAIL: admin@pgadmin.com<br>      PGADMIN_DEFAULT_PASSWORD: admin<br>    networks:<br>      - backend_network<br>    ports:<br>      - &#39;5050:80&#39;<br>    depends_on:<br>      - postgres<br><br>networks:<br>  backend_network:<br>    driver: bridge<br><br>volumes:<br>  postgres_data: {}</pre><ul><li>Defines two additional services: <strong>postgres</strong> for a PostgreSQL database and <strong>pgdmin</strong> for the pgAdmin tool.</li><li>Specifies the container names, images, and Dockerfiles for building containers. nest-backend <strong>depends</strong> on a Postgres container.</li><li>Sets environment variables for <strong>nest-backend</strong> to configure the connection to a PostgreSQL database.</li><li>Connects all services to a common network named backend_network for <strong>inter-container communication</strong>.</li><li>Defines volumes to persist data: <strong>postres_data</strong> for PostgreSQL database data.</li><li>PgAdmin is included for managing PostgreSQL and runs on port <strong>5050</strong>.</li></ul><p>The ormConfig.ts file would use all the environmental variables mentioned in docker-compose.yaml file.</p><p><strong>src/config/ormConfig.ts</strong></p><pre>import { configDotenv } from &#39;dotenv&#39;;<br>import { User } from &#39;src/entities/user.entity&#39;;<br>import { TypeOrmModuleOptions } from &#39;@nestjs/typeorm&#39;;<br>import { Topic } from &#39;src/entities/topic.entity&#39;;<br>import { Comment } from &#39;src/entities/comment.entity&#39;;<br><br>configDotenv();<br><br>export const PostgreSqlDataSource: TypeOrmModuleOptions = {<br>  type: process.env.DB_TYPE<br>  host: process.env.PG_HOST,<br>  port: parseInt(process.env.PG_PORT),<br>  username: process.env.PG_USER,<br>  password: process.env.PG_PASSWORD,<br>  database: process.env.PG_DB,<br>  schema: process.env.DB_SCHEMA,<br>  entities: [User, Topic, Comment],<br>  autoLoadEntities: true,<br>  synchronize: true,<br>  logging: true,<br>};</pre><h4>Step 4: Setup CacheModule configurations and add Redis service</h4><p>Here’s a CacheModule implementation applied globally to a NestJS app in the <strong>app.module.ts</strong> file:</p><p><strong>src/app.module.ts</strong></p><pre>import { Module } from &#39;@nestjs/common&#39;;<br>import { AppController } from &#39;./app.controller&#39;;<br>import { AppService } from &#39;./app.service&#39;;<br>import { MongooseModule } from &#39;@nestjs/mongoose&#39;;<br>import { ConfigModule} from &#39;@nestjs/config&#39;;<br>import { AuthModule } from &#39;./auth/auth.module&#39;;<br>import { UserModule } from &#39;./user/user.module&#39;;<br>import { PostgreSqlDataSource } from &#39;./config/ormConfig&#39;;<br>import { TypeOrmModule } from &#39;@nestjs/typeorm&#39;;<br>import { CacheModule } from &#39;@nestjs/cache-manager&#39;;<br>import * as redisStore from &#39;cache-manager-redis-store&#39;;<br><br>@Module({<br>  imports: [<br>    ConfigModule.forRoot({<br>      isGlobal: true,<br>      envFilePath: `.env`,<br>    }),<br>    CacheModule.register({<br>      isGlobal: true,<br>      store: redisStore,<br>      host: &#39;redis&#39;,<br>      port: 6379,<br>    }),<br>    TypeOrmModule.forRoot(PostgreSqlDataSource),<br>    AuthModule,<br>    UserModule,<br>  ],<br>  controllers: [AppController],<br>  providers: [AppService],<br>  exports: [ConfigModule],<br>})<br>export class AppModule {}</pre><ul><li><strong>CacheModule.register</strong>: This method registers and configures the caching module, indicating the choice of Redis as the caching store.</li><li><strong>isGlobal: true</strong>: By setting <strong>isGlobal</strong> to true, the cache module becomes accessible application-wide, enabling any module within the application to access and utilize the caching functionality.</li><li><strong>store: redisStore</strong>: The configuration specifies the use of Redis as the caching store.</li><li><strong>host: &#39;redis&#39; and port: 6379</strong>: These properties define the host and port of the Redis server. In this instance, the Redis server is configured to connect on the same Docker network, with the hostname &quot;redis&quot; and port 6379.</li></ul><p>In terms of tweaking the docker-compose file to set up a local Redis server, you can make the following changes:</p><pre>services:<br>  nest-backend:<br>    # ... other Nest container configs<br>    depends_on:<br>      - postgres<br>      - redis<br><br>  postgres:<br>    # ... other postgres container configs<br><br>  pgadmin:<br>   # ... other pgadmin container configs<br><br>  redis:<br>    container_name: redis-db<br>    image: redis<br>    environment:<br>      - REDIS_PORT=6379<br>    ports:<br>      - 6379:6379<br>    networks:<br>      - backend_network<br>    restart: unless-stopped<br>    volumes:<br>      - redis:/data<br><br>networks:<br>  backend_network:<br>    driver: bridge<br><br>volumes:<br>  postgres_data: {}<br>  redis:<br>    driver: local</pre><ul><li>Inside the <strong>nest-backend</strong> service, include <strong>redis</strong> as a <strong>depends_on</strong> dependency. This will ensure that the Redis container starts before the API container.</li><li>A new container is named <strong>redis</strong> which uses the Redis docker image.</li><li>The <strong>volumes</strong> configurations enable the cache to persist between container restarts</li></ul><h4>Step 5: Build the Dockerfile</h4><blockquote>Please note that in my setup using TypeScript version 5.1.3, enabling hot reload requires making specific configurations in the tsconfig.json file.</blockquote><p><strong>tsconfig.json</strong></p><pre>&quot;watchOptions&quot;: {<br>    // Use native file system events for files and directories<br>    &quot;watchFile&quot;: &quot;priorityPollingInterval&quot;,<br>    &quot;watchDirectory&quot;: &quot;dynamicprioritypolling&quot;,<br>    // Poll files for updates more frequently<br>    // when they&#39;re updated a lot.<br>    &quot;fallbackPolling&quot;: &quot;dynamicPriority&quot;,<br>    // Don&#39;t coalesce watch notification<br>    &quot;synchronousWatchDirectory&quot;: true,<br>    // Finally, two additional settings for reducing the amount of possible<br>    // files to track  work from these directories<br>    &quot;excludeDirectories&quot;: [&quot;**/node_modules&quot;, &quot;dist&quot;]<br>  }</pre><p>The complete docker-compose is:</p><p><strong>docker-compose.yaml</strong></p><pre>services:<br>  nest-backend:<br>    container_name: nest-app<br>    image: nest-api<br>    build:<br>      dockerfile: Dockerfile<br>      context: .<br>    ports:<br>      - 5000:5000<br>    environment:<br>      - DB_TYPE=postgres<br>      - DB_SCHEMA=public<br>      - PG_HOST=postgres<br>      - PG_USER=postgres<br>      - PG_PASSWORD=postgres<br>      - PG_DB=postgres<br>      - PG_PORT=5432<br>    networks:<br>      - backend_network<br>    volumes:<br>      - .:/usr/src/app<br>      - /usr/src/app/node_modules<br>    restart: unless-stopped<br>    depends_on:<br>      - postgres<br>      - redis<br><br>  redis:<br>    container_name: redis-db<br>    image: redis<br>    environment:<br>      - REDIS_PORT=6379<br>    ports:<br>      - 6379:6379<br>    networks:<br>      - backend_network<br>    restart: unless-stopped<br>    volumes:<br>      - redis:/data<br><br>  postgres:<br>    container_name: postgres-db<br>    image: postgres:12<br>    ports:<br>      - 5432:5432<br>    environment:<br>      - POSTGRES_USER=postgres<br>      - POSTGRES_PASSWORD=postgres<br>      - POSTGRES_DB=postgres<br>    networks:<br>      - backend_network<br>    restart: unless-stopped<br>    volumes:<br>      - postgres_data:/var/lib/postgresql/data<br><br>  pgadmin:<br>    container_name: pgadmin<br>    image: dpage/pgadmin4<br>    restart: always<br>    environment:<br>      PGADMIN_DEFAULT_EMAIL: admin@pgadmin.com<br>      PGADMIN_DEFAULT_PASSWORD: admin<br>    networks:<br>      - backend_network<br>    ports:<br>      - &#39;5050:80&#39;<br>    depends_on:<br>      - postgres<br><br>networks:<br>  backend_network:<br>    driver: bridge<br><br>volumes:<br>  postgres_data: {}<br>  redis:<br>    driver: local</pre><p>Now use <strong>docker-compose build --no-cache</strong> to ensure a completely clean and fresh build of your Docker containers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DM2l5OlFUK314qO2s-dlag.png" /><figcaption><strong>A complete rebuild without using cached layers</strong></figcaption></figure><p>After the build is complete, you can use <strong>docker-compose up</strong> to start your containers using the newly built images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*x_DmaVjZMQl3_Fmb3h_bWw.png" /><figcaption><strong>Docker containers are up and running</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*Jvh8sbNoBZvUJv8zC7BcvA.png" /><figcaption><strong>Docker Desktop shows all the running containers</strong></figcaption></figure><h4>Step 6: Access each service on the mapped port</h4><p>On <strong>localhost:5000</strong>, we can access the NestJS server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*033_iXh-xfjN0DHJX9RgLg.png" /><figcaption><strong>Swagger allows you to describe the structure of your APIs</strong></figcaption></figure><p>Validating with <strong>Auth APIs</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/751/1*ErDoysvlPLIR0V6RUrIdxQ.png" /><figcaption><strong>Registering a user</strong></figcaption></figure><p>Now finding the user using PgAdmin that can be accessed on <strong>localhost:5050</strong> . Make sure to use the email and password mentioned in the docker-compose file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n1ixWxgmGlX3sclWyLgckQ.png" /><figcaption><strong>User found on get query</strong></figcaption></figure><p>In the Redis demo, I’ve implemented an API in the <strong>user.controller.ts</strong> file. Within this API, I make calls to another API, and I&#39;ve implemented caching to store and retrieve the response.</p><p><strong>src/user/user.controller.ts</strong></p><pre>  @Get(&#39;/:id&#39;)<br>  async getPokemon(@Param(&#39;id&#39;) id: number) {<br>    try {<br>      const data = await this.userService.getPokemon(id);<br>      return { success: true, data: data };<br>    } catch (err) {<br>      return { success: false, message: err.message };<br>    }<br>  }</pre><p><strong>src/user/user.service.ts</strong></p><pre>  async getPokemon(id: number): Promise&lt;string&gt; {<br>    // check if data is in cache:<br>    const cachedData = await this.cacheService.get&lt;{ name: string }&gt;(<br>      id.toString(),<br>    );<br>    if (cachedData) {<br>      console.log(`Getting data from cache!`);<br>      return `${cachedData.name}`;<br>    }<br><br>    // if not, call API and set the cache:<br>    const { data } = await this.httpService.axiosRef.get(<br>      `https://pokeapi.co/api/v2/pokemon/${id}`,<br>    );<br>    await this.cacheService.set(id.toString(), data);<br>    return await `${data.name}`;<br>  }</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/1*x7z2OyCS2EZrvnP0IZYqlw.png" /><figcaption><strong>Response without caching</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/915/1*J4MsvKm-Nmsj0m_19rz77g.png" /><figcaption><strong>Response with caching</strong></figcaption></figure><p>Using Redis store, the API response time was reduced from <strong>653</strong> ms to <strong>14</strong> ms.</p><h3>Conclusion</h3><p>In this blog, we’ve explored how to set up a local development environment for a web application built with NestJS, Redis, and Postgres using Docker Compose. This approach simplifies the development workflow, improves consistency, and enhances portability. It’s a powerful tool for any developer looking to create a reliable, reproducible, and efficient local development environment.</p><p>By embracing Docker Compose, you can streamline your development process and focus on building high-quality applications.</p><p>Access the complete source code at <a href="https://github.com/Syed007Hassan/NestJs-Template">https://github.com/Syed007Hassan/NestJs-Template</a><a href="https://github.com/Syed007Hassan/NestJs-Template">.</a></p><p>If you enjoyed this article, please click on the clap button 👏 and share to help others find it!</p><h3>Stackademic 🎓</h3><p>Thank you for reading until the end. Before you go:</p><ul><li>Please consider <strong>clapping</strong> and <strong>following</strong> the writer! 👏</li><li>Follow us <a href="https://twitter.com/stackademichq"><strong>X</strong></a> | <a href="https://www.linkedin.com/company/stackademic"><strong>LinkedIn</strong></a> | <a href="https://www.youtube.com/c/stackademic"><strong>YouTube</strong></a> | <a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a></li><li>Visit our other platforms: <a href="https://plainenglish.io/"><strong>In Plain English</strong></a> | <a href="https://cofeed.app/"><strong>CoFeed</strong></a> | <a href="https://differ.blog/"><strong>Differ</strong></a></li><li>More content at <a href="https://stackademic.com/"><strong>Stackademic.com</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=586ab132b60c" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/using-docker-compose-to-run-nestjs-applications-with-redis-and-postgres-586ab132b60c">Using Docker Compose to Run NestJS Applications with Redis and Postgres</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Document Querying with LLMs — Google PaLM API]]></title>
            <link>https://medium.com/@syed007hassan/document-querying-with-llms-google-palm-api-9ba73a935e5?source=rss-72ea816b21a------2</link>
            <guid isPermaLink="false">https://medium.com/p/9ba73a935e5</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <category><![CDATA[vector-database]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Syed Muhammed Hassan Ali]]></dc:creator>
            <pubDate>Fri, 08 Sep 2023 10:48:29 GMT</pubDate>
            <atom:updated>2023-09-08T13:32:50.564Z</atom:updated>
            <content:encoded><![CDATA[<h3>Document Querying with LLMs — Google PaLM API (Semantic Search With LLM Embeddings)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*3UgHhpZZyv0mkoAj.png" /></figure><h3>Introduction</h3><p>With Large Language Models (LLMs), we can integrate domain-specific data to answer questions. This is especially useful for data unavailable to the model during its initial training, like a company’s internal documentation or knowledge base.</p><p>The architecture is called Retrieval Augmentation Generation or less commonly used <em>Generative Question Answering.</em></p><p>This article helps you understand how to implement this architecture using LLMs and a Vector Database. We can significantly decrease the hallucinations that are commonly associated with LLMs.</p><p>It can be used for a wide range of use cases. It reduces the time we need to interact with documents. There is no need for us anymore to search for answers in search results. The LLM takes care of precisely finding the most relevant documents and using them to generate the answer right from your documents.</p><h3>Apply LLMs to your domain-specific data</h3><p>In this example, I will be using Google PaLM API that will help <a href="https://developers.generativeai.google/examples/doc_search_emb">document search with embeddings</a> and the usage of <a href="https://developers.generativeai.google/examples/vectordb_with_chroma">VectorDB with Chroma.</a></p><h3>What is a Vector Database?</h3><p>Efficient data processing has become more crucial than ever for applications that involve large language models, generative AI, and semantic search.</p><p>All of these new applications rely on <a href="https://www.pinecone.io/learn/vector-embeddings-for-developers/"><strong>vector embeddings</strong></a>, a type of data representation that carries within it semantic information that’s critical for the AI to gain understanding and maintain a long-term memory they can draw upon when executing complex tasks.</p><p><strong>Embeddings</strong> are generated by AI models (such as Large Language Models) and have a large number of attributes or features, making their representation challenging to manage. In the context of AI and machine learning, these features represent different dimensions of the data that are essential for understanding patterns, relationships, and underlying structures.</p><p>With a vector database, we can add advanced features to our AIs, like semantic information retrieval, long-term memory, and more. The diagrams below give us a better understanding of the role of vector databases in this type of application:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OpWfxh65AMfSTaTs.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uFwaYl31AgYNWUrv.png" /></figure><h3>Let&#39;s jump to the Code</h3><h3>Step 1: Enable Needed APIs</h3><p>Run gcloud init to authenticate with your GCP user and project.</p><p>Enable the necessary APIs:</p><pre>gcloud services enable aiplatform.googleapis.com --async</pre><p>Then generate a PaLM key from the <a href="https://developers.generativeai.google/tutorials/setup">console.</a></p><h3>Step 2: Setup a Flask Project</h3><p>Although in this example I have used Flask, you can use any Python-based framework. You can also use <a href="https://github.com/Syed007Hassan/Flask-MVC-Template-">this template.</a></p><p>Install the necessary packages:</p><pre>pip install Flask Flask-Cors google-generativeai requests PyPDF2</pre><h3>Step 3: Encode any PDF</h3><p>The API developed will require an encoded PDF as a request body and a query parameter that specifies what the API should do with the document. Example:</p><pre>curl -X POST &quot;https://your-api-url.com/search-pdf?query=keyword&quot; \<br>  -H &quot;Content-Type: application/pdf&quot; \<br>  --data-binary &quot;@path/to/encoded.pdf&quot;</pre><p>In the routes file, define the API endpoint.</p><pre>from flask import Blueprint<br>from src.controllers.reorderController import getSolutions<br><br>reorderBlueprint = Blueprint(&#39;blueprintt&#39;, __name__)<br><br># suggesting solutions by reading PDF file<br><br>reorderBlueprint.route(&#39;/getSolutions/&lt;query&gt;&#39;, methods=[&#39;POST&#39;])(getSolutions)</pre><p>The controller forwards the API request to the service layer.</p><pre>from flask import jsonify, request<br>from src.services.palm_api_Service import get_solutions<br><br><br>def set_response_headers(response):<br>    response.headers[&#39;Access-Control-Allow-Origin&#39;] = &#39;*&#39;<br>    response.headers[&#39;Access-Control-Allow-Headers&#39;] = &#39;Origin, X-Requested-With, Content-Type, Accept&#39;<br>    return response<br><br><br>def getSolutions(query):<br>    try:<br>        pdf_content = request.get_data()<br>        solutions = get_solutions(pdf_content, query)<br>        structuredData = jsonify(data=solutions)<br>        response = set_response_headers(structuredData)<br>        return response, 200<br><br>    except Exception as e:<br>        return jsonify(error=str(e)), 500</pre><blockquote>Then comes the implementation of the service layer.</blockquote><p>Decode the original PDF of n pages to get back the content. And then pass the whole content to the vector database. Create a collection store where you store your embeddings, documents, and any metadata.</p><pre>@staticmethod<br>def get_solutions(pdf_content, query):<br>    try:<br>        # Decode the base64-encoded PDF content<br>        pdf_content_dict = json.loads(pdf_content)<br>        pdf_content_encoded = pdf_content_dict[&#39;pdfContentEncoded&#39;]<br>        pdf_content_decoded = base64.b64decode(pdf_content_encoded)<br><br>        # Read the content of each page of the PDF and concatenate them into a single string<br>        pdf_reader = PyPDF2.PdfReader(io.BytesIO(pdf_content_decoded))<br>        pdf_content = &quot;&quot;<br>        for page in range(len(pdf_reader.pages)):<br>            pdf_content += pdf_reader.pages[page].extract_text()<br><br>        # Set up the DB<br>        db = create_chroma_db([pdf_content], &quot;exampleDBCollection&quot;)<br><br>        temperature = 0.65<br><br>        response = answer(text_model, query, db, temperature)<br><br>        # Return the concatenated text<br>        return response<br>    except Exception as e:<br>        print(&quot;Error generating response: &quot; + str(e))<br>        return str(e)</pre><blockquote>Configure the PaLM with your API key and get the embeddings and text generation model</blockquote><pre>palm.configure(api_key=&#39;YOUR_API_KEY&#39;)<br><br><br>models = [m for m in palm.list_models(<br>) if &#39;embedText&#39; in m.supported_generation_methods]<br>model = models[0]<br><br><br>text_models = [m for m in palm.list_models(<br>) if &#39;generateText&#39; in m.supported_generation_methods]<br>text_model = text_models[0]</pre><p>You will create a <a href="https://docs.trychroma.com/embeddings#custom-embedding-functions">custom function</a> for performing embedding using the PaLM API. By inputting a set of documents into this custom function, you will receive vectors or embeddings of the documents.</p><pre>def embed_function(texts: Documents) -&gt; Embeddings:<br>    # Embed the documents using any supported method<br>    return [palm.generate_embeddings(model=model, text=text)[&#39;embedding&#39;]<br>            for text in texts]</pre><blockquote>Note that the embedding function from above is passed as an argument to the create_collection .</blockquote><pre><br><br>def create_chroma_db(documents, name):<br>    chroma_client = chromadb.Client()<br>    db = chroma_client.create_collection(<br>        name=name, embedding_function=embed_function)<br>    for i, d in enumerate(documents):<br>        db.add(<br>            documents=d,<br>            ids=str(i)<br>        )<br>    return db</pre><h3>Step 4: Getting the relevant document</h3><p>Db is a Chroma collection object. You can call query on it to perform a nearest neighbours search to find similar embeddings or documents.</p><pre>def get_relevant_passage(query, db):<br>    passage = db.query(query_texts=[query], n_results=1)[&#39;documents&#39;][0][0]<br>    return passage</pre><p>Now that you have found the relevant passage in your set of documents, you can use it to make a prompt to pass into the PaLM API.</p><pre>def make_prompt(query, relevant_passage):<br>    escaped = relevant_passage.replace(<br>        &quot;&#39;&quot;, &quot;&quot;).replace(&#39;&quot;&#39;, &quot;&quot;).replace(&quot;\n&quot;, &quot; &quot;)<br>    prompt = (&quot;&quot;&quot;You are a helpful and informative bot that answers questions using text from the reference passage included below. \<br>  Be sure to respond in a complete sentence, being comprehensive, including all relevant background information. \<br>  However, you are talking to a non-technical audience, so be sure to break down complicated concepts and \<br>  strike a friendly and converstional tone. \<br>  If the passage is irrelevant to the answer, you may ignore it.<br>  QUESTION: &#39;{query}&#39;<br>  PASSAGE: &#39;{relevant_passage}&#39;<br><br>    ANSWER:<br>  &quot;&quot;&quot;).format(query=query, relevant_passage=escaped)<br><br>    return prompt<br><br>def answer(model, query, db, temperature):<br>    passage = get_relevant_passage(query, db)<br>    prompt = make_prompt(query, passage)<br>    answer = palm.generate_text(prompt=prompt, model=model, candidate_count=3,<br>                                temperature=temperature, max_output_tokens=1024,)<br>    return answer.candidates[0][&#39;output&#39;]</pre><p>The answer function will generate a response based on the query you have passed in. It retrieves the relevant document, and from there calls the PaLM text generation API to generate a response to the query.</p><h3>Example API</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yIx0WFo1y0qH3Gq6" /></figure><h3>Conclusion</h3><p>Searching through documents to find answers is tedious and time-consuming. However new AI techniques are emerging to automate this process. As outlined in this article, Large Language Models like Google’s PaLM can be leveraged to rapidly search documents and generate natural language answers to queries.</p><p>The key is encoding documents into vector embeddings that capture semantic meaning. These are indexed in a vector database to enable fast semantic search. When a question is asked, the most relevant passage is retrieved to provide context. PaLM takes this passage and generates a conversational response, eliminating the need to read through documents.</p><p>You can access the complete source code at <a href="https://github.com/Syed007Hassan/Document-Querying-With-VectorDB">https://github.com/Syed007Hassan/Document-Querying-With-VectorDB</a>.</p><p>If you enjoyed this article, please click on the clap button 👏 and share to help others find it!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9ba73a935e5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unlocking the Power of LangChain’s SQL Agent: A Deep Dive into Natural Language Database…]]></title>
            <link>https://medium.com/@syed007hassan/unlocking-the-power-of-langchains-sql-agent-a-deep-dive-into-natural-language-database-4b2b2dcd6d18?source=rss-72ea816b21a------2</link>
            <guid isPermaLink="false">https://medium.com/p/4b2b2dcd6d18</guid>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[langchain-agents]]></category>
            <category><![CDATA[langchain]]></category>
            <dc:creator><![CDATA[Syed Muhammed Hassan Ali]]></dc:creator>
            <pubDate>Sat, 19 Aug 2023 09:49:24 GMT</pubDate>
            <atom:updated>2023-08-19T09:49:24.206Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Unlocking the Power of LangChain’s SQL Agent: A Deep Dive into Natural Language Database Interactions</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EXyezY1bJ0AvcOPrAuzE0Q.png" /></figure><h3>Introduction</h3><p>Natural language querying allows users to interact with databases more intuitively and efficiently. By leveraging the power of LangChain, SQL Agents, and OpenAI’s Large Language Models (LLMs) like ChatGPT, we can create applications that enable users to query databases using natural language. In this blog post, we’ll discuss the key features of these technologies and provide a step-by-step guide on how to implement them for natural language database querying.</p><h3>LangChain</h3><p>LangChain is a framework designed to build applications powered by language models. It provides a standard interface for chains, integrates with various tools, and offers end-to-end chains for typical applications. The two main features of LangChain are data-awareness and agentic behaviour.</p><p>Data awareness enables the language model to connect to other sources of data, while agentic behaviour allows the model to interact with its environment. Using agents, LangChain can dynamically decide which tools to call based on user input. This makes agents extremely powerful when used correctly.</p><figure><img alt="The framework is organized into seven modules. Each module allows you to manage a different aspect of the interaction with the LLM." src="https://cdn-images-1.medium.com/max/700/0*L4PVsnQ4EtJ1d8Me.jpeg" /><figcaption><a href="https://www.kdnuggets.com/2023/04/langchain-101-build-gptpowered-applications.html">LangChain 101: Build Your Own GPT-Powered Applications</a></figcaption></figure><h3>Tools, Agents, and Toolkits in LangChain</h3><p>A. Tools are functions that perform specific duties, such as Google Search, database lookups, or Python REPL. They take a string as input and return a string as output.</p><p>B. Agents are responsible for determining which actions to take and in what order. They can use tools, observe the output, or return to the user.</p><p>C. Toolkits are collections of tools that can be utilized by agents. LangChain supports various toolkits to help developers create powerful applications.</p><h3>Agents</h3><p>The SQL Agent provided by LangChain is a tool that allows you to interact with SQL databases using natural language. It is designed to be more flexible and more powerful than the standard SQLDatabaseChain, and it can be used to answer more general questions about a database, as well as recover from errors.</p><p>Here are some of the advantages of using the SQL Agent:</p><ul><li>It can answer questions based on the databases’ schema as well as on the databases’ content (like describing a specific table)</li><li>It can recover from errors by running a generated query, catching the traceback, and regenerating it correctly.</li><li>It is compatible with any SQL dialect supported by SQLAlchemy and TypeORM (e.g., MySQL, PostgreSQL, Oracle SQL, Databricks, SQLite).</li></ul><h3>Prerequisites</h3><p>The following guide, we are using on our machine:</p><p>· SQLite database</p><p>· NodeJS</p><p>You’ll also need an OpenAI API Key. You can get one at <a href="https://openai.com/">https://openai.com</a>.</p><h3>Getting started</h3><p>To get started, we’ll create a NodeJS/ExpressJS project:</p><pre>npm init --y</pre><p>Update the new <strong>package.json</strong> file to use ES Modules by adding this line of configuration:</p><pre>&quot;type&quot;: &quot;module&quot;,</pre><p>Next, we’ll install the required npm packages:</p><pre>npm install cors dotenv express langchain typeorm</pre><p>Create a .env file in the root directory of the project</p><pre>OPENAI_API_KEY=&lt;YOUR-OWN-KEY&gt;</pre><p>Create an index.js file with the following code</p><pre>import express from &quot;express&quot;;<br>import cors from &quot;cors&quot;;<br>import { OpenAI } from &quot;langchain/llms/openai&quot;;<br>import { SqlDatabase } from &quot;langchain/sql_db&quot;;<br>import { createSqlAgent, SqlToolkit } from &quot;langchain/agents/toolkits/sql&quot;;<br>import { DataSource } from &quot;typeorm&quot;;<br>import { configDotenv } from &quot;dotenv&quot;;<br><br>// Load configuration<br>try {<br>  configDotenv();<br>} catch (e) {<br>  console.error(&quot;Error loading configuration:&quot;, e);<br>  process.exit(1);<br>}<br><br>// Create server<br>const app = express();<br>app.use(cors());<br><br>// Create database connection<br>const datasource = new DataSource({<br>  type: &quot;sqlite&quot;,<br>  database: &quot;./data/northwind.db&quot;,<br>});<br>const db = await SqlDatabase.fromDataSourceParams({<br>  appDataSource: datasource,<br>});<br>const toolkit = new SqlToolkit(db);<br><br>// Create OpenAI model<br>const model = new OpenAI({<br>  temperature: 0,<br>});<br>const executor = createSqlAgent(model, toolkit);<br><br>// Route handler<br>app.get(&quot;/api/query&quot;, async (req, res) =&gt; {<br>  const prompt = req.query.prompt;<br><br>  console.log(&quot;prompt: &quot; + prompt);<br><br>  let response = {<br>    prompt: prompt,<br>    sqlQuery: &quot;&quot;,<br>    result: [],<br>    error: &quot;&quot;,<br>  };<br><br>  try {<br>    const result = await executor.call({ input: prompt });<br><br>    result.intermediateSteps.forEach((step) =&gt; {<br>      if (step.action.tool === &quot;query-sql&quot;) {<br>        response.prompt = prompt;<br>        response.sqlQuery = step.action.toolInput;<br>        response.result = JSON.parse(step.observation);<br>      }<br>    });<br><br>    console.log(<br>      `Intermediate steps ${JSON.stringify(result.intermediateSteps, null, 2)}`<br>    );<br><br>    res.status(200).json(response);<br>  } catch (e) {<br>    console.log(e + &quot; &quot; + &quot;my error message&quot;);<br>    response.error = &quot;Server error. Try again with a different prompt.&quot;;<br><br>    res.status(500).json(response);<br>  }<br><br>  await datasource.destroy();<br>});<br><br>// Start server<br>app.listen(5000, () =&gt; {<br>  console.log(&quot;Server started on port 5000&quot;);<br>});</pre><blockquote>To run the server, use either <strong>nodemon</strong> or <strong>npm run start</strong></blockquote><p><strong>The schema of the example database is:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/974/0*EN7BVYInSWB_fWHD.png" /></figure><p><strong>For testing API:</strong></p><pre>import axios from &quot;axios&quot;;<br><br>const prompt = &quot;Get all the employees who were hired after 2005?&quot;;<br><br>try {<br>  const response = await axios.get(<br>    `http://localhost:5000/api/query?prompt=${encodeURIComponent(prompt)}`<br>  );<br><br>  const data = response.data;<br>  console.log(data);<br><br>} catch (error) {<br>  console.log(error);<br>}</pre><blockquote>Use <strong>node test.js</strong> to run the file</blockquote><p><strong>Output:</strong></p><pre><br>[<br>  {<br>    &quot;action&quot;: {<br>      &quot;tool&quot;: &quot;list-tables-sql&quot;,<br>      &quot;toolInput&quot;: &quot;&quot;,<br>      &quot;log&quot;: &quot;Action: list-tables-sql\nAction Input: \&quot;\&quot;&quot;<br>    },<br>    &quot;observation&quot;: &quot;Categories, CustomerCustomerDemo, CustomerDemographics, Customers, EmployeeTerritories, Order Details, Orders, Products, Regions, Shippers, Suppliers, Territories, Employees&quot;<br>  },<br>  {<br>    &quot;action&quot;: {<br>      &quot;tool&quot;: &quot;info-sql&quot;,<br>      &quot;toolInput&quot;: &quot;Employees&quot;,<br>      &quot;log&quot;: &quot; I should look at the Employees table to see what columns I can query.\nAction: info-sql\nAction Input: \&quot;Employees\&quot;&quot;<br>    },<br>    &quot;observation&quot;: &quot;CREATE TABLE Employees (\nEmployeeID INTEGER , LastName TEXT , FirstName TEXT , Title TEXT , TitleOfCourtesy TEXT , BirthDate DATE , HireDate DATE , Address TEXT , City TEXT , Region TEXT , PostalCode TEXT , Country TEXT , HomePhone TEXT , Extension TEXT , ReportsTo INTEGER , PhotoPath TEXT ) \nSELECT * FROM \&quot;Employees\&quot; LIMIT 3;\n EmployeeID LastName FirstName Title TitleOfCourtesy BirthDate HireDate Address City Region PostalCode Country HomePhone Extension ReportsTo PhotoPath\n 1 Davolio Nancy Sales Representative Ms. 1968-12-08 2012-05-01 507 - 20th Ave. E.Apt. 2A Seattle North America 98122 USA (206) 555-9857 5467 2 http://accweb/emmployees/davolio.bmp\n 2 Fuller Andrew Vice President, Sales Dr. 1972-02-19 2012-08-14 908 W. Capital Way Tacoma North America 98401 USA (206) 555-9482 3457 null http://accweb/emmployees/fuller.bmp\n 3 Leverling Janet Sales Representative Ms. 1983-08-30 2012-04-01 722 Moss Bay Blvd. Kirkland North America 98033 USA (206) 555-3412 3355 2 http://accweb/emmployees/leverling.bmp\n&quot;<br>  },<br>  {<br>    &quot;action&quot;: {<br>      &quot;tool&quot;: &quot;query-sql&quot;,<br>      &quot;toolInput&quot;: &quot;SELECT * FROM Employees WHERE HireDate &gt; &#39;2005-01-01&#39; LIMIT 10;&quot;,<br>      &quot;log&quot;: &quot; I should query the Employees table for employees hired after 2005.\nAction: query-sql\nAction Input: SELECT * FROM Employees WHERE HireDate &gt; &#39;2005-01-01&#39; LIMIT 10;&quot;<br>    },<br>    &quot;observation&quot;: &quot;[{\&quot;EmployeeID\&quot;:1,\&quot;LastName\&quot;:\&quot;Davolio\&quot;,\&quot;FirstName\&quot;:\&quot;Nancy\&quot;,\&quot;Title\&quot;:\&quot;Sales Representative\&quot;,\&quot;TitleOfCourtesy\&quot;:\&quot;Ms.\&quot;,\&quot;BirthDate\&quot;:\&quot;1968-12-08\&quot;,\&quot;HireDate\&quot;:\&quot;2012-05-01\&quot;,\&quot;Address\&quot;:\&quot;507 - 20th Ave. E.Apt. 2A\&quot;,\&quot;City\&quot;:\&quot;Seattle\&quot;,\&quot;Region\&quot;:\&quot;North America\&quot;,\&quot;PostalCode\&quot;:\&quot;98122\&quot;,\&quot;Country\&quot;:\&quot;USA\&quot;,\&quot;HomePhone\&quot;:\&quot;(206) 555-9857\&quot;,\&quot;Extension\&quot;:\&quot;5467\&quot;,\&quot;ReportsTo\&quot;:2,\&quot;PhotoPath\&quot;:\&quot;http://accweb/emmployees/davolio.bmp\&quot;},{\&quot;EmployeeID\&quot;:2,\&quot;LastName\&quot;:\&quot;Fuller\&quot;,\&quot;FirstName\&quot;:\&quot;Andrew\&quot;,\&quot;Title\&quot;:\&quot;Vice President, Sales\&quot;,\&quot;TitleOfCourtesy\&quot;:\&quot;Dr.\&quot;,\&quot;BirthDate\&quot;:\&quot;1972-02-19\&quot;,\&quot;HireDate\&quot;:\&quot;2012-08-14\&quot;,\&quot;Address\&quot;:\&quot;908 W. Capital Way\&quot;,\&quot;City\&quot;:\&quot;Tacoma\&quot;,\&quot;Region\&quot;:\&quot;North America\&quot;,\&quot;PostalCode\&quot;:\&quot;98401\&quot;,\&quot;Country\&quot;:\&quot;USA\&quot;,\&quot;HomePhone\&quot;:\&quot;(206) 555-9482\&quot;,\&quot;Extension\&quot;:\&quot;3457\&quot;,\&quot;ReportsTo\&quot;:null,\&quot;PhotoPath\&quot;:\&quot;http://accweb/emmployees/fuller.bmp\&quot;},{\&quot;EmployeeID\&quot;:3,\&quot;LastName\&quot;:\&quot;Leverling\&quot;,\&quot;FirstName\&quot;:\&quot;Janet\&quot;,\&quot;Title\&quot;:\&quot;Sales Representative\&quot;,\&quot;TitleOfCourtesy\&quot;:\&quot;Ms.\&quot;,\&quot;BirthDate\&quot;:\&quot;1983-08-30\&quot;,\&quot;HireDate\&quot;:\&quot;2012-04-01\&quot;,\&quot;Address\&quot;:\&quot;722 Moss Bay Blvd.\&quot;,\&quot;City\&quot;:\&quot;Kirkland\&quot;,\&quot;Region\&quot;:\&quot;North America\&quot;,\&quot;PostalCode\&quot;:\&quot;98033\&quot;,\&quot;Country\&quot;:\&quot;USA\&quot;,\&quot;HomePhone\&quot;:\&quot;(206) 555-3412\&quot;,\&quot;Extension\&quot;:\&quot;3355\&quot;,\&quot;ReportsTo\&quot;:2,\&quot;PhotoPath\&quot;:\&quot;http://accweb/emmployees/leverling.bmp\&quot;},{\&quot;EmployeeID\&quot;:4,\&quot;LastName\&quot;:\&quot;Peacock\&quot;,\&quot;FirstName\&quot;:\&quot;Margaret\&quot;,\&quot;Title\&quot;:\&quot;Sales Manager\&quot;,\&quot;TitleOfCourtesy\&quot;:\&quot;Ms.\&quot;,\&quot;BirthDate\&quot;:\&quot;1963-09-19\&quot;,\&quot;HireDate\&quot;:\&quot;2005-05-03\&quot;,\&quot;Address\&quot;:\&quot;4110 Old Redmond Rd.\&quot;,\&quot;City\&quot;:\&quot;Redmond\&quot;,\&quot;Region\&quot;:\&quot;North America\&quot;,\&quot;PostalCode\&quot;:\&quot;98052\&quot;,\&quot;Country\&quot;:\&quot;USA\&quot;,\&quot;HomePhone\&quot;:\&quot;(206) 555-8122\&quot;,\&quot;Extension\&quot;:\&quot;5176\&quot;,\&quot;ReportsTo\&quot;:2,\&quot;PhotoPath\&quot;:\&quot;http://accweb/emmployees/peacock.bmp\&quot;},{\&quot;EmployeeID\&quot;:5,\&quot;LastName\&quot;:\&quot;Buchanan\&quot;,\&quot;FirstName\&quot;:\&quot;Steven\&quot;,\&quot;Title\&quot;:\&quot;Sales Manager\&quot;,\&quot;TitleOfCourtesy\&quot;:\&quot;Mr.\&quot;,\&quot;BirthDate\&quot;:\&quot;1975-03-04\&quot;,\&quot;HireDate\&quot;:\&quot;2013-10-17\&quot;,\&quot;Address\&quot;:\&quot;14 Garrett Hill\&quot;,\&quot;City\&quot;:\&quot;London\&quot;,\&quot;Region\&quot;:\&quot;British Isles\&quot;,\&quot;PostalCode\&quot;:\&quot;SW1 8JR\&quot;,\&quot;Country\&quot;:\&quot;UK\&quot;,\&quot;HomePhone\&quot;:\&quot;(71) 555-4848\&quot;,\&quot;Extension\&quot;:\&quot;3453\&quot;,\&quot;ReportsTo\&quot;:2,\&quot;PhotoPath\&quot;:\&quot;http://accweb/emmployees/buchanan.bmp\&quot;},{\&quot;EmployeeID\&quot;:6,\&quot;LastName\&quot;:\&quot;Suyama\&quot;,\&quot;FirstName\&quot;:\&quot;Michael\&quot;,\&quot;Title\&quot;:\&quot;Sales Representative\&quot;,\&quot;TitleOfCourtesy\&quot;:\&quot;Mr.\&quot;,\&quot;BirthDate\&quot;:\&quot;1983-07-02\&quot;,\&quot;HireDate\&quot;:\&quot;2013-10-17\&quot;,\&quot;Address\&quot;:\&quot;Coventry House\\nMiner Rd.\&quot;,\&quot;City\&quot;:\&quot;London\&quot;,\&quot;Region\&quot;:\&quot;British Isles\&quot;,\&quot;PostalCode\&quot;:\&quot;EC2 7JR\&quot;,\&quot;Country\&quot;:\&quot;UK\&quot;,\&quot;HomePhone\&quot;:\&quot;(71) 555-7773\&quot;,\&quot;Extension\&quot;:\&quot;428\&quot;,\&quot;ReportsTo\&quot;:5,\&quot;PhotoPath\&quot;:\&quot;http://accweb/emmployees/suyama.bmp\&quot;},{\&quot;EmployeeID\&quot;:7,\&quot;LastName\&quot;:\&quot;King\&quot;,\&quot;FirstName\&quot;:\&quot;Robert\&quot;,\&quot;Title\&quot;:\&quot;Sales Representative\&quot;,\&quot;TitleOfCourtesy\&quot;:\&quot;Mr.\&quot;,\&quot;BirthDate\&quot;:\&quot;1980-05-29\&quot;,\&quot;HireDate\&quot;:\&quot;2014-01-02\&quot;,\&quot;Address\&quot;:\&quot;Edgeham Hollow\\nWinchester Way\&quot;,\&quot;City\&quot;:\&quot;London\&quot;,\&quot;Region\&quot;:\&quot;British Isles\&quot;,\&quot;PostalCode\&quot;:\&quot;RG1 9SP\&quot;,\&quot;Country\&quot;:\&quot;UK\&quot;,\&quot;HomePhone\&quot;:\&quot;(71) 555-5598\&quot;,\&quot;Extension\&quot;:\&quot;465\&quot;,\&quot;ReportsTo\&quot;:5,\&quot;PhotoPath\&quot;:\&quot;http://accweb/emmployees/king.bmp\&quot;},{\&quot;EmployeeID\&quot;:8,\&quot;LastName\&quot;:\&quot;Callahan\&quot;,\&quot;FirstName\&quot;:\&quot;Laura\&quot;,\&quot;Title\&quot;:\&quot;Inside Sales Coordinator\&quot;,\&quot;TitleOfCourtesy\&quot;:\&quot;Ms.\&quot;,\&quot;BirthDate\&quot;:\&quot;1978-01-09\&quot;,\&quot;HireDate\&quot;:\&quot;2014-03-05\&quot;,\&quot;Address\&quot;:\&quot;4726 - 11th Ave. N.E.\&quot;,\&quot;City\&quot;:\&quot;Seattle\&quot;,\&quot;Region\&quot;:\&quot;North America\&quot;,\&quot;PostalCode\&quot;:\&quot;98105\&quot;,\&quot;Country\&quot;:\&quot;USA\&quot;,\&quot;HomePhone\&quot;:\&quot;(206) 555-1189\&quot;,\&quot;Extension\&quot;:\&quot;2344\&quot;,\&quot;ReportsTo\&quot;:2,\&quot;PhotoPath\&quot;:\&quot;http://accweb/emmployees/callahan.bmp\&quot;},{\&quot;EmployeeID\&quot;:9,\&quot;LastName\&quot;:\&quot;Dodsworth\&quot;,\&quot;FirstName\&quot;:\&quot;Anne\&quot;,\&quot;Title\&quot;:\&quot;Sales Representative\&quot;,\&quot;TitleOfCourtesy\&quot;:\&quot;Ms.\&quot;,\&quot;BirthDate\&quot;:\&quot;1986-01-27\&quot;,\&quot;HireDate\&quot;:\&quot;2014-11-15\&quot;,\&quot;Address\&quot;:\&quot;7 Houndstooth Rd.\&quot;,\&quot;City\&quot;:\&quot;London\&quot;,\&quot;Region\&quot;:\&quot;British Isles\&quot;,\&quot;PostalCode\&quot;:\&quot;WG2 7LT\&quot;,\&quot;Country\&quot;:\&quot;UK\&quot;,\&quot;HomePhone\&quot;:\&quot;(71) 555-4444\&quot;,\&quot;Extension\&quot;:\&quot;452\&quot;,\&quot;ReportsTo\&quot;:5,\&quot;PhotoPath\&quot;:\&quot;http://accweb/emmployees/dodsworth.bmp\&quot;}]&quot;<br>  }<br>]</pre><p><strong>Data in the form of a table from the above observations</strong></p><pre>| EmployeeID | LastName  | FirstName | Title                 | TitleOfCourtesy | BirthDate   | HireDate   | Address                     | City       | Region        | PostalCode | Country | HomePhone       | Extension | ReportsTo | PhotoPath                            |<br>|------------|-----------|-----------|-----------------------|-----------------|-------------|------------|-----------------------------|------------|---------------|------------|---------|-----------------|-----------|-----------|-------------------------------------|<br>| 1          | Davolio   | Nancy     | Sales Representative | Ms.             | 1968-12-08  | 2012-05-01 | 507 - 20th Ave. E.Apt. 2A   | Seattle    | North America | 98122      | USA     | (206) 555-9857 | 5467      | 2         | http://accweb/emmployees/davolio.bmp |<br>| 2          | Fuller    | Andrew    | Vice President, Sales| Dr.             | 1972-02-19  | 2012-08-14 | 908 W. Capital Way          | Tacoma     | North America | 98401      | USA     | (206) 555-9482 | 3457      | null      | http://accweb/emmployees/fuller.bmp  |<br>| 3          | Leverling | Janet     | Sales Representative | Ms.             | 1983-08-30  | 2012-04-01 | 722 Moss Bay Blvd.          | Kirkland   | North America | 98033      | USA     | (206) 555-3412 | 3355      | 2         | http://accweb/emmployees/leverling.bmp |<br>| 4          | Peacock   | Margaret  | Sales Manager        | Ms.             | 1963-09-19  | 2005-05-03 | 4110 Old Redmond Rd.        | Redmond    | North America | 98052      | USA     | (206) 555-8122 | 5176      | 2         | http://accweb/emmployees/peacock.bmp |<br>| 5          | Buchanan  | Steven    | Sales Manager        | Mr.             | 1975-03-04  | 2013-10-17 | 14 Garrett Hill             | London     | British Isles | SW1 8JR    | UK      | (71) 555-4848  | 3453      | 2         | http://accweb/emmployees/buchanan.bmp |<br>| 6          | Suyama    | Michael   | Sales Representative | Mr.             | 1983-07-02  | 2013-10-17 | Coventry House\nMiner Rd.   | London     | British Isles | EC2 7JR    | UK      | (71) 555-7773  | 428       | 5         | http://accweb/emmployees/suyama.bmp  |<br>| 7          | King      | Robert    | Sales Representative | Mr.             | 1980-05-29  | 2014-01-02 | Edgeham Hollow\nWinchester Way| London     | British Isles | RG1 9SP   | UK      | (71) 555-5598  | 465       | 5         | http://accweb/emmployees/king.bmp   |<br>| 8          | Callahan  | Laura     | Inside Sales Coordinator | Ms.         | 1978-01-09  | 2014-03-05 | 4726 - 11th Ave. N.E.       | Seattle    | North America | 98105      | USA     | (206) 555-1189 | 2344      | 2         | http://accweb/emmployees/callahan.bmp |<br>| 9          | Dodsworth | Anne      | Sales Representative | Ms.             | 1986-01-27  | 2014-11-15 | 7 Houndstooth Rd.           | London     | British Isles | WG2 7LT   | UK      | (71) 555-4444  | 452       | 5         | http://accweb/emmployees/dodsworth.bmp |</pre><p>The SQL Agent works by first generating a SQL query from the user’s natural language question. It then executes the query and returns the results to the user. If the query fails, the SQL Agent will attempt to recover by interpreting the error in a subsequent LLM call and rectifying the issue.</p><p>You can access the GUI-based application and view the complete source code by visiting<a href="https://github.com/Syed007Hassan/NextJs-Langchain-Agents-SQL/"> https://github.com/Syed007Hassan/NextJs-Langchain-Agents-SQL/</a>.</p><p>If you enjoyed this article, please click on the clap button 👏 and share to help others find it!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4b2b2dcd6d18" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>