<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nelson LIN on Medium]]></title>
        <description><![CDATA[Stories by Nelson LIN on Medium]]></description>
        <link>https://medium.com/@nelsonlin0321?source=rss-24280f9abccc------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*_JO6nrU4N_exLdDAu7nLTQ.jpeg</url>
            <title>Stories by Nelson LIN on Medium</title>
            <link>https://medium.com/@nelsonlin0321?source=rss-24280f9abccc------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 11:30:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nelsonlin0321/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building Content Sharing Platform, like Medium but dedicated for Agent Engineering.]]></title>
            <link>https://medium.com/@nelsonlin0321/building-content-sharing-platform-like-medium-but-dedicated-for-agent-engineering-75a51d57eb5a?source=rss-24280f9abccc------2</link>
            <guid isPermaLink="false">https://medium.com/p/75a51d57eb5a</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[agent-engineering]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[headless-cms]]></category>
            <dc:creator><![CDATA[Nelson LIN]]></dc:creator>
            <pubDate>Sun, 19 Apr 2026 08:35:35 GMT</pubDate>
            <atom:updated>2026-04-19T08:35:35.129Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’re fascinated by the world of AI agents and modern web development, I have something special to share with you today. I’m incredibly excited to introduce a project I’ve been building for the community: <a href="https://www.agent-engineering.dev/">agent-engineering.dev</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*auoQbvZfNTq2-aurNZBrWA.jpeg" /><figcaption>Agent-Engineering.dev</figcaption></figure><p>It is a specialized knowledge-sharing platform designed specifically for professionals, researchers, and enthusiasts in the rapidly evolving field of agent engineering. Unlike general publishing platforms like Medium, agent-engineering.dev is completely free. There are no paywalls or membership limits here. This space is entirely dedicated to open access for cutting-edge ideas, tutorials, case studies, and thought leadership around agent-based systems and AI agents.</p><p><strong>Moving forward, any article I write will be posted directly on agent-engineering.dev.</strong></p><p>Today, I want to pull back the curtain and share a technical perspective on how we built it.</p><h3><strong>The Tech Stack</strong></h3><p>At its core, `agent-engineering.dev` is a modern web application built for speed, SEO, and an excellent user experience.</p><ul><li>Framework: Next.js 16 (App Router)</li><li>Database &amp; ORM: MongoDB with Prisma ORM</li><li>Authentication: Clerk</li><li>Styling: Tailwind CSS (v4) &amp; Shadcn UI (Radix UI)</li><li>Content Editor: <a href="https://www.blocknotejs.org/">BlockNote</a> supporting rich text editing</li><li>State Management &amp; Fetching:React Query &amp; Zustand</li><li>Caching/Rate Limiting: Upstash Redis</li></ul><h3><strong>Rich Text Authoring with BlockNote</strong></h3><p>A knowledge-sharing platform is only as good as its authoring experience. To provide a modern, Notion-style rich text editor, we integrated <a href="https://www.blocknotejs.org/">BlockNote</a>, an open-source, block-based editor for React that ships with great defaults and stays fully customizable.</p><p>After the core rich text experience is in place, we layer in AI assistance. By using `@blocknote/xl-ai` together with the Vercel AI SDK, authors can co-author directly inside the editor , brainstorming ideas, refining tone, and summarizing sections.</p><h3>Join the Community</h3><p>Building <a href="https://www.agent-engineering.dev/">agent-engineering.dev</a> has been an enjoyable journey of combining modern web technologies (Next.js, Prisma, Clerk, BlockNote) to serve a purpose that agent engineering worths sharing.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75a51d57eb5a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building NewsClocker: A Deep Dive into Our Event-Driven Custom Scheduler Infrastructure — Part 1]]></title>
            <link>https://medium.com/@nelsonlin0321/building-newsclocker-a-deep-dive-into-our-event-driven-custom-scheduler-infrastructure-part-1-c6dadde5a6bd?source=rss-24280f9abccc------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6dadde5a6bd</guid>
            <category><![CDATA[scheduling]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[architecture]]></category>
            <dc:creator><![CDATA[Nelson LIN]]></dc:creator>
            <pubDate>Sat, 11 Jan 2025 17:12:25 GMT</pubDate>
            <atom:updated>2025-01-13T14:07:38.312Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building NewsClocker: A Deep Dive into Our Event-Driven Custom Scheduler Infrastructure — Part 1</h3><p><a href="https://newsclocker.com/">NewsClocker</a> is an AI-driven news intelligence and subscription platform designed to empower professionals with personalized news delivery and actionable insights. <strong>The system to deliver news insights with the preferred delivery schedules reliably and at scale is the core Infrastructure of newsclocker.</strong></p><p>https://newsclocker.com</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5VxYLz18IFMe7XmwbZ8m2w.png" /><figcaption>NewsClocker</figcaption></figure><p>This post is to share the core infrastructure behind NewsClocker we built using Terrform, our AI-powered news intelligence platform. If you’re intereseted in how we architected the event-driven system to deliver personalized news insights reliably and at scale as schedule, this technical articles will help you build up the modern job custom schedule system.</p><p>The source code: <a href="https://github.com/Nelsonlin0321/newsclocker-scheduler">https://github.com/Nelsonlin0321/newsclocker-scheduler</a></p><p>The source codes also includes terrform infrastucture as code to build up the whole infrascture in a few command lines in development and production environments.</p><h3>The Core Infrastructure</h3><p>Let me walk you through our main architecture diagram:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OIDg08ELeeuwUtlMbsfOZA.png" /><figcaption>The architecture of news scheduler</figcaption></figure><p>Our infrastructure consists of five key components that work together to process and deliver personalized news:</p><ol><li><strong>EventBridge</strong></li><li><strong>Lambda-based Database Polling</strong></li><li><strong>SQS Job Queue</strong></li><li><strong>LLM Generation Workflow</strong></li><li><strong>API to use SES to send Email Notification</strong></li></ol><h3>How It All Works Together:</h3><h3>1. EventBridge:</h3><p>We can use EventBridge to trigger Lambda-based database polling every 3 minutes, ensuring timely execution of user-defined scheduling settings:</p><p>The Terraform code you provided sets up <strong>an automated process to trigger a Lambda function for database polling, every 3 minutes</strong> using AWS EventBridge.</p><pre>resource &quot;aws_cloudwatch_event_rule&quot; &quot;db_polling_trigger_rule&quot; {<br>    name = &quot;${local.eventbridge_name}&quot;<br>    description = &quot;trigger newsclocker polling lambda&quot;<br>    schedule_expression = &quot;rate(3 minutes)&quot;<br>}<br><br>resource &quot;aws_cloudwatch_event_target&quot; &quot;db_polling_trigger_rule_lambda_target&quot; {<br>  arn = aws_lambda_function.db_polling_lambda.arn<br>  rule = aws_cloudwatch_event_rule.db_polling_trigger_rule.name<br>}<br><br>resource &quot;aws_lambda_permission&quot; &quot;allow_cloudwatch_to_trigger_db_polling_lambda&quot; {<br>  statement_id = &quot;AllowExecutionFromCloudWatch&quot;<br>  action = &quot;lambda:InvokeFunction&quot;<br>  function_name = aws_lambda_function.db_polling_lambda.function_name<br>  principal = &quot;events.amazonaws.com&quot;<br>  source_arn = aws_cloudwatch_event_rule.db_polling_trigger_rule.arn<br>}</pre><ul><li><strong>aws_cloudwatch_event_rule:</strong> This defines a rule named db_polling_trigger_rule within EventBridge. The rule is set to trigger every 3 minutes based on the schedule_expression.</li><li><strong>aws_cloudwatch_event_target:</strong> This connects the previously defined rule (db_polling_trigger_rule) to the Lambda function (db_polling_lambda).</li><li><strong>aws_lambda_permission:</strong> The block grants EventBridge the permission to invoke your Lambda function.</li></ul><h3>2. <strong>Lambda-based Database Polling</strong>:</h3><p>It’s a design pattern leveraging the serverless capabilities of AWS Lambda to periodically query a database for active subscriptions to be executed within a specific time window. Typically, <strong>the time window should be larger than the periodical frequency</strong>. For example, every 3 minutes we query the database, the time window for execution should be larger than 3 minutes to avoid missing any subscriptions due to execution timing and everybridge latency. <strong>The timeline is demostrated below, every 3 minutes, the lambda will be triggered to query the database to check what subscription schedule should be executed in 5 minutes and update the their next run time.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aAICQUkMTP5kQI2xSDkQcg.png" /><figcaption>The timeline of eventBrige frequnce and db polling time window.</figcaption></figure><p>For each eligible subscription, it sends the subscription ID to an SQS queue and updates the subscription’s next execution time in the database. This ensures timely and reliable processing of subscriptions at their defined frequencies.</p><pre>import os<br>import traceback<br>from loguru import logger<br>from datetime import datetime, timedelta, timezone<br>from tqdm import tqdm<br>from src.aws_utils import send_message_to_queue<br>from src.dependencies import get_db<br><br>INTERVAL_MINUTES = int(os.getenv(&quot;INTERVAL&quot;, &quot;5&quot;))<br><br>FREQUENCY_TO_DELTA_DICT = {<br>    &quot;every_12_hour&quot;: timedelta(hours=12),<br>    &quot;every_day&quot;: timedelta(days=1),<br>    &quot;every_week&quot;: timedelta(days=7)<br>}<br><br><br>def main():<br>    left_datetime = datetime.now(<br>        timezone.utc)<br><br>    right_datetime = left_datetime + timedelta(seconds=INTERVAL_MINUTES*60)<br><br>    logger.info(&quot;Starting Polling.&quot;)<br>    db = get_db()<br>    #  All subscriptions that&#39;s should be delivered within 5 minutes<br>    subscriptions = list(db[&#39;NewsSubscription&#39;].find({<br>        &#39;$and&#39;: [{<br>            &#39;active&#39;: {&#39;$eq&#39;: True},<br>            &#39;nextRunTime&#39;: {<br>                &#39;$gte&#39;: left_datetime,<br>                &#39;$lte&#39;: right_datetime<br>            }<br>        }]<br>    }))<br><br>    for sub in tqdm(subscriptions):<br>        try:<br>            subscriptionId = sub[&quot;_id&quot;]<br>            <br>            #  Send message to SQS message queue<br><br>            send_message_to_queue(subscriptionId)<br>            next_run_time = sub[&#39;nextRunTime&#39;]<br>            frequency = sub[&#39;frequency&#39;]<br>            updated_next_run_time = next_run_time + \<br>                FREQUENCY_TO_DELTA_DICT[frequency]<br>            <br>            # Update the subscription with new next run time<br>            db[&#39;NewsSubscription&#39;].update_one(<br>                filter={&quot;_id&quot;: subscriptionId},<br>                update={&quot;$set&quot;: {&quot;nextRunTime&quot;: updated_next_run_time}}<br>            )<br>        except Exception as e:<br>            error_details = traceback.format_exc()<br>            error_message = f&quot;&quot;&quot;Subscription: {<br>                subscriptionId} unable to be triggered with error: {e} {error_details}&quot;&quot;&quot;<br>            logger.error(error_message)<br><br>    logger.info(&quot;Polling Completed.&quot;)<br><br><br>if __name__ == &quot;__main__&quot;:<br>    main()</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6dadde5a6bd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Scalable Data Processing: Implementing AWS S3 Event-Driven Ingestion Workflows]]></title>
            <link>https://medium.com/@nelsonlin0321/scalable-data-processing-implementing-aws-s3-event-driven-ingestion-workflows-4ec5d7995ade?source=rss-24280f9abccc------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ec5d7995ade</guid>
            <category><![CDATA[s3]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[architecture]]></category>
            <dc:creator><![CDATA[Nelson LIN]]></dc:creator>
            <pubDate>Fri, 06 Sep 2024 10:57:37 GMT</pubDate>
            <atom:updated>2024-09-06T10:57:53.741Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction:</h3><p>In today’s data-driven world, efficient and automated data ingestion is crucial. AWS S3 event triggers offer a powerful solution, transforming how we handle data pipelines. This feature turns S3 from a simple storage service into a catalyst for event-driven architectures, automating workflows the instant new data arrives.</p><p>Codes for this tutorial: <a href="https://github.com/Nelsonlin0321/lambda-event-trigger-tutorial">https://github.com/Nelsonlin0321/lambda-event-trigger-tutorial</a></p><p>Throughout this tutorial, we’ll:</p><ol><li>Design an S3 event-triggered data ingestion architecture</li><li>Simple Lambda functions for data processing and ingestion</li><li>Write Terraform code to define and provision the infrastructure with configurations of S3 event notifications</li><li>Deploy the entire system using Terraform</li></ol><p>Whether you’re a data engineer, cloud architect, or tech enthusiast, you’ll learn how to leverage S3 event triggers to streamline your data ingestion processes. Let’s dive in and unlock the potential of real-time data processing.</p><h3>Architecture Overview</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/859/1*pt99G6niFqacYXzmEx8E7w.png" /><figcaption>AWS S3 Event-Driven Ingestion Workflows</figcaption></figure><ol><li><strong>Amazon S3</strong>:<br> This serves as the entry point for our data pipeline. JSON files are uploaded to an S3 bucket, triggering the ingestion process.</li><li><strong>AWS Lambda</strong>:<br> The core processing unit of our architecture. Lambda functions are triggered by S3 events when new files are uploaded. These functions run in a Docker container, allowing for custom runtime environments and dependencies.</li><li><strong>Amazon ECR (Elastic Container Registry)</strong>:<br> Hosts the Docker images used by our Lambda functions, enabling consistent and reproducible execution environments.</li><li><strong>AWS Secrets Manager</strong>:<br> Securely stores and manages sensitive information, such as database credentials, API keys, or other secrets required by the Lambda function.</li><li><strong>MongoDB</strong>:<br> The target database where processed data is stored. This could be hosted on AWS (e.g., DocumentDB) or an external MongoDB instance</li><li><strong>Amazon CloudWatch Logs</strong>:<br> Captures logs from the Lambda function, enabling monitoring and troubleshooting of the data ingestion process.</li></ol><p><strong>Data Flow:</strong></p><ol><li>A JSON file is uploaded to the S3 bucket.</li><li>This upload triggers an event notification.</li><li>The event invokes the Lambda function, which runs in a Docker container pulled from ECR.</li><li>The Lambda function processes the data, potentially accessing secrets from AWS Secrets Manager.</li><li>Processed data is then stored in MongoDB.</li><li>Throughout this process, Lambda logs its activities to CloudWatch Logs.</li></ol><p>This architecture provides a scalable, event-driven approach to data ingestion. By using S3 events and Lambda, we ensure that data is processed as soon as it’s available. The use of Docker containers in Lambda allows for complex processing requirements, while MongoDB provides a flexible storage solution for the ingested data.</p><h3>Lambda Development</h3><p>Here are the source code:<a href="https://github.com/Nelsonlin0321/lambda-event-trigger-tutorial/tree/main/lambda">https://github.com/Nelsonlin0321/lambda-event-trigger-tutorial/tree/main/lambda</a> to execute the lambda to ingest json to MongoDB. Overally, it contains below step.</p><ol><li>Reacts to new files in an S3 bucket.</li><li>Downloads the JSON file.</li><li>Safely fetches a MongoDB connection string from AWS Secrets Manager.</li><li>Connects to MongoDB and inserts the JSON data into a collection.</li><li>Returns a success message or handles errors.</li></ol><h3>AWS Infrastructure Deployment</h3><p>This are how the aws components are implemented, how S3 event notifications and Lambda functions are configured, and how the entire system is deployed using Terraform.</p><p><strong>AM Role and Policy:</strong></p><ul><li>An IAM role (aws_iam_role.lambda_role) is created for the Lambda function.</li><li>An IAM policy (aws_iam_role_policy.lambda_policy) is attached to this role, granting necessary permissions.</li></ul><pre>terraform {<br>  required_providers {<br>    aws = {<br>      source  = &quot;hashicorp/aws&quot;<br>      version = &quot;~&gt; 5.18&quot;<br>    }<br>  }<br><br>  required_version = &quot;&gt;= 1.2.0&quot;<br>}<br><br>resource &quot;aws_iam_role&quot; &quot;lambda_role&quot; {<br>  name = &quot;${var.lambda_function_name}-role&quot;<br><br>  assume_role_policy = &lt;&lt;EOF<br>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Action&quot;: &quot;sts:AssumeRole&quot;,<br>      &quot;Principal&quot;: {<br>        &quot;Service&quot;: &quot;lambda.amazonaws.com&quot;<br>      },<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Sid&quot;: &quot;&quot;<br>    }<br>  ]<br>}<br>EOF<br>}<br><br><br>resource &quot;aws_iam_role_policy&quot; &quot;lambda_policy&quot; {<br>  name = &quot;${var.lambda_function_name}-policy&quot;<br>  role = aws_iam_role.lambda_role.name<br>  policy = data.aws_iam_policy_document.s3_trigger_policy.json<br>}</pre><p><strong>Lambda Function:</strong></p><ul><li>The Lambda function (aws_lambda_function.s3_trigger_lambda) is defined using a Docker image.</li><li>It’s configured with specific settings like timeout, memory size, and architecture.</li></ul><pre>resource &quot;aws_lambda_function&quot; &quot;s3_trigger_lambda&quot; {<br><br>  function_name = var.lambda_function_name<br>  role          = aws_iam_role.lambda_role.arn<br>  image_uri     = &quot;${var.account_id}.dkr.ecr.${var.region}.amazonaws.com/${var.lambda_function_name}:latest&quot;<br>  package_type  = &quot;Image&quot;<br>  timeout       = 10<br>  memory_size   = 128<br>  architectures = [&quot;arm64&quot;]<br>  environment {<br>    variables = {<br>      ENV = &quot;${var.env}&quot;<br>    }<br>  }<br>}</pre><p><strong>S3 Bucket:</strong></p><ul><li>The S3 bucket is assumed to be pre-existing.</li></ul><p><strong>AWS Secrets Manager:</strong></p><ul><li>The Lambda function is given permission to access a specific secret in AWS Secrets Manager, likely containing MongoDB connection details.</li></ul><p><strong>Configuring S3 Event Notifications and Lambda Function:</strong></p><p>Lambda Permission:</p><ul><li>aws_lambda_permission.allow_bucket grants S3 the permission to invoke the Lambda function.</li></ul><p>S3 Bucket Notification:</p><ul><li>aws_s3_bucket_notification.prod_bucket_notification sets up the event trigger.</li><li>It configures the S3 bucket to trigger the Lambda function when a new object is created (PUT event).</li><li>The trigger is filtered to only act on .json files in a specific prefix matching the environment.</li></ul><p>This event-driven architecture using AWS S3, Lambda, and Terraform provides a scalable and efficient solution for real-time data ingestion. By automating the data processing pipeline, organizations can unlock valuable insights from their data quickly and reliably. For a deeper dive into the implementation, check out the complete code and examples in the linked GitHub repository: <a href="https://github.com/Nelsonlin0321/lambda-event-trigger-tutorial">https://github.com/Nelsonlin0321/lambda-event-trigger-tutorial</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ec5d7995ade" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building A Modern Chat PDF AI SaSS Application By Google Genimi API]]></title>
            <link>https://medium.com/@nelsonlin0321/building-a-modern-chat-pdf-ai-sass-application-by-google-genimi-api-9892fbe645b1?source=rss-24280f9abccc------2</link>
            <guid isPermaLink="false">https://medium.com/p/9892fbe645b1</guid>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[nextjs]]></category>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[full-stack]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Nelson LIN]]></dc:creator>
            <pubDate>Sat, 16 Mar 2024 11:20:08 GMT</pubDate>
            <atom:updated>2024-03-18T07:53:11.303Z</atom:updated>
            <content:encoded><![CDATA[<p>In this post, we’ll explore how to build a modern full-stack web application that allows users to upload PDF files, perform vector similarity searches on the contents, and engage in AI-powered chat conversations. The application is based on the below YouTube tutorial with my secondary development and optimizations.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fr895rFUbGtE%3Fstart%3D15158%26feature%3Doembed%26start%3D15158&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dr895rFUbGtE&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fr895rFUbGtE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/958bef44d76874639ddfe0b553784fb2/href">https://medium.com/media/958bef44d76874639ddfe0b553784fb2/href</a></iframe><p>We’ll leverage cutting-edge technologies like FastAPI, Jina, Vercel AI SDK, MongoDB Atlas Search, Prisma ORM, and Genimi to create a model and production-ready application with the best practice.</p><p>FrontEnd Github:<a href="https://github.com/Nelsonlin0321/chat-pdf-ai-assistant">https://github.com/Nelsonlin0321/chat-pdf-ai-assistant</a></p><p>BackeEnd API Github:<a href="https://github.com/Nelsonlin0321/chat-pdf-ai-assistant-fastapi">https://github.com/Nelsonlin0321/chat-pdf-ai-assistant-fastapi</a></p><h3><strong>Application Preview:</strong></h3><p>Before diving into the implementation details, let&#39;s see what it looks like:</p><p><a href="http://www.chat-pdf-ai.com/">www.chat-pdf-ai.com</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*8p7uaNi1Mt7uoCX-5lfL_A.gif" /></figure><h3>Highlights &amp; Optimizations:</h3><ul><li>MongoDB Atlas for storage, vector search, keyword search, and customized hybrid search.</li><li>Backend APIs for file uploading, and context retrieval deployed by CloudRun and AWS Lambda with CI/CD, infrastructure as code.</li><li>Google Genimi API for chat generation</li><li>Purely chat mode available to disable reading PDF</li><li>Jina AI for embedding and retrieving reranking</li><li>UI optimizations for model devices</li><li>Prisma ORM with Edge Accelerate</li></ul><h3>Full Stack Architecture:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XQL2XauXPGzTa3drr63yGQ.png" /></figure><h3>Backend:</h3><ul><li>The FastAPI server acts as the main backend API, handling complex logic.</li><li>It uses the Jina framework for embedding text data and reranking for retrieval data.</li><li>MongoDB act as the backend database and performs vector similarity and keyword search operations for ingested vectors and text.</li><li>The custom “Read &amp; Chunk” component to preprocess PDF files, splitting pages into sentences and them merging them into chunks for embedding and indexing.</li><li>The “api/hybrid_search” route handles hybrid search queries, combining traditional text search (BM25) with vector similarity search with Jina AI reranking.</li></ul><h3>Frontend:</h3><ul><li>The front end is built with the Next.js React framework, providing server-side rendering and static site generation capabilities.</li><li>It integrates with third-party services like Stripe for payment processing, Clerk for user authentication (sign-in, sign-up), and an AI SDK for chat functionality (api/ai/chat).</li><li>The frontend communicates with the backend API for CRUD operations, managing data related to the application’s core functionality.</li><li>Vercel is used for hosting and deploying the Next.js frontend application.</li></ul><h3>Data Flow:</h3><ul><li>PDF files are ingested through the backend API (api/ingest_file) and processed (read, chunked, embedded) before storing in the vector database.</li><li>User queries (questions) are embedded using the Jina API, and a vector similarity search is performed against the ingested vector database to retrieve relevant chunks or documents.</li><li>The retrieved chunks are re-ranked using the Rerank Jina API and other algorithms (BM25) to produce the final search results.</li><li>The front end interacts with the backend API to fetch and display these search results, as well as to handle other functionality like authentication, payments, and AI-powered chat.</li></ul><h3>Build an advanced RAG Pipeline</h3><p><strong>Ingest File Workflow:</strong></p><p>To enable comprehensive information retrieval and language generation, The ingestion process involves splitting PDFs into pages, extracting text into sentences and overlapping chunks, embedding the textual data into high-dimensional vectors using techniques like Jina’s Embedding API, and storing these vector representations in a database optimized for vector similarity search. By indexing and embedding PDF content, the RAG application can effectively retrieve relevant information to augment its language generation capabilities, providing accurate and informative responses grounded in factual data.</p><p><a href="https://github.com/Nelsonlin0321/chat-pdf-ai-assistant-fastapi/blob/main/app/routers/v1/endpoints.py#L40">https://github.com/Nelsonlin0321/chat-pdf-ai-assistant-fastapi/blob/main/app/routers/v1/endpoints.py#L40</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_4rQgxOlcoLeGYD83twCEw.png" /></figure><pre>@router.post(&quot;/ingest_file&quot;)<br>async def ingest_file(file_key: str = Form(...), chat_id: str = Form(...), file: UploadFile = File(...)):<br><br>    file_path = utils.save_file(file=file)<br>    upload_file_to_s3(file_path, file_key)<br><br>    full_text, chunk_metas = pdf_parser.parse(file_path=file_path)<br><br>    chunks = [chunk[&#39;text&#39;] for chunk in chunk_metas]<br>    embeddings = jina_ai.get_embeddings(chunks)<br>    for embedding, metas in zip(embeddings, chunk_metas):<br>        metas[&#39;embedding&#39;] = embedding<br><br>    file_name = os.path.basename(file_key)<br>    _ = mongo_db_engine.insert_file(<br>        file_name, file_key, full_text)<br><br>    for chunk in chunk_metas:<br>        chunk[&#39;chat_id&#39;] = chat_id<br>        chunk[&#39;file_key&#39;] = file_key<br>        chunk[&#39;file_name&#39;] = file_name<br><br>    # chunk_metas: List[<br>    # {&quot;text&quot;: str,<br>    # &quot;page_number&quot;: List[int]),<br>    # &quot;word_size&quot;: int,<br>    # &quot;chunk_id&quot;: int<br>    # &quot;file_name&quot;: str,<br>    # &quot;embedding&quot;: List[List[float]]<br>    # &quot;file_key&quot;:str<br>    # uploaded_file_id: str<br>    # chat_id: str<br>    # }[<br><br>    mongo_db_engine.insert_embedding(chunk_metas)<br><br>    return {&quot;messages&quot;: &quot;Ingested file successfully&quot;}</pre><p>There are steps to achieve advanced RAG file ingestion.</p><ul><li>The PDF file is first split into individual pages using a PDF PyPDF.</li><li>Each page is then processed using a textblob library to convert the page content into sentences.</li><li>The sentences from each page are merged into larger chunks with overlapping text between consecutive chunks. <strong>This overlapping helps maintain context during search and retrieval.</strong></li><li>The chunked text data is then passed through an embedding pipeline using the Jina Embedding API. This step converts the textual data into high-dimensional vector representations.</li><li>The resulting vector embeddings, along with their corresponding text chunks, are ingested and stored in a vector database like MongoDB.</li><li>The original PDF file also is uploaded to an S3 storage service for front display.</li></ul><p><strong>Search Retrieval Workflow:</strong></p><p>we achieve this by implementing an API called hybrid_search:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*uA4f0qBbKG_dbXDp3uYzzQ.png" /></figure><p>There are steps to achieve advanced content retrieval by hybrid search.</p><ul><li>User question embedding by using Jina embedding API</li><li>Perform vector and keyword search with top k relevancy</li><li>Deduplicate vector and keyword search results and rerank them by calling Jina rerank</li></ul><p><a href="https://github.com/Nelsonlin0321/chat-pdf-ai-assistant-fastapi/blob/main/app/routers/v1/endpoints.py#L98">https://github.com/Nelsonlin0321/chat-pdf-ai-assistant-fastapi/blob/main/app/routers/v1/endpoints.py#L98</a></p><pre>@router.get(&quot;/hybrid_search&quot;)<br>async def hybrid_search(query: str, chat_id: str, limit: int = 5):<br>    keyword_search_results = await keyword_search(<br>        query=query, chat_id=chat_id, limit=limit)<br><br>    vector_search_results = await vector_search(<br>        query=query, chat_id=chat_id, limit=limit)<br><br>    deduplicated_search_result = deduplicate(vector_search_results,<br>                                             keyword_search_results, id_field=&#39;chunk_id&#39;)<br><br>    chunks = [item[&quot;text&quot;] for item in deduplicated_search_result]<br><br>    reranked_indics, relevance_scores = jina_ai.rerank(<br>        query=query, chunks=chunks, top_n=limit)<br><br>    reranked_results = np.array(deduplicated_search_result)[<br>        reranked_indics].tolist()<br><br>    for score, item in zip(relevance_scores, reranked_results):<br>        item[&quot;score&quot;] = score<br><br>    return reranked_results</pre><h3>Prompt Engineering with Genimi for RAG</h3><p>Prompt Engineering using Genimi is more difficult than using ChatGPT because ChatGPT offers the system role with instruction that allows users to adjust ChatGPT how to answer questions based on the retrieved context. However, this is not the case for Genimi. Genimi only has a user and model role, which means that we have to rewrite the user question with a prompt to ask Genimi generation questions based on the retrieved context. This comes with a problem the rewritten questions will be displayed with long text on the chat history. So, we have to reverse the question to the original question.</p><p>Here are the main functions to do prompt generation in typescript with Genimi for RAG:</p><p><strong>The function to do prompt engineering for the user question: </strong><a href="https://github.com/Nelsonlin0321/chat-pdf-ai-assistant/blob/main/src/app/api/ai/chat/route.ts#L13">https://github.com/Nelsonlin0321/chat-pdf-ai-assistant/blob/main/src/app/api/ai/chat/route.ts#L13</a></p><pre>function getPrompt(message: Message, context: string) {<br>  return {<br>    ...message,<br>    role: &quot;user&quot;,<br>    content: `Answer below question without any prior knowledge and only based on the given documents, <br>    ,and tell user where you get the answers if the question is relevant to the question: <br>    ---------------------<br>    QUESTION -&gt; ${message.content}.<br>    ---------------------<br>    Documents -&gt; :<br>    ${context}`,<br>  } as Message;<br>}</pre><p><strong>The function to call the backend search function for context retrieval:</strong> <a href="https://github.com/Nelsonlin0321/chat-pdf-ai-assistant/blob/main/src/lib/search.ts">https://github.com/Nelsonlin0321/chat-pdf-ai-assistant/blob/main/src/lib/search.ts</a></p><pre>const BACKEND_URL = process.env.NEXT_PUBLIC_LAMBDA_BACKEND_URL;<br><br>type Props = {<br>  chat_id: string;<br>  query: string;<br>  search_type: &quot;keyword_search&quot; | &quot;vector_search&quot; | &quot;hybrid_search&quot;;<br>  limit: string;<br>};<br><br>export type DocMeta = {<br>  text: string;<br>  page_number: number[];<br>};<br><br>export async function search({ chat_id, query, search_type, limit }: Props) {<br>  const data = { chat_id, query, limit };<br>  const searchParams = new URLSearchParams(data);<br><br>  const response = await fetch(<br>    `${BACKEND_URL}/${search_type}?${searchParams}`,<br>    {<br>      method: &quot;GET&quot;,<br>      headers: {<br>        &quot;Content-Type&quot;: &quot;application/json&quot;,<br>      },<br>    }<br>  );<br><br>  const searchResults: DocMeta[] = await response.json();<br>  return searchResults;<br>}</pre><p><strong>The function to build the prompt with retrieved context before feeding to Genimi API:</strong></p><p><a href="https://github.com/Nelsonlin0321/chat-pdf-ai-assistant/blob/main/src/app/api/ai/chat/route.ts#L60">https://github.com/Nelsonlin0321/chat-pdf-ai-assistant/blob/main/src/app/api/ai/chat/route.ts#L60</a></p><pre>async function buildRAGPrompt(messages: Message[], chat_id: string) {<br>  const lastMessage = messages[messages.length - 1];<br><br>  const searchResults = await search({<br>    chat_id,<br>    query: lastMessage.content,<br>    search_type: &quot;hybrid_search&quot;,<br>    limit: &quot;5&quot;,<br>  });<br><br>  const context = searchResults.map((doc) =&gt; doc.text).join(&quot;\n&quot;);<br><br>  const prompt: Message = getPrompt(lastMessage, context);<br><br>  messages[messages.length - 1] = prompt;<br><br>  return messages;<br>}</pre><p><strong>Here is the API to call Genimi API with the RAG prompt:</strong> <a href="https://github.com/Nelsonlin0321/chat-pdf-ai-assistant/blob/main/src/app/api/ai/chat/route.ts#L79">https://github.com/Nelsonlin0321/chat-pdf-ai-assistant/blob/main/src/app/api/ai/chat/route.ts#L79</a></p><p>There is a parameter called ReadPDF, which controls if we let Genimi answer our question only based on the retrieved context. If we turn it off, it won’t call search API and acts like a chatbot only without any context.</p><pre>export async function POST(req: Request) {<br>  // Extract the `prompt` from the body of the request<br>  const { messages, chat_id, readPDF } = await req.json();<br><br>  const lastMessage = messages[messages.length - 1];<br><br>  let promptMessages: Message[];<br><br>  if (readPDF) {<br>    promptMessages = await buildRAGPrompt(messages, chat_id);<br>  } else {<br>    promptMessages = messages;<br>  }<br><br>  const chatPrompt = buildChatPrompt(promptMessages);<br><br>  const geminiStream = await genAI<br>    .getGenerativeModel({ model: &quot;gemini-pro&quot; })<br>    .generateContentStream(buildGoogleGenAIPrompt(chatPrompt));<br><br>  // Convert the response into a friendly text-stream<br>  const stream = GoogleGenerativeAIStream(geminiStream, {<br>    // onStart: async () =&gt; {},<br>    onCompletion: async (completion) =&gt; {<br>      await prisma.message.create({<br>        data: {<br>          chatId: chat_id,<br>          content: lastMessage.content,<br>          role: lastMessage.role,<br>        },<br>      });<br><br>      await prisma.message.create({<br>        data: {<br>          chatId: chat_id,<br>          content: completion,<br>          role: &quot;assistant&quot;,<br>        },<br>      });<br>    },<br>  });<br><br>  // Respond with the stream<br>  return new StreamingTextResponse(stream);<br>}</pre><p>For example, if we ask it to write codes with ReadPDF on, it cannot answer the question because the retrieved document doesn’t contain this information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m5ovU-LVWvsuxx4h9o5zRg.png" /></figure><p>In this post, we’ve learned how to build a robust full-stack chat PDF application leveraging modern technologies like FastAPI, Next.js, Jina, and Vercel. By combining traditional search techniques with vector similarity search, we’ve created a powerful application that can efficiently search and retrieve relevant information from PDF documents. Additionally, we know how to integrate with Genimi API to enhance to achieve custom prompt engineering.</p><p>For more details, please feel free to check out the GitHub repositories or visit the real application to play around!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9892fbe645b1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Full Stack Implementation to Build an RAG (Retrieval Augmented Generation) Application]]></title>
            <link>https://medium.com/@nelsonlin0321/full-stack-implementation-to-build-an-rag-retrieval-augmented-generation-application-68611a6ffec6?source=rss-24280f9abccc------2</link>
            <guid isPermaLink="false">https://medium.com/p/68611a6ffec6</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[nextjs]]></category>
            <category><![CDATA[react]]></category>
            <dc:creator><![CDATA[Nelson LIN]]></dc:creator>
            <pubDate>Mon, 25 Dec 2023 14:40:41 GMT</pubDate>
            <atom:updated>2024-01-22T12:00:08.035Z</atom:updated>
            <content:encoded><![CDATA[<h3>Full Stack Implementation to Build a RAG (Retrieval Augmented Generation) Application</h3><h3>Introduction:</h3><p>Retrieval-Augmented Generation (RAG) is an architectural approach that enhances the effectiveness of large language model (LLM) applications by incorporating custom data. It achieves this by retrieving pertinent data or documents related to a specific question or task and utilizing them as contextual information for the LLM. RAG has proven to be successful in supporting chatbots and question-answering systems that require access to current information or domain-specific knowledge. In this blog, we will present a comprehensive solution for developing an RAG application using the following technology stack.</p><p><a href="https://rag-delta.vercel.app/">https://rag-delta.vercel.app/</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vj5uewvm8XPPzUPZJsKQpw.png" /></figure><h3>Tech Stack</h3><ol><li>Database: MongoDB with Atlas vector search, Prisma For ORM</li><li>LLM and model: ChaGPT, Huggingface sentence transformer</li><li>API: FastAPI</li><li>FrontEnd: Next.js with React</li><li>UI: Material UI &amp; Semantic UI &amp; Radix UI</li><li>Cloud: Google Cloud Platform &amp; Vercel</li><li>Tools: LlamaIndex</li></ol><h3>Architecture Overview:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MDQbuqqgKqLw7QbetU2_ZQ.png" /></figure><p><strong>Backend Github</strong>: <a href="https://github.com/Nelsonlin0321/webdev-rag-backend-api">https://github.com/Nelsonlin0321/webdev-rag-backend-api</a></p><p><strong>Frontend Github</strong>: <a href="https://github.com/Nelsonlin0321/webdev-nextjs-rag">https://github.com/Nelsonlin0321/webdev-nextjs-rag</a></p><h3>Backend:</h3><p>At the backend, ingestion, and Retrieval Generation are two key parts of the RAG application.</p><h4>Ingestion:</h4><p>The purpose of the ingestion is to prepare for effective retrieval of relevant information from a given PDF document by transforming the text into meaningful chunks, encoding them into numerical representations, and storing them in a vector search database for efficient retrieval.</p><p>The process includes reading a PDF, making chunks, encoding sentences to embeddings, and loading them into a vector search database. It involves the following steps:</p><ol><li>Reading a PDF and Making Chunks: We use llamma-index to help us read the pdf and divide the extracted text into smaller, meaningful segments.</li><li>Encoding sentences to embeddings: Sentence encoding is the process of converting sentences into fixed-length numerical representations called embeddings. Embeddings capture the semantic meaning of the sentences, enabling comparison and similarity calculations. By encoding sentences to embeddings, we can perform efficient and accurate similarity searches. We leverage sentence_transfomders to encode sentences to embeddings</li><li>Loading embeddings into a vector search database: A vector search database is a specialized database that efficiently stores and retrieves vector representations (embeddings) based on their similarity. We use MongoDB Atlas Vector search and create the below vector index:</li></ol><pre>{<br>  &quot;fields&quot;: [<br>    {<br>      &quot;numDimensions&quot;: 384,<br>      &quot;path&quot;: &quot;embedding&quot;,<br>      &quot;similarity&quot;: &quot;dotProduct&quot;,<br>      &quot;type&quot;: &quot;vector&quot;<br>    },<br>    {<br>      &quot;path&quot;: &quot;fileName&quot;,<br>      &quot;type&quot;: &quot;filter&quot;<br>    }<br>  ]<br>}</pre><p>This is the endpoint to ingest files, process files and load the embedding to the database for a vector search query</p><pre>@app.post(f&quot;{PREFIX}/ingest&quot;)<br>async def ingest_file(file: UploadFile = File(...)):<br>    try:<br>        if not mongo_db_engine.file_exist(file_name=file.filename):<br>            save_file_path = utils.save_file(file=file)<br>            doc_meta_list = embedding_generator(save_file_path)<br>            mongo_db_engine.insert_embedding(doc_meta_list)<br>            mongo_db_engine.insert_document(file_name=file.filename)<br>            os.remove(save_file_path)<br><br>            return {&quot;message&quot;:<br>                    f&quot;The file: {file.filename} has been successfully ingested and processed!&quot;}<br><br>    # pylint: disable=broad-exception-caught<br>    except Exception as e:<br>        raise HTTPException(status_code=500, detail=str(e)) from e</pre><pre>&quot;&quot;&quot;PDT to sentence embedding&quot;&quot;&quot;<br>from sentence_transformers import SentenceTransformer<br>from llama_index import SimpleDirectoryReader<br><br>class PDFToSentenceEmbedding():<br>    def __init__(self):<br><br>        self.model = SentenceTransformer(&#39;BAAI/bge-small-en-v1.5&#39;)<br><br>    def load_document(self, file_path):<br>        documents = SimpleDirectoryReader(<br>            input_files=[file_path]<br>        ).load_data()<br><br>        return documents<br><br>    def generate_embedding(self, file_path):<br>        documents = self.load_document(file_path)<br>        texts = [doc.text for doc in documents]<br>        embeddings = self.model.encode(texts, normalize_embeddings=True)<br>        document_meta_list = [{&quot;fileName&quot;: doc.metadata[&#39;file_name&#39;],<br>                               &quot;textIdx&quot;: idx,<br>                               &quot;pageLabel&quot;: doc.metadata[&#39;page_label&#39;],<br>                               &quot;text&quot;: doc.text,<br>                               &quot;embedding&quot;: embeddings[idx].tolist(),<br>                               } for idx, doc in enumerate(documents)]<br>        return document_meta_list<br><br>    def __call__(self, file_path):<br>        document_meta_list = self.generate_embedding(file_path)<br>        return document_meta_list<br><br>embedding_generator = PDFToSentenceEmbedding()</pre><h4>Retrieval Generation</h4><p>The Retrieval Augmented Generation (RAG) in general combines the capabilities of information retrieval and language generation models to enhance the process of generating responses or outputs in a conversational or question-answering system.</p><p>The process includes encoding user query to an embedding, information retrieval by vector searching, Prompt Generation, and Question Answering as below.</p><ol><li><strong>User Query Embedding</strong>: Converting a textual user query into a fixed-length numerical representation that captures its semantic meaning. This process allows for efficient comparison and similarity calculations between queries. Remember to use the same embedding model (BAAI/bge-small-en-v1.5) to encode sentences to embeddings.</li></ol><pre>query_vector = embedding_generator.model.encode(pay_load.question).tolist()py</pre><p>2. <strong>Information retrieval by vector searching</strong>: This process allows for efficient and accurate retrieval of similar or related items. We use MongoDB Atlas Search as a search engine. This is the function of how to perform a vector search after creating a vector search index.</p><pre>    def vector_search(self, query_vector: List[float], file_name: str) -&gt; List[Dict]:<br>        results = self.db[EMBEDDING_COLLECTION].aggregate([<br>            {<br><br>                &quot;$vectorSearch&quot;: {<br>                    &quot;index&quot;: &quot;vector_index&quot;,<br>                    &quot;path&quot;: &quot;embedding&quot;,<br>                    &quot;queryVector&quot;: query_vector,<br>                    &quot;numCandidates&quot;: 5,<br>                    &quot;limit&quot;: 5,<br><br>                    &quot;filter&quot;: {&quot;fileName&quot;: {&quot;$eq&quot;: file_name}}<br>                }<br><br>            },<br>            {<br><br>                &#39;$project&#39;: {<br>                    &#39;embedding&#39;: 0,<br>                    &quot;_id&quot;: 0,<br>                    &quot;score&quot;: {&quot;$meta&quot;: &quot;vectorSearchScore&quot;},<br>                }<br><br>            }<br><br>        ])<br><br>        return list(results)</pre><p>3. <strong>Prompt Generation</strong>: A prompt is generated using the retrieved sentences from the document. The generated prompt acts as the context for the subsequent question-answering step, providing relevant information to the RAG model.</p><p>This is how we generate the prompt after retrieval.</p><pre>def generate_prompt(retrieved_results):<br>    # pylint:disable=line-too-long<br>    num_retrieved = len(retrieved_results)<br>    retrieved_documents = &quot;\n&quot;.join([<br>        f&quot;{idx+1}. Retrieved sentences: {retrieval[&#39;text&#39;]}&quot; for idx, retrieval in enumerate(retrieved_results)])<br>    prompt = f&quot;Given the top {num_retrieved} retrieved document sentences, please generate a concise and accurate answer to the following question: \n {retrieved_documents}&quot;<br>    return prompt</pre><p>4. <strong>Question Answering</strong>: The generated prompt and the user’s question are passed to the OpenAI GPT-3.5 Turbo model using the client.chat.completions.create() method. The RAG model processes the prompt and the user&#39;s question to generate an answer. The temperature parameter controls the randomness of the generated answer. A lower value produces more focused and deterministic answers, while a higher value introduces more randomness.</p><pre>@app.post(f&quot;{PREFIX}/retrieval_generate&quot;)<br>async def retrieval_generate(pay_load: PayLoad):<br>    # try:<br>    query_vector = embedding_generator.model.encode(pay_load.question).tolist()<br>    retrieved_results = mongo_db_engine.vector_search(<br>        query_vector=query_vector, file_name=pay_load.file_name)<br>    <br>    prompt = utils.generate_prompt(retrieved_results)<br><br>    completion = client.chat.completions.create(<br>        model=&#39;gpt-3.5-turbo&#39;,<br>        messages=[<br>            {&#39;role&#39;: &#39;system&#39;,<br>             &#39;content&#39;: prompt,<br>             },<br>            {&quot;role&quot;: &quot;user&quot;,<br>             &quot;content&quot;: pay_load.question}<br>        ],<br>        temperature=0,<br>        stream=False<br>    )<br><br>    return {&quot;question&quot;: pay_load.question,<br>            &quot;file_name&quot;: pay_load.file_name,<br>            &quot;answer&quot;: completion.choices[0].message.content,<br>            &quot;uuid&quot;: str(uuid4())}</pre><h3>FrontEnd</h3><p>In general, the front end is implemented by Next.js with React based on Radix and Semantic UI. At the front end, these are major components:</p><ol><li>Upload File</li><li>Search File</li><li>Question and Answering</li></ol><h4>1. Upload File</h4><p>The upload file component will trigger the API api/ingest to ingest, process files, and load document chunk embeddings to vector database.</p><pre>&quot;use client&quot;;<br>import React, { useState } from &quot;react&quot;;<br>import apiClient from &quot;../services/api-client&quot;;<br>import { AxiosError } from &quot;axios&quot;;<br>import Spinner from &quot;./Spinner&quot;;<br>import toast, { Toaster } from &quot;react-hot-toast&quot;;<br>import { useRouter } from &quot;next/navigation&quot;;<br>import { Button } from &quot;semantic-ui-react&quot;;<br><br>const FileUploader = () =&gt; {<br>  const [file, setFile] = useState&lt;File&gt;();<br>  const [isSubmitting, setSubmitting] = useState(false);<br>  const router = useRouter();<br><br>  const onSubmit = async (e: React.FormEvent&lt;HTMLFormElement&gt;) =&gt; {<br>    e.preventDefault();<br>    if (!file) return;<br>    if (!file.name.endsWith(&quot;.pdf&quot;)) {<br>      toast.error(&quot;Only PDF document supported&quot;, { duration: 1000 });<br>      return;<br>    }<br>    setSubmitting(true);<br>    try {<br>      const data = new FormData();<br>      data.set(&quot;file&quot;, file);<br>      await apiClient.post(&quot;/api/ingest&quot;, data, {<br>        headers: {<br>          &quot;Content-Type&quot;: &quot;multipart/form-data&quot;,<br>        },<br>      });<br>      router.refresh();<br>      toast.success(&quot;File uploaded successfully!&quot;, { duration: 1000 });<br>    } catch (error) {<br>      const response = (error as AxiosError).response?.data;<br>      const message = (response as { message: string }).message;<br>      const errorMessage = message || &quot;File Uploading Failed!&quot;;<br>      toast.error(errorMessage, { duration: 1000 });<br>    } finally {<br>      setSubmitting(false);<br>    }<br>  };<br><br>  return (<br>    &lt;div className=&quot;w-full&quot;&gt;<br>      &lt;form onSubmit={onSubmit}&gt;<br>        &lt;input<br>          className=&quot;mb-2 block w-full cursor-pointer rounded-lg border dark:border-gray-300&quot;<br>          type=&quot;file&quot;<br>          name=&quot;file&quot;<br>          onChange={(e) =&gt; {<br>            if (e.target.files !== null) {<br>              setFile(e.target.files[0]);<br>            }<br>          }}<br>        /&gt;<br>        &lt;Button type=&quot;submit&quot; className=&quot;cursor-pointer&quot; color=&quot;blue&quot;&gt;<br>          {isSubmitting ? &quot;Processing File&quot; : &quot;Upload and Process File&quot;}<br>          {isSubmitting &amp;&amp; &lt;Spinner /&gt;}<br>        &lt;/Button&gt;<br>      &lt;/form&gt;<br>      &lt;Toaster /&gt;<br>    &lt;/div&gt;<br>  );<br>};<br><br>export default FileUploader;https://github.com/Nelsonlin0321/webdev-nextjs-rag</pre><h4>2. Search File</h4><p>Search File is an important component that allows users to select which document they ask to. We implement the search file component using the Material UI Auto Complete component that enables us to search document quickly:</p><pre>&quot;use client&quot;;<br>import TextField from &quot;@mui/material/TextField&quot;;<br>import Autocomplete from &quot;@mui/material/Autocomplete&quot;;<br>import { InputAdornment } from &quot;@mui/material&quot;;<br>import SearchIcon from &quot;@mui/icons-material/Search&quot;;<br>import { useState } from &quot;react&quot;;<br><br>interface Props {<br>  fileNames: string[];<br>  setFileName: (fileName: string) =&gt; void;<br>}<br><br>const FileSearcher = ({ fileNames, setFileName }: Props) =&gt; {<br>  const [value, setValue] = useState&lt;string | null&gt;(fileNames[0]);<br>  const [inputValue, setInputValue] = useState&lt;string | undefined&gt;(<br>    fileNames[0]<br>  );<br><br>  return (<br>    &lt;Autocomplete<br>      value={value}<br>      onChange={(event: any, newValue: string | null) =&gt; {<br>        setValue(newValue);<br>      }}<br>      inputValue={inputValue}<br>      onInputChange={(event: any, newInputValue) =&gt; {<br>        setInputValue(newInputValue);<br>        setFileName(newInputValue);<br>      }}<br>      className=&quot;rounded-lg border w-full&quot;<br>      disablePortal<br>      id=&quot;combo-box-demo&quot;<br>      options={fileNames}<br>      renderInput={(params) =&gt; (<br>        &lt;TextField<br>          size=&quot;small&quot;<br>          InputLabelProps={params.InputLabelProps}<br>          InputProps={{<br>            ...params.InputProps,<br>            startAdornment: (<br>              &lt;InputAdornment position=&quot;start&quot;&gt;<br>                &lt;SearchIcon /&gt;<br>                Search PDF<br>              &lt;/InputAdornment&gt;<br>            ),<br>          }}<br>          id={params.id}<br>          inputProps={params.inputProps}<br>          fullWidth={params.fullWidth}<br>        /&gt;<br>      )}<br>    /&gt;<br>  );<br>};<br><br>export default FileSearcher;</pre><h4>3. Question and Answering</h4><p>This component involves question-asking and chat history maintenance. In the question-asking component, we will trigger api/retrieval-generate API to update the chat records state.</p><pre>&quot;use client&quot;;<br>import { TextField } from &quot;@radix-ui/themes&quot;;<br>import { useRef, useState } from &quot;react&quot;;<br>import { Button } from &quot;semantic-ui-react&quot;;<br>import toast, { Toaster } from &quot;react-hot-toast&quot;;<br>import apiClient from &quot;../services/api-client&quot;;<br>import { chatRecord } from &quot;./Chatbot&quot;;<br>// import { AxiosError } from &quot;axios&quot;;<br>import Spinner from &quot;./Spinner&quot;;<br><br>interface Props {<br>  fileName: string;<br>  fileNames: string[];<br>  chatRecords: chatRecord[];<br>  setChatRecords: (records: chatRecord[]) =&gt; void;<br>}<br><br>const QuestionField = ({<br>  fileName,<br>  fileNames,<br>  chatRecords,<br>  setChatRecords,<br>}: Props) =&gt; {<br>  const questionRef = useRef&lt;HTMLInputElement&gt;(null);<br>  const [isLoading, setLoading] = useState(false);<br>  const submitData = { question: &quot;&quot;, file_name: fileName };<br><br>  return (<br>    &lt;div className=&quot;w-full&quot;&gt;<br>      &lt;form<br>        onSubmit={async (event) =&gt; {<br>          event.preventDefault();<br><br>          if (questionRef.current != null) {<br>            if (questionRef.current.value.split(&quot; &quot;).length &lt; 3) {<br>              toast.error(&quot;The question requires at least 3 words&quot;);<br>              return;<br>            }<br><br>            if (!fileNames.includes(fileName)) {<br>              toast.error(&quot;The selected PDF document doesn&#39;t exist!&quot;);<br>              return;<br>            }<br>            submitData.question = questionRef.current.value;<br>            setLoading(true);<br>            try {<br>              await apiClient<br>                .post&lt;chatRecord&gt;(&quot;/api/retrieval_generate&quot;, submitData)<br>                .then((res) =&gt; {<br>                  setChatRecords([res.data, ...chatRecords]);<br>                  setLoading(false);<br>                });<br>            } catch (error) {<br>              // const response = (error as AxiosError).response?.data;<br>              // const message = (response as { message: string }).message;<br>              // const errorMessage = message || &quot;Unexpected Error&quot;;<br>              const errorMessage = &quot;Unexpected Error&quot;;<br>              toast.error(errorMessage, { duration: 1000 });<br>            } finally {<br>              setLoading(false);<br>            }<br>          }<br>        }}<br>      &gt;<br>        &lt;div&gt;<br>          &lt;span&gt;<br>            Please write down your question related to selected PDF Document<br>          &lt;/span&gt;<br>          &lt;TextField.Root className=&quot;mb-2&quot;&gt;<br>            &lt;TextField.Input<br>              placeholder=&quot;Example Question: What are steps to take when finding projects to build your AI experience ?&quot;<br>              ref={questionRef}<br>            /&gt;<br>          &lt;/TextField.Root&gt;<br>          &lt;Button type=&quot;submit&quot; className=&quot;cursor-pointer&quot; color=&quot;blue&quot;&gt;<br>            Ask<br>            {isLoading &amp;&amp; &lt;Spinner /&gt;}<br>          &lt;/Button&gt;<br>        &lt;/div&gt;<br>      &lt;/form&gt;<br>      &lt;Toaster /&gt;<br>    &lt;/div&gt;<br>  );<br>};<br><br>export default QuestionField;</pre><p>To render the chat history, we use the Semantic UI Accordion component instead of using the chatbot conversation style because the answer could be very long. Accordion enables use to collapse the long answer with a document tag.</p><pre>&quot;use client&quot;;<br>import { useEffect, useState } from &quot;react&quot;;<br>import { Accordion, Icon, Label } from &quot;semantic-ui-react&quot;;<br>import { chatRecord } from &quot;./Chatbot&quot;;<br>import { Text } from &quot;@radix-ui/themes&quot;;<br><br>interface Props {<br>  chatRecords: chatRecord[];<br>}<br><br>const ChatHistory = ({ chatRecords }: Props) =&gt; {<br>  const [activeIndex, setActiveIndex] = useState(0);<br><br>  useEffect(() =&gt; {<br>    setActiveIndex(0);<br>  }, [chatRecords]);<br><br>  return (<br>    &lt;Accordion fluid styled&gt;<br>      {chatRecords.map((message, index) =&gt; (<br>        &lt;div key={index}&gt;<br>          &lt;Accordion.Title<br>            active={activeIndex === index}<br>            onClick={() =&gt; {<br>              if (activeIndex == index) {<br>                setActiveIndex(-1);<br>              } else {<br>                setActiveIndex(index);<br>              }<br>            }}<br>          &gt;<br>            &lt;Icon name=&quot;dropdown&quot; /&gt;<br>            {message.question}<br>            &lt;div&gt;<br>              &lt;Label color=&quot;orange&quot; ribbon=&quot;right&quot;&gt;<br>                &lt;p style={{ maxWidth: 256 }} className=&quot;truncate&quot;&gt;<br>                  {message.file_name}<br>                &lt;/p&gt;<br>              &lt;/Label&gt;<br>            &lt;/div&gt;<br>          &lt;/Accordion.Title&gt;<br><br>          &lt;Accordion.Content active={activeIndex === index}&gt;<br>            &lt;Text className=&quot;text-gray-800 mb-4 whitespace-pre-line&quot;&gt;<br>              {message.answer}<br>            &lt;/Text&gt;<br>          &lt;/Accordion.Content&gt;<br>        &lt;/div&gt;<br>      ))}<br>    &lt;/Accordion&gt;<br>  );<br>};<br><br>export default ChatHistory;</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=68611a6ffec6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Connect and work on two Mac like a single one, sharing files by just dragging with one keyboard and…]]></title>
            <link>https://medium.com/@nelsonlin0321/working-on-two-mac-computers-like-a-single-machine-sharing-files-by-just-dragging-with-one-43a5b25aab4c?source=rss-24280f9abccc------2</link>
            <guid isPermaLink="false">https://medium.com/p/43a5b25aab4c</guid>
            <category><![CDATA[mac]]></category>
            <category><![CDATA[productivity]]></category>
            <dc:creator><![CDATA[Nelson LIN]]></dc:creator>
            <pubDate>Sun, 03 Dec 2023 07:09:10 GMT</pubDate>
            <atom:updated>2023-12-04T03:23:17.858Z</atom:updated>
            <content:encoded><![CDATA[<h3>Connect and work on two Mac like a single one, sharing files by just dragging with one keyboard and mouse.</h3><p>Ususally, we have to work on two mac computers to do some tasks simultaneously but I have been tired of constantly switching between multiple Mac computers while working on multiple projects? Now, I’ve explore the convenience of seamlessly merging the power of two Mac computers into one, allowing you to work as if you were using a single machine.</p><p>This post explores a solution that allows you to seamlessly integrate two Mac computers like a single machine and enables you to effortlessly share files between two Mac computers by simply dragging and droppin. With this innovative approach, you can streamline your workflow, maximize your resources, and experience a newfound level of convenience and efficiency in your day-to-day work. Let’s delve into the world of working on two Mac computers like a single machine and discover how this feature can revolutionize your workflow.</p><p>Now, I shared the solution by taking Mac Mini and Macbook Pro to connect them and working on them like a single machine.</p><h3><strong>Setup Overview:</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W5wDhxbr6_UwwH99ZxZ0kQ.png" /></figure><h3>How it works ?</h3><p>Based on my observations, I have formulated an understanding of the operational mechanism involved. <strong>A key component in this setup is the HDMI Switcher, which features two input ports and one output port.</strong> Traditionally, the switcher is responsible for controlling the display on the second monitor, allowing the Mac Mini or MacBook Pro screen to be displayed. However, when switching to the MacBook Pro screen, <strong>the HDMI Switcher also accepts signals from the Type C Hub connected to the Mac Mini and shares the signal with the MacBook Pro. </strong>Consequently, the Macbook Pro can accept the signal from, and controlled by the keyboard and mouse. This configuration also enables file transfers between the two Mac computers by utilizing the HDMI switcher, which connects the two Type C hubs from the Mac Mini and MacBook Pro.</p><h3>Lee see how it works</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FSkThwV45z6s%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DSkThwV45z6s&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FSkThwV45z6s%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/3e7ba6e6ae65984c1bfbbe3051653faa/href">https://medium.com/media/3e7ba6e6ae65984c1bfbbe3051653faa/href</a></iframe><p>On the left is the MacBook Pro, while on the right is the Mac Mini. We have established the connections between them exactly as demonstrated. Notably, the mouse seamlessly moves between the screens, allowing for easy file transfer through drag-and-drop functionality. Both the keyboard and mouse can be connected to either the Mac Mini or the MacBook’s Type C Hub. However, it is important to note that certain shortcuts, such as Ctrl+C, will only function on the machine to which the keyboard is connected.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=43a5b25aab4c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[architecture of React Interactive Movie Recommendation System Based on Behavior Sequence…]]></title>
            <link>https://medium.com/@nelsonlin0321/architecture-of-react-interactive-movie-recommendation-system-based-on-behavior-sequence-1b1597d8981e?source=rss-24280f9abccc------2</link>
            <guid isPermaLink="false">https://medium.com/p/1b1597d8981e</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[recommendations]]></category>
            <category><![CDATA[architecture]]></category>
            <dc:creator><![CDATA[Nelson LIN]]></dc:creator>
            <pubDate>Tue, 10 Oct 2023 14:30:33 GMT</pubDate>
            <atom:updated>2024-08-14T07:38:38.893Z</atom:updated>
            <content:encoded><![CDATA[<h3>Architecture and Application of Interative Transformer-based Movie Recommendation System</h3><h3><a href="https://github.com/Nelsonlin0321/webdev-react-movies-with-recommendation/tree/main#brief-introduction">Brief Introduction</a><strong>:</strong></h3><p>As machine learning engineers, we may only focus on how we train a model and deploy the model but the skills like architecture design and implementation may enable us to deliver more impactful solutions, collaborate effectively with cross-functional teams, and design robust systems that meet the demands of real-world applications. So, this article will introduce a full stack machine learning serverless architecture, which includes a model training pipeline, model deployment, data serving, and frontend integration to enable you to become a potential machine learning architect with more product ownership and entrepreneurship. All source codes are available in different microservice components.</p><p>As we know, transformer-based models have been proven to be effective in capturing sequential signals. With inspiration from the User Behavior Sequence Transformer (BST) for e-commerce recommendation at Taobao, which was proposed by Alibaba in this paper (<a href="https://lnkd.in/g3jYR_bH">https://lnkd.in/g3jYR_bH</a>), this article will introduce the architecture of the movie recommendation system.</p><h3><a href="https://github.com/Nelsonlin0321/webdev-react-movies-with-recommendation/tree/main#react-frontend-preview">React Frontend Preview</a></h3><p>Usually, a recommendation system recommends movies according to the movie view history and user demographic features. This React Frontend will ask users to input age, gender, and what movies they have watched.</p><p>React Reall App: <a href="https://main.d2sn7g07jz0u4u.amplifyapp.com/">https://main.d2sn7g07jz0u4u.amplifyapp.com/</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lVTkElMV974cVymq.png" /></figure><h3><a href="https://github.com/Nelsonlin0321/webdev-react-movies-with-recommendation/tree/main#full-stack-machine-learning-serverless-architecture">Full Stack Machine Learning Serverless Architecture</a></h3><p>The Interactive Movie Recommendation System follows a full stack machine learning serverless and microservices architecture. The architecture consists of several major components that work together to provide a seamless movie recommendation experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*slGIlPPhFarnCy9e.png" /></figure><p>There are the following major components:</p><ol><li>Orchestrated Traning Pipeline</li><li>Movie Recommendation Model Deployment CI/CD Pipeline</li><li>Database and Backend Movie APIs</li><li>React Application integrated with APIs</li></ol><h3><a href="https://github.com/Nelsonlin0321/webdev-react-movies-with-recommendation/tree/main#1-orchestrated-training-pipeline">(1) Orchestrated Training Pipeline:</a></h3><p><strong>Tech Stack</strong>: Pytorch, Prefect, Mlflow, Transformer</p><p><strong>Github</strong>: <a href="https://github.com/Nelsonlin0321/ml-bst-movielens1m-recommender-training">https://github.com/Nelsonlin0321/ml-bst-movielens1m-recommender-training</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/0*2yjWiCpPL8aAFZ_U.png" /></figure><p>The training pipeline is responsible for training a behavior sequence transformer model. This model is based on the algorithm proposed by Alibaba, which can be found in the paper link to the <a href="https://arxiv.org/abs/1905.06874">paper</a>. The pipeline uses the 1M version of the Movielens dataset to train the recommendation model. It takes into account the users’ basic demographic features and sequence of movie views. Note that during the inference stage, the system assumes that the users are anonymous and without any rating history, and therefore does not use them as features. A tutorial on how to implement the behavior sequence transformer model with all features can be found in this <a href="https://github.com/Nelsonlin0321/ml-behavior-sequence-transformer-for-movie-recommendation">GitHub</a> repository. The training pipeline was orchestrated by <a href="https://www.prefect.io/cloud">Prefect Cloud</a> but not deployed to the cloud and it’s trained by the local machine.</p><h3><a href="https://github.com/Nelsonlin0321/webdev-react-movies-with-recommendation/tree/main#2-model-deployment">(2) Model Deployment:</a></h3><p><strong>Tech Stack</strong>: Pytorch,Lambda, APIGateway, FastAPI and Terraform</p><p><strong>Github</strong>: <a href="https://github.com/Nelsonlin0321/ml-bst-movielens1m-recommender-serving">https://github.com/Nelsonlin0321/ml-bst-movielens1m-recommender-serving</a></p><p>The model trained in the training pipeline is deployed on AWS using FastAPI, Lambda, and API Gateway. This allows the model to be served as a serverless API. The API takes the users’ demographic features and sequence of movie views as input and predicts the movies that users are most likely to like. The deployment process involves using Terraform to provision the necessary AWS resources and configuring the APIGateway to handle the API requests and we schedule an EventBridge to regularly invoke health checks to avoid a cold start for fast serving.</p><h3><a href="https://github.com/Nelsonlin0321/webdev-react-movies-with-recommendation/tree/main#3-backend-movies-apis">(3) Database and Backend Movies APIs</a></h3><p><strong>Tech Stack</strong>: Dynamodb, S3, CloudFront, Lambda, APIGateway, FastAPI and Terraform</p><p><strong>Github</strong>: <a href="https://github.com/Nelsonlin0321/movielens1m-data-serving-api">https://github.com/Nelsonlin0321/movielens1m-data-serving-api</a></p><p>The system also includes a backend Movies API that is built using DynamoDB, FastAPI, Lambda, and APIGateway. The data from the <a href="http://files.grouplens.org/datasets/movielens/ml-1m.zip">Movielens dataset</a> is ingested into DynamoDB. And we crawl their cover images on Google and save them on S3. This API is responsible for serving the movie data and image URLs. DynamoDB is used as the database to store the movie data, while CloudFront is used as the content delivery network to ensure fast and reliable access to the movie cover images. The APIGateway acts as the entry point for the API requests, and Lambda functions handle the request processing.</p><h3><a href="https://github.com/Nelsonlin0321/webdev-react-movies-with-recommendation/tree/main#4-react--typescript-movie-recommendation-application">(4) React + Typescript Movie Recommendation Application</a></h3><p><strong>Tech Stack</strong>: React, Typescript, Vite, Axios, Chakra UI, AWS Amplify, Route 53</p><p><strong>Github</strong>:<a href="https://github.com/Nelsonlin0321/webdev-react-movies-with-recommendation">https://github.com/Nelsonlin0321/webdev-react-movies-with-recommendation</a></p><p>The front end of the Interactive Movie Recommendation System is built using React Typescript. The application integrates with the deployed recommendation API and the backend Movies API to provide a seamless user experience using Axios to handle HTTP requests. The UI is designed using Chakra UI, which provides a set of accessible and customizable UI components. Vite is used as the build tool for fast development and HMR (Hot Module Replacement). The application is deployed using AWS Amplify, which simplifies the deployment process. Route 53 is used for DNS management and routing the application traffic pointing to the domain <a href="https://nelsonds.com/">nelsonds.com</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1b1597d8981e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>