<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Flora Wu on Medium]]></title>
        <description><![CDATA[Stories by Flora Wu on Medium]]></description>
        <link>https://medium.com/@florawu_83458?source=rss-7e1b5f0809b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*CrAqcrZCHE1S9Pf9.</url>
            <title>Stories by Flora Wu on Medium</title>
            <link>https://medium.com/@florawu_83458?source=rss-7e1b5f0809b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 18:06:13 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@florawu_83458/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Databricks vs. Snowflake]]></title>
            <link>https://medium.com/@florawu_83458/databricks-vs-snowflake-8e83228f305a?source=rss-7e1b5f0809b------2</link>
            <guid isPermaLink="false">https://medium.com/p/8e83228f305a</guid>
            <dc:creator><![CDATA[Flora Wu]]></dc:creator>
            <pubDate>Thu, 07 May 2026 15:09:05 GMT</pubDate>
            <atom:updated>2026-05-07T16:38:56.949Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Databricks vs. Snowflake</strong></p><p>Databricks and Snowflake are both leading cloud data platforms with distinct features and strengths. This blog aims to provide a brief history of both companies, their strengths and weaknesses at high-level, and discuss when to use what platforms.</p><p><strong>History of the two companies</strong></p><p>Databricks and Snowflake started out to solve very different use cases: Databricks is founded by Apache Spark creators to provide managed Apache Spark service. Snowflake started as the SQL data warehouse on the Cloud. As time goes by, they both evolved into multi-faceted data cloud platforms with overlapping in their capabilities.</p><p>A timeline for the development of Databricks and Snowflake platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sCgzt5rRmI6B7_OgScMjEQ.png" /></figure><p><strong>High-level feature comparison</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o16jxMhG5nfSJTPK4bY5zw.png" /></figure><p><strong>In what scenarios would it make sense to use Snowflake?</strong></p><p>Unified Data Platform (UDP) backed by Databricks is Northwestern Mutual’s enterprise data and analytics environment, providing a comprehensive set of capabilities that enables the convergence of data, data governance, data privacy, data engineering, metadata management, artificial intelligence and machine learning. If a data platform or Data Warehouse Technology is not listed in Data Platform and Data Warehouse Control Solutions, it needs an exception approval.</p><p>However, for certain use cases, Snowflake might be a better choice. While Databricks stand out in handling complex data engineering and machine learning tasks, Snowflake is particularly suited for specific use cases involving data warehousing, BI and reporting, real-time analytics, data sharing, and tasks requiring scalable and secure data management. In scenarios where structured data analysis and SQL-based querying are essential, or where seamless integration with popular BI and ETL tools is a priority, Snowflake stands out as the better choice.</p><p><strong>Here are some use cases Where Snowflake is a Better Solution:</strong></p><p><strong>Using Data from Snowflake Marketplace: </strong>Access a wide variety of third-party datasets available on Snowflake Marketplace to enrich your internal data for deeper analytics and insights without significant ETL (Extract, Transform, Load) overhead.</p><p><strong>Traditional Data Warehousing: </strong>Designed to handle large-scale SQL queries efficiently. Suitable for organizations that rely heavily on structured data and SQL-based analytics.</p><p><strong>Business Intelligence (BI) and Reporting: </strong>Seamlessly works with BI tools like Tableau, Looker, Power BI, and others, facilitating the creation and maintenance of reports and dashboards.</p><p><strong>Ad-Hoc Querying: </strong>Allows business users to perform ad-hoc queries and data explorations without extensive technical knowledge, thus empowering self-service analytics.</p><p><strong>References</strong></p><p><a href="https://www.invene.com/blog/snowflake-vs-databricks">https://www.invene.com/blog/snowflake-vs-databricks</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e83228f305a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build a text-to-SQL chatbot using Agents for Amazon Bedrock with a RAG approach]]></title>
            <link>https://medium.com/@florawu_83458/build-a-text-to-sql-chatbot-using-agents-for-amazon-bedrock-with-a-rag-approach-bdd237380610?source=rss-7e1b5f0809b------2</link>
            <guid isPermaLink="false">https://medium.com/p/bdd237380610</guid>
            <dc:creator><![CDATA[Flora Wu]]></dc:creator>
            <pubDate>Tue, 03 Feb 2026 22:49:33 GMT</pubDate>
            <atom:updated>2026-02-03T22:49:33.038Z</atom:updated>
            <content:encoded><![CDATA[<p>When I worked in Amazon Web Service (AWS) as the Sr. GenAI Solution Architect, I worked with large enterprise customers to build end-to-end GenAI applications. One of my customers is a healthcare analytics company. Its mission is to empower healthcare organizations to improve quality, reduce costs, optimize performance, and drive innovation through trusted data, analytics, and Generative AI.</p><p>In this post, I will discuss the use case of building an AI chatbot using <a href="https://aws.amazon.com/bedrock/agents/">Agents for Amazon Bedrock</a> for health care plan users to help them understand and analyze healthcare provider data stored in an <a href="http://aws.amazon.com/s3">Amazon Simple Storage Service</a> (Amazon S3) data lake using natural language. The chatbot helps users understand the data, identify providers based on complex criteria, resolve data discrepancies, and improve provider network quality. It can be challenging to build a generative AI-powered application for users to retrieve data from different data sources efficiently and dynamically using natural language and provide low-latency responses with safety and quality controls. I start with presenting the solution architecture and then walk you through the building blocks. Furthermore, I’ll discuss advanced Retrieval Augmented Generation (RAG), focusing on guardrails and system evaluation. Finally, we discuss some of the architecture decisions and trade-offs.</p><h3>Solution overview</h3><p>Text-to-SQL represents an advancement in generative AI, revolutionizing the way we interact with databases. This innovative technology bridges the gap between natural language processing (NLP) and database querying by enabling users to express their queries in natural language rather than SQL syntax. For this post, I present a generative AI-powered text-to-SQL solution using Agents for Amazon Bedrock. This solution enables users to ask a question in natural language and convert that question to SQL automatically to retrieve data from an S3 data lake.</p><p>There are three critical components in this solution architecture: Agents for Amazon Bedrock for workflow orchestration, RAG with database metadata, and <a href="http://aws.amazon.com/athena">Amazon Athena</a> as the query engine.</p><p>Agents for Amazon Bedrock enables generative AI applications to run multistep tasks across company systems and data sources. Agents orchestrate interactions between large language models (LLMs), data sources, software applications, and user conversations. In addition, agents automatically call APIs to take actions and invoke knowledge bases to supplement information for these actions. Handling complex SQL generation usually requires breaking down the work into multiple steps, including schema retrieval, initial SQL generation, SQL execution, and self-correction. To automate and simplify the orchestration, we use Agents for Amazon Bedrock to orchestrate and run tasks using the tools provided.</p><p><a href="https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-customize-rag.html">RAG</a> is an architecture pattern that provides the most relevant and contextually important data to your LLMs used in generative AI applications to enhance accuracy and performance. We use the RAG approach to retrieve the relevant metadata such as schema descriptions and query examples to augment user requests for SQL generation to make sure the request for the LLM is related to the right tables, columns, and datasets.</p><p>We use Athena as our query engine, which integrates with the <a href="https://aws.amazon.com/glue">AWS Glue</a> Data Catalog and enables us to query data stored in multiple data sources, including Amazon S3. It also provides us with access control, fast query performance, and high availability.</p><p>The following diagram illustrates the solution architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/432/1*6Rs2NEONSHdoLuJdFlpDdw.png" /></figure><p>The workflow includes the following steps:</p><p>1. Users send a request to the chatbot web UI in natural language.</p><p>2. The chatbot web UI summons the Amazon Bedrock agent with the user request.</p><p>3. The Amazon Bedrock agent interacts with the selected LLM (Anthropic Claude 3 Sonnet on Amazon Bedrock) and an action group (a set of APIs and corresponding business logic implemented in an <a href="http://aws.amazon.com/lambda">AWS Lambda</a> function) to process user requests:</p><p>a. The agent uses the LLM to preprocess and categorize user requests. If the request is valid, it will proceed to the next step; otherwise, it will generate a definitive answer, such as “Sorry, I don’t have enough information to answer that,” and finish the process.</p><p>b. If the user request is valid, the agent uses the LLM to generate an orchestration plan and interact with the LLM multiple times until the agent is ready to respond to the user or until it needs to prompt the user for additional information.</p><p>4. The LLM interacts with the agent to generate an orchestration plan and a <em>rationale </em>that lays out the logic for the next step it should take. The LLM will interact with the agent multiple times until it reaches a result.</p><p>5. The agent invokes action groups implemented by Lambda to run tasks.</p><p>6. The Lambda function queries an Amazon Bedrock knowledge base or vector database to retrieve additional context, such as schema and example SQLs, to augment user requests for SQL generation.</p><p>7. The Lambda function invokes Athena to query data stored in the S3 data lake using the generated SQLs.</p><p>a. Athena uses the Data Catalog to retrieve table metadata for the Amazon S3 data and runs the SQL query.</p><p>b. The Data Catalog stores metadata about S3 data, including its location, schema, and runtime metrics.</p><p>c. The S3 data lake has healthcare provider data and other data.</p><p>8. The monitoring framework automatically collects logging, metrics, and traces from the generative AI application using <a href="http://aws.amazon.com/cloudwatch">Amazon CloudWatch</a> and <a href="http://aws.amazon.com/cloudtrail">AWS CloudTrail</a> and stores them in the S3 bucket.</p><p>9. The agent generates a final response and returns it to the user through the web UI.</p><p>The following diagram depicts the processing flow of this generative AI-powered text-to-SQL solution.</p><h3>Building blocks and technologies</h3><p>One of the main components of this solution is Agents for Amazon Bedrock. You provide an agent access to your organization’s enterprise systems, processes, knowledge bases, and some building block functions. Then, the agent produces the logic using <a href="https://arxiv.org/abs/2201.11903">Chain-of-Thought (CoT)</a> reasoning, determines what APIs to call and when to call them, and completes the transactions in the right sequence. When an agent needs a piece of information from the user, it automatically asks the user for those additional details using NLP. In our use case, we use Agents for Amazon Bedrock to break down the text-to-SQL tasks into multiple steps and orchestrate the workflow.</p><p>Agents for Amazon Bedrock uses instructions, a selected Amazon Bedrock LLM, and a set of available actions and knowledge bases. The following diagram (<a href="https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html">source</a>) shows the main components of the agent.</p><p>Agent instructions describe what the agent is designed to do and provide instructions for the agent to reason and execute the tasks. The following screenshot is an example of our instructions.</p><p>An action group defines actions that the agent can help end-users perform. Each action group has a description, an API schema to describe the actions, and a Lambda function to implement the action logic. Our Lambda function uses <a href="https://python.langchain.com/v0.1/docs/use_cases/sql/">SQLDatabaseChain</a> to help generate and run SQLs. SQLDatabaseChain is a LangChain framework that helps you create, implement, and run SQL queries, using effective text-to-SQL conversions and implementations.</p><p>The following screenshot is an example of our action group setup.</p><p><a href="https://aws.amazon.com/bedrock/knowledge-bases/">Knowledge Bases for Amazon Bedrock</a> is a fully managed capability that helps you implement the entire RAG workflow, from ingestion to retrieval and prompt augmentation, without having to build custom integrations to data sources, manage data flows, and maintain your own vector database. In our use case, we use a knowledge base to store the database schema information, background documentation, example user queries, and corresponding generated SQLs. Augmenting the prompt with a database schema and example question and SQL pairs significantly increases the accuracy of SQL generation.</p><h3>Advanced RAG</h3><p>RAG augments the prompt with retrieved information to reduce hallucinations that usually come with LLM applications. In our use case, we use RAG to retrieve relevant table schemas, background information, and query examples to assist the LLM in generating more accurate SQL queries. We also augment the prompt with retrieved information for final response generation by the LLM.</p><p><a href="https://www.deeplearning.ai/short-courses/building-evaluating-advanced-rag/">Advanced RAG</a> has evolved as a new paradigm with targeted enhancements to address some of the challenges and limitations of basic RAG. This paradigm comprises a set of techniques categorized into pre-retrieval, retrieval, and post-retrieval optimizations. In our use case, we set up guardrails to improve security and build system evaluation in post-retrieval optimizations.</p><p>AI guardrails are safety mechanisms offering guidelines and boundaries to safeguard generative AI applications. In our use case, we use <a href="https://aws.amazon.com/bedrock/guardrails/">Guardrails for Amazon Bedrock</a>, which evaluates user inputs and outputs on use case-specific policies and provides an additional layer of safeguards regardless of the underlying foundation models. For example, we can provide a prompt to filter content, deny topics, protect personally identifiable information (PII), and perform regex filters.</p><p>LLM system evaluation is a critical process used to assess the overall accuracy and performance of the LLM-based application. We measured the application’s performance using a combination of LLM-generated and user-provided feedback. The following are a few metrics to track the application’s performance.</p><p>We measured application usage using the number of questions answered, as well as the security and fair usage of the app through the guardrail. We measured this performance through the approval rate. We also measured the execution rate, the successfully run SQLs over the total SQLs generated, to check if the application builds syntactically correct SQL queries. Finally, we measured accuracy using user feedback and an LLM judge based on some ground truth queries.</p><p>We also measured the execution rate by the level of the question’s complexity. The following graph shows some of our tests.</p><h3>Decisions and trade-offs</h3><p>We made some architectural decisions along the way. The customer team started with Amazon application APIs directly to build their own custom agent. After experimenting with Agents for Amazon Bedrock, the team decided to transition to use an agent to help them scale with more data sources and complex workflows. With the experiments we conducted with both options, we found that if the steps we need to run in the workflow are static, it’s better to control it programmatically using APIs directly. This option provided more control of the workflow and more robust results. However, if we want more flexibility, it’s better to use Agents for Amazon Bedrock and let the agent decide the execution plan dynamically and orchestrate between functions or calling the same function with different inputs. Using Agents for Amazon Bedrock can help us scale and automate the workflows with more data sources and users onboarded.</p><h3>Conclusion</h3><p>In this post, I shared a chatbot use case and associated challenges, then I provided a text-to-SQL and RAG-based solution with a deep dive into the building blocks. Furthermore, I introduced advanced RAG and techniques used in this solution, including guardrails and system evaluation. Finally, I discussed some of the architecture decisions and trade-offs.</p><p>I hope this post provides a common solution for text-to-SQL and RAG-based generative AI applications and helps you resolve any challenges you’re facing in building such applications. To get started, refer to the <a href="https://github.com/aws-samples/amazon-bedrock-samples/tree/main/agents-for-bedrock/use-case-examples/text-2-sql-agent">Text to SQL Bedrock Agent workshop</a> to quickly build and scale generative AI application using Agents for Amazon Bedrock. For a deep dive into text-to-SQL approaches, refer to the <a href="https://github.com/aws-samples/text-to-sql-bedrock-workshop">Text-to-SQL Bedrock Workshop</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bdd237380610" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>