<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ima Miri on Medium]]></title>
        <description><![CDATA[Stories by Ima Miri on Medium]]></description>
        <link>https://medium.com/@ima-miri?source=rss-5bc99e07b356------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*QomLh3BG5KJB6yNONaTiTw.jpeg</url>
            <title>Stories by Ima Miri on Medium</title>
            <link>https://medium.com/@ima-miri?source=rss-5bc99e07b356------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 19:35:08 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ima-miri/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Cost Efficient Big Data Processing with Amazon EMR]]></title>
            <link>https://ima-miri.medium.com/cost-efficient-big-data-processing-with-amazon-emr-9bf2af51ef1f?source=rss-5bc99e07b356------2</link>
            <guid isPermaLink="false">https://medium.com/p/9bf2af51ef1f</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-cloud]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[emr]]></category>
            <dc:creator><![CDATA[Ima Miri]]></dc:creator>
            <pubDate>Thu, 05 Dec 2024 03:11:54 GMT</pubDate>
            <atom:updated>2024-12-05T03:11:54.002Z</atom:updated>
            <content:encoded><![CDATA[<p>This blog is a snippet of the “Mastering Solutions Architecture” course on udemy. Check out the full course <a href="https://www.udemy.com/course/mastering-solutions-architecture/?referralCode=EA50F091C66BB075B001">here</a>.</p><p>When it comes to managing large-scale data processing, Amazon EMR (Elastic MapReduce) offers a powerful solution. In this post, we’ll break down how EMR clusters work, their components, and how to set them up to achieve optimal scalability and cost-efficiency.</p><p>The video tutorial of this blog is available <a href="https://youtu.be/ryMlYLO0FCk">here</a>.</p><h3>Understanding Amazon EMR Clusters</h3><p>The core component of Amazon EMR is the <strong>cluster</strong> — a collection of Amazon EC2 instances that work together to process data.</p><figure><img alt="Amazon EMR Cluster" src="https://cdn-images-1.medium.com/max/361/1*uP-vCB1xnz7PC9tjqznaWg.png" /><figcaption>Amazon EMR Cluster</figcaption></figure><p>Each instance in the cluster is referred to as a <strong>node</strong>, and these nodes have specific roles within the cluster:</p><ol><li><strong>Primary Node</strong>: This node coordinates the cluster by managing data and task distribution, tracking task statuses, and monitoring the health of the cluster.</li><li><strong>Core Node</strong>: This node performs tasks and stores data in the Hadoop Distributed File System (HDFS). It’s essential for clusters requiring data storage.</li><li><strong>Task Node</strong>: Task nodes only perform tasks and don’t store data in HDFS. They are optional and can be added based on workload requirements.</li></ol><p>Every EMR cluster includes at least one primary node, and for multi-node clusters, core nodes are necessary, with task nodes added as needed.</p><h3>Setting Up an EMR Cluster: Why Use Instance Fleets?</h3><p>For optimal scalability and cost-efficiency, we recommend using <strong>Instance Fleets</strong> when setting up an EMR cluster.</p><figure><img alt="Instance Fleet for Amazon EMR" src="https://cdn-images-1.medium.com/max/954/1*Il32Lq59A_ERH73c9HVCwg.png" /><figcaption>Instance Fleet for Amazon EMR</figcaption></figure><p>With instance fleet configurations, you can choose multiple instance types and purchase options (On-Demand and Spot instances) within a single cluster, allowing for greater flexibility. Here’s why instance fleets are beneficial:</p><ul><li><strong>Cost Savings with Spot Instances</strong>: Spot instances are more cost-effective than On-Demand options. Using Spot instances within an instance fleet can significantly reduce costs while maintaining the necessary computational power.</li><li><strong>Automatic Resource Scaling</strong>: Instance fleets enable automatic scaling based on availability and demand within a region or availability zone, ensuring the cluster adapts to workload requirements.</li><li><strong>Enhanced Resilience</strong>: By distributing instances across different availability zones, the cluster remains operational even if some instances become unavailable.</li></ul><h3>Choosing the Right EC2 Instance Types for Your Cluster</h3><p>Selecting the appropriate instance type for EC2 instances in the EMR cluster is crucial for efficient processing.</p><figure><img alt="EC2 Instance Types" src="https://cdn-images-1.medium.com/max/861/1*uGZgnrmQ1-0TTjfhg43myQ.png" /><figcaption>EC2 Instance Types</figcaption></figure><p>Instance types vary based on the nature of the workload:</p><ul><li><strong>Compute-Optimized</strong>: Best for high-performance computations.</li><li><strong>Memory-Optimized</strong>: Ideal for tasks requiring large memory, such as data aggregation.</li><li><strong>General Purpose</strong>: Suitable for a balanced mix of compute and memory needs.</li><li><strong>Storage-Optimized</strong>: Effective for large-scale data storage and retrieval.</li></ul><p>For example, if you’re aggregating patient data, memory-optimized instances can handle data processing efficiently without memory bottlenecks.</p><h3>Building a Data Processing Workflow on EMR</h3><p>A robust data processing workflow on EMR often includes these four stages:</p><ol><li><strong>Data Cleaning</strong>: This stage ensures data quality by validating completeness, accuracy, and consistency. Cleaning involves standardizing data formats, detecting errors, and addressing issues like missing values or outliers.</li><li><strong>Data Verification</strong>: This step checks that the data adheres to the quality standards required for subsequent processing.</li><li><strong>Data Transformation</strong>: This stage normalizes and aggregates data, such as summarizing lab results or combining treatment histories. Filtering out redundant data ensures the final dataset is streamlined for analysis.</li><li><strong>Data Storage</strong>: The processed data is loaded into a storage solution like Amazon S3 or Amazon Redshift. Using partitions — such as by state, patient ID, or data type — enables efficient querying and retrieval for analysis.</li></ol><h3>Post-Processing: Cluster Termination</h3><p>Once the EMR job completes, the cluster automatically terminates, avoiding unnecessary costs. This feature makes EMR an excellent choice for managing data workflows without needing to maintain infrastructure long-term.</p><h3>Conclusion</h3><p>Amazon EMR provides a scalable, cost-effective solution for complex data processing tasks. By leveraging instance fleets, choosing the right EC2 instance types, and following a structured workflow, you can build efficient systems that adapt to varying workloads and keep costs under control.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9bf2af51ef1f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How AI Is Revolutionizing Content Creation: Serverless Workflow]]></title>
            <link>https://ima-miri.medium.com/how-ai-is-revolutionizing-content-creation-serverless-workflow-80ba383de477?source=rss-5bc99e07b356------2</link>
            <guid isPermaLink="false">https://medium.com/p/80ba383de477</guid>
            <category><![CDATA[serverless-architecture]]></category>
            <category><![CDATA[amazon-bedrock]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-solution]]></category>
            <category><![CDATA[generative-ai-solution]]></category>
            <dc:creator><![CDATA[Ima Miri]]></dc:creator>
            <pubDate>Thu, 21 Nov 2024 23:59:38 GMT</pubDate>
            <atom:updated>2024-11-21T23:59:38.237Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="AI Content Creation Workflow with Serverless Architecture" src="https://cdn-images-1.medium.com/max/1024/1*nuj5aOUJ1VeEgvAe-k3Wfg.png" /><figcaption>AI Content Creation Workflow with Serverless Architecture</figcaption></figure><p>In today’s fast-paced digital world, the demand for high-quality, engaging video content is higher than ever. Video captivates audiences in ways that text alone can’t, making it ideal for commuters, multitaskers, and anyone who wants to absorb information quickly. Traditionally, creating such content has required significant time and resources, leading many businesses to rely on text-based content. However, AI is transforming this landscape by making high-quality video content creation more accessible and cost-effective.</p><p>The video tutorial for this blog is available <a href="https://youtu.be/ptx142-SqgE">here</a>.</p><p>Imagine a workflow where creating a video involves merely uploading a text file. This is now possible through an AI-driven content creation pipeline, which automates generating images, summarizing text, producing voiceovers, and creating dynamic videos—all while ensuring data privacy. This article explores how AI-driven workflows streamline content creation, enabling businesses to shift from text-based to video-based content effortlessly.</p><h3>The Benefits of an AI-Driven Content Workflow</h3><p>The AI workflow we’ll examine uses Amazon’s serverless architecture to transform text-based content into engaging video content.</p><figure><img alt="AI Content Creation Workflow" src="https://cdn-images-1.medium.com/max/1024/1*EQS3_Dgt1ytruF2Y60Zshg.png" /><figcaption>AI Content Creation Workflow</figcaption></figure><p>Here’s what this automated workflow offers:</p><ul><li><strong>Saves Time and Reduces Costs:</strong> Businesses can focus on other priorities while AI generates videos, images, and voiceovers quickly.</li><li><strong>Dynamic Content Generation:</strong> AI transforms static content into multimedia, enhancing audience engagement.</li><li><strong>Privacy Assurance:</strong> Using a controlled architecture ensures data privacy as it automates content creation.</li></ul><h3>Technical Architecture: The AI Content Creation Workflow</h3><p>The technical backbone of this workflow is powered by Amazon Bedrock Foundation Models and other AWS services, enabling automation with minimal human intervention.</p><figure><img alt="AI Content Creation Technical Architecture" src="https://cdn-images-1.medium.com/max/1024/1*1Hi8vkzc7NnuKQfD8rT9fw.png" /><figcaption>AI Content Creation Technical Architecture</figcaption></figure><p>Here’s a breakdown of the architecture:</p><ul><li><strong>Amazon S3 Bucket:</strong> This is the primary storage location for input files (text) and output files (images, audio, and video). Uploading a file to the S3 bucket triggers the workflow.</li><li><strong>Event Notification:</strong> When a file is uploaded to the S3 bucket, an event notification triggers an AWS Lambda function, initiating the content creation process.</li><li><strong>Lambda Functions and Step Functions:</strong> Each part of the workflow, from text extraction to video generation, is managed by Lambda functions in conjunction with Step Functions, enabling seamless transitions between tasks.</li></ul><h3>Detailed Workflow Steps</h3><ol><li><strong>Text Summarization:</strong></li></ol><p>The workflow starts by using Amazon Bedrock’s Titan Text model to summarize the uploaded text. The text extractor function receives the file, extracts its content, and sends it to the summarization model.</p><p><strong>2. Image Generation:</strong></p><p>Using Amazon Bedrock’s Titan Image Generator, the AI generates relevant images based on the summarized text, storing them back in the S3 bucket for further use.</p><p><strong>3. Voiceover Creation:</strong></p><p>Amazon Polly converts the summarized text into a natural-sounding voiceover, allowing for engaging and realistic audio narration. Different voice options are available to match various tones.</p><p><strong>4. Video Generation:</strong></p><p>AWS Elemental MediaConvert combines the images and voiceover, creating a cohesive video. The workflow assembles images with the voiceover to generate a final video, which is stored in the S3 bucket.</p><h3>Automated Role Management and Security</h3><p>Each component in this workflow, from Lambda functions to AWS Elemental MediaConvert, requires permissions to interact with AWS resources. The architecture follows the principle of least privilege, ensuring that each function has access only to the resources it needs, thus securing the process.</p><h3>Using the Final Content</h3><p>Once the workflow is complete, the S3 bucket contains the final video and images, all generated automatically. The video can be used across various digital platforms—whether for blog posts, newsletters, or marketing campaigns. Additionally, the AI-generated images can enhance your content strategy by providing visual assets without the need for external sourcing.</p><h3>Final Thoughts: Embracing AI for Digital Transformation</h3><p>This AI-driven workflow is more than a shortcut; it represents a shift towards digital transformation in content creation. With automation handling time-consuming tasks, businesses can produce dynamic and engaging video content efficiently. This innovation not only saves time but also positions businesses to thrive in a content-driven, visual-centric world.</p><p>If you’re looking to leverage AI-driven content automation, this workflow can be a game-changer. It reduces manual effort, supports privacy, and meets the demands of today’s digital audience.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=80ba383de477" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[5 Reasons to Choose Serverless Architecture for Your AI Project]]></title>
            <link>https://ima-miri.medium.com/5-reasons-to-choose-serverless-architecture-for-your-ai-project-7f009614cbaa?source=rss-5bc99e07b356------2</link>
            <guid isPermaLink="false">https://medium.com/p/7f009614cbaa</guid>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[aws-cloud]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[cost-optimization]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Ima Miri]]></dc:creator>
            <pubDate>Wed, 20 Nov 2024 00:00:56 GMT</pubDate>
            <atom:updated>2024-11-20T00:00:56.236Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oKgbc_rMzfVmYbcaoPkIFA.png" /><figcaption>5 Reasons to Choose Serverless Architecture for Your AI Project</figcaption></figure><p>Are you frustrated by the hefty monthly costs of maintaining AWS cloud infrastructure? Serverless architecture might be the solution you’re looking for. We’ve consistently chosen serverless architecture for our projects, enabling us to build efficient, cost-effective systems with flexible scaling options. Here’s why serverless has become our go-to approach, and how it’s helped our clients reduce their AWS costs dramatically.</p><p>The video tutorial for this blog is available <a href="https://youtu.be/_DtV5-00Rig">here</a>.</p><h3>What Is Serverless Architecture?</h3><p>Serverless architecture allows developers to build and deploy applications without managing the underlying infrastructure. AWS services like <strong>Lambda</strong>, <strong>Step Functions</strong>, and <strong>EventBridge</strong> handle infrastructure provisioning and scaling automatically. In serverless environments, resources are only utilized as needed, which means you pay only for what you use rather than for constant uptime.</p><h3>Benefits of Serverless Architecture on AWS</h3><ol><li><strong>Automatic Scaling and Cost Efficiency</strong></li></ol><p>Serverless services like <strong>AWS Lambda</strong> and <strong>EventBridge</strong> adjust resources based on demand, so you never pay for unused capacity. This flexibility ensures that costs align with actual usage, making serverless ideal for unpredictable workloads or applications that experience fluctuations in demand.<br>2. <strong>Simplified Machine Learning Deployments with SageMaker</strong></p><p>With <strong>Amazon SageMaker Serverless Inference</strong>, deploying machine learning models is straightforward, eliminating the need to manage infrastructure. Serverless inference pricing applies only when functions are executed, so costs stay low, and we can focus more on building accurate models than on server upkeep.</p><p><strong>3. Reduced Upfront Costs and Efficient Resource Management</strong></p><p>Serverless allows you to avoid significant upfront infrastructure expenses. For example, we recently implemented a serverless architecture for a fintech client, with estimated monthly costs under $15 for the first year. This cost-effective approach is perfect for startups and small businesses that need to keep overhead low as they scale.</p><p><strong>4. Accelerated Development and Deployment</strong></p><p>Serverless architecture speeds up development, especially for proof-of-concept projects. For instance, our team has used serverless to validate ideas quickly for various clients, including integrating AI services like <strong>Amazon Rekognition</strong> and <strong>Bedrock</strong> for intelligent automation. With serverless, deployment doesn’t get bogged down by complex infrastructure setup, allowing us to bring solutions to market faster.</p><p><strong>5. Built-In Security and Maintenance</strong></p><p>AWS manages security patches and updates for serverless environments, freeing up our team to focus on application-level security. This managed approach to security reduces operational burdens, making serverless ideal for applications requiring high security without significant overhead.</p><h3>Case Study: Serverless for Fintech</h3><p>For one fintech company, we built a serverless system that included secure data processing and integration with other AWS services, all while keeping costs below $50 per month during the initial 12 months. This architecture scaled seamlessly with user growth, demonstrating serverless’s efficiency for both low-cost startups and growing enterprises.</p><h3>Conclusion: A Smarter Way to Build on AWS</h3><p>Serverless architecture is a powerful solution for teams looking to optimize cloud spending while building scalable applications. By leveraging AWS serverless services, we can build and deploy systems quickly and affordably without sacrificing performance or security.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f009614cbaa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Intro to Big Data, Streaming Data and MapReduce]]></title>
            <link>https://ima-miri.medium.com/intro-to-big-data-streaming-data-and-mapreduce-34891cd36aa8?source=rss-5bc99e07b356------2</link>
            <guid isPermaLink="false">https://medium.com/p/34891cd36aa8</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[data-processing]]></category>
            <category><![CDATA[streaming-data-processing]]></category>
            <category><![CDATA[mapreduce]]></category>
            <dc:creator><![CDATA[Ima Miri]]></dc:creator>
            <pubDate>Sat, 16 Nov 2024 02:46:43 GMT</pubDate>
            <atom:updated>2024-11-16T02:46:43.881Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Intro to Big Data, Streaming Data and MapReduce" src="https://cdn-images-1.medium.com/max/1024/1*8idAVLWZLOcS21E4R7DYfg.png" /><figcaption>Intro to Big Data, Streaming Data and MapReduce</figcaption></figure><p>This blog is a snippet of the “Mastering Solutions Architecture” course on udemy. Check out the full course <a href="https://www.udemy.com/course/mastering-solutions-architecture/?referralCode=EA50F091C66BB075B001">here</a>.</p><p>Data is everywhere. In today’s digital age, nearly every aspect of our lives involves data in one form or another. For professionals in tech and beyond, understanding the basics of Big Data is critical. But what exactly is Big Data? Simply put, Big Data refers to extremely large and diverse sets of data — both structured and unstructured — that grow rapidly over time. As digital advancements continue to drive data creation, it’s essential for companies to grasp key Big Data concepts to remain competitive and innovative.</p><p>The video tutorial for this blog is available <a href="https://youtu.be/_0Z-eHoCby0">here</a>.</p><p>This blog provides an overview of foundational Big Data concepts, including the “Three V’s” (Volume, Velocity, and Variety) and how they define Big Data. We’ll also dive into two primary processing methods for Big Data — streaming and batch processing — and explain the MapReduce framework.</p><h3>The “Three V’s” of Big Data</h3><p>Big Data is defined by three main characteristics: <strong>Volume</strong>, <strong>Velocity</strong>, and <strong>Variety</strong> — often referred to as the “Three V’s.” These terms, coined by Gartner in 2001, are still highly relevant today.</p><ol><li><strong>Volume</strong>: Volume describes the enormous scale of data generated continuously from multiple sources. This can include social media posts, sensor data, financial transactions, and more.</li><li><strong>Velocity</strong>: Velocity is the speed at which new data is generated and processed. Today, much data is produced in near real-time and requires quick processing to unlock actionable insights.</li><li><strong>Variety</strong>: Unlike traditional datasets that are structured and fit neatly into databases, Big Data comes in many forms. It includes unstructured data (such as images, audio, and text) and semi-structured data (such as JSON files or sensor data). These diverse formats create a complex landscape for data management.</li></ol><p>Together, these characteristics describe the essence of Big Data and underscore the need for specialized tools to manage and analyze it effectively.</p><h3>The Growth of Big Data and Its Role in Modern Technology</h3><p>Digital technologies like IoT (Internet of Things), mobile devices, and AI (Artificial Intelligence) are driving the explosion of data. This exponential growth has pushed companies to adopt new tools and frameworks to collect, process, and analyze data at the required speed and scale. Big Data has become essential in machine learning, predictive modeling, and advanced analytics, empowering businesses to solve complex problems and make informed decisions.</p><h3>Big Data Processing Methods: Streaming vs. Batch Processing</h3><p>There are two primary approaches to Big Data processing: <strong>streaming</strong> and <strong>batch processing</strong>. Each method has unique advantages suited to specific scenarios.</p><ol><li><strong>Streaming Data</strong>: Streaming data processing handles data as it arrives, enabling real-time analysis and action. This low-latency approach is ideal for applications that need immediate insights, such as fraud detection, network monitoring, or personalized recommendations. Tools such as <strong>Apache Kafka</strong>, <strong>Amazon Kinesis</strong>, and <strong>Google Cloud Pub/Sub</strong> are used for data ingestion, while frameworks like <strong>Apache Flink</strong> and <strong>Apache Spark Streaming</strong> facilitate continuous data processing.</li><li><strong>Batch Processing (MapReduce)</strong>: For scenarios where large volumes of data are processed in bulk rather than in real-time, batch processing is ideal. <strong>MapReduce</strong>, a distributed computing framework, is commonly used for batch processing. It excels at handling large datasets and is effective for tasks like indexing, log analysis, and extensive data mining.</li></ol><h3>Understanding Streaming Data Architecture</h3><p>Streaming data architecture enables businesses to process data in real-time. Key components include:</p><ul><li><strong>Data Ingestion</strong>: Collecting data from various sources using tools like Apache Kafka and Google Pub/Sub.</li><li><strong>Stream Processing</strong>: Processing the data with frameworks such as Apache Flink and Apache Storm, applying transformations and aggregations.</li><li><strong>Storage</strong>: Storing processed data in NoSQL databases (like Cassandra) or time-series databases (like InfluxDB) for further analysis.</li><li><strong>Real-time Analytics</strong>: Real-time analytics tools like Elasticsearch and Kibana help monitor data flows and respond to events as they occur.</li></ul><figure><img alt="Streaming data" src="https://cdn-images-1.medium.com/max/761/1*eZQYnbOGj_y0gwUMBKWlMA.png" /><figcaption>Streaming data</figcaption></figure><p>For solutions architects, understanding streaming data architecture is essential. With tools like Apache Flink and Apache Spark Streaming, architects can create robust, scalable systems that support real-time data needs.</p><h3>Introduction to MapReduce</h3><p><strong>MapReduce</strong> is a Java-based framework within the Apache Hadoop ecosystem. It simplifies distributed computing by breaking data processing into two main steps: <strong>Map</strong> and <strong>Reduce</strong>.</p><ol><li><strong>Map Phase</strong>: Data is split into smaller chunks and processed in parallel. The map function applies specific logic to each data chunk, transforming it accordingly.</li><li><strong>Shuffle and Sort</strong>: After mapping, the data is shuffled and sorted by key, preparing it for the reduction phase.</li><li><strong>Reduce Phase</strong>: The reduce function aggregates the mapped data based on key-value pairs, producing a final output.</li></ol><figure><img alt="Map Reduce — Finding Maximum Temperature by City" src="https://cdn-images-1.medium.com/max/1024/1*qm_ywmBljpvA8oVQvzlABg.png" /><figcaption>Map Reduce — Finding Maximum Temperature by City</figcaption></figure><h4>Example: Finding Maximum Temperature by City</h4><p>Let’s look at an example. Suppose you have data across five files, each containing temperature records for different cities. Using MapReduce, each mapper processes a file and outputs the maximum temperature for each city. The reducer then aggregates these results, providing the highest temperature for each city across all files.</p><p>MapReduce’s straightforward approach and resilience make it ideal for handling vast amounts of data. Its fault tolerance and distributed nature ensure that tasks can continue even if some nodes fail.</p><h3>Conclusion</h3><p>Big Data and tools like MapReduce and streaming frameworks are reshaping how we process and analyze data. With the right tools, businesses can harness Big Data for real-time insights and improved decision-making. Solutions architects and developers must stay updated on frameworks like Apache Kafka and Hadoop to build scalable, efficient systems that meet the demands of modern applications.</p><p>As the Big Data landscape evolves, understanding these concepts becomes vital for leveraging data to drive innovation, enhance customer experiences, and achieve strategic goals.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=34891cd36aa8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Choose the Right Foundation Model for Your AI Project]]></title>
            <link>https://ima-miri.medium.com/how-to-choose-the-right-foundation-model-for-your-ai-project-0f0001e41cec?source=rss-5bc99e07b356------2</link>
            <guid isPermaLink="false">https://medium.com/p/0f0001e41cec</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[foundation-models]]></category>
            <category><![CDATA[machine-learning-ai]]></category>
            <category><![CDATA[machine-learning-tools]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Ima Miri]]></dc:creator>
            <pubDate>Wed, 23 Oct 2024 02:37:44 GMT</pubDate>
            <atom:updated>2024-10-24T10:19:43.241Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3lcW4UGJaOq_7IkN_2xxIA.png" /><figcaption>How to choose the right foundation model for your project</figcaption></figure><p>In today’s fast-paced business environment, organizations are constantly seeking innovative solutions to boost efficiency, enhance customer experiences, and drive growth. <strong>Generative AI</strong> is emerging as a powerful tool to achieve these goals by automating processes, personalizing interactions, and generating content at scale. However, choosing the right <strong>foundation model</strong> is crucial for unlocking the full potential of generative AI and ensuring the success of your AI solution.</p><p>In this post, we will explore how to select the right foundation model for your project, and I’ll introduce platforms that can help you compare and evaluate different models.</p><p>The video recording for this blog is available <a href="https://youtu.be/CMf-VGRbJrM">here</a>.</p><h3>What is a Foundation Model?</h3><p>A foundation model serves as the backbone of your AI application. It provides the core capabilities required to perform a variety of tasks, from text generation to image and video generation. These pre-trained models are designed to handle a wide range of tasks across domains and can be fine-tuned for specific use cases.</p><p><strong>Choosing the right foundation model</strong> is essential because it can dramatically impact the effectiveness and scalability of your AI solution. The model must align with your business objectives and technical requirements.</p><figure><img alt="Generative AI Experimentation and Development Lifecycle" src="https://cdn-images-1.medium.com/max/1024/1*WN6hCrfbRPTco4d__Pe55w.jpeg" /><figcaption>Generative AI Experimentation and Development Lifecycle</figcaption></figure><h3>Key Considerations for Selecting a Foundation Model</h3><p>When selecting the right foundation model for your AI solution, here are the key factors to consider:</p><h4>1. Business Use Case Alignment</h4><p>The first step in choosing a foundation model is to clearly outline your business objectives. Are you aiming to improve customer support with AI-powered chatbots? Or perhaps you want to generate personalized marketing content?</p><figure><img alt="Choosing Foundation Model based on Business usecase" src="https://cdn-images-1.medium.com/max/1024/1*EhYRTxqZ23hrSI2KfHQ1ug.png" /><figcaption>Which LLM?</figcaption></figure><p>By defining your needs, you can narrow down the models that are best suited for the task at hand. For example:</p><ul><li>If your focus is on <strong>natural language processing (NLP)</strong>, models like GPT-4 or BERT are strong candidates.</li><li>For <strong>text classification</strong> tasks, BERT is a great option.</li><li>For <strong>video generation</strong>, models like PyramidFlow may be more suitable.</li></ul><p>Each model is specialized for particular tasks, so aligning the model’s strengths with your specific use case is essential.</p><h4>2. Performance Metrics</h4><p>Once you have a shortlist of models that align with your task, the next step is to evaluate their performance based on relevant metrics. These could include:</p><ul><li><strong>Accuracy</strong>: How well does the model perform the task?</li><li><strong>Latency</strong>: How fast is the model in delivering results?</li><li><strong>Scalability</strong>: Can the model scale efficiently as your data and demands grow?</li></ul><p>Choosing a model with high accuracy but poor latency may not serve your business well if real-time processing is critical. Similarly, ensure the model can handle the scale of data you expect to process.</p><h4>3. Fine-Tuning Capabilities</h4><p>Some foundation models come pre-trained and are ready to use with minimal customization. However, others may need <strong>fine-tuning</strong> to better align with your business requirements. Fine-tuning allows you to adapt a model to your specific dataset or industry, providing better results in niche applications.</p><p>If you don’t require fine-tuning, <strong>prompt engineering</strong> — where you craft specific inputs to guide the model — might be enough to achieve the desired outcomes.</p><h4>4. Cost Considerations</h4><p>Cost is another critical factor. The pricing for foundation models can vary widely depending on the model’s size, the cloud platform you’re using, and whether you’re doing inference or training. Some models, especially large-scale ones, can be expensive to deploy and maintain.</p><p>For example, we used <strong>Amazon Bedrock</strong> and <strong>AI21 Labs</strong> for text generation in one of our projects, and the cost considerations varied depending on the complexity and scale of our use cases. Always consider both the short-term and long-term cost implications of using a particular model.</p><h3>Platforms for Foundation Model Comparison</h3><p>Choosing the right foundation model can be overwhelming, but several platforms can help you compare models and make an informed decision:</p><ul><li><strong>Hugging Face</strong>: One of the best platforms for exploring different foundation models. Hugging Face allows you to browse a wide range of pre-trained models across various categories, such as NLP, image processing, and video generation. You can also see how many people have used a model, look at performance metrics, and assess the model’s popularity within the community.</li></ul><figure><img alt="Huggingface platform" src="https://cdn-images-1.medium.com/max/1024/1*HqLW6kpU2e29VsOA7KxO4w.png" /><figcaption>Huggingface</figcaption></figure><ul><li><strong>Amazon Bedrock</strong>: This AWS service offers easy access to foundation models from leading providers, including models from AI21 Labs, Anthropic, and Stability AI. Bedrock provides managed services, so you don’t have to worry about underlying infrastructure, allowing you to focus on building your AI solution.</li></ul><p>These platforms make it easier to evaluate and compare models based on factors such as cost, performance, and use case suitability.</p><h3>Conclusion</h3><p>Selecting the right foundation model is a crucial decision that can influence the success of your AI solution. By considering your business objectives, evaluating performance metrics, and understanding the costs and fine-tuning capabilities of each model, you can choose the foundation model that best fits your needs.</p><p>With platforms like Hugging Face and Amazon Bedrock at your disposal, you can streamline the process of comparing models and ensure you make the best choice for your project.</p><p>Ready to take the next step in your AI journey? Explore your options, choose the right model, and start building solutions that drive real business impact.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0f0001e41cec" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an AI Stylist with Amazon Bedrock]]></title>
            <link>https://ima-miri.medium.com/building-an-ai-stylist-with-amazon-bedrock-a744e9419c7a?source=rss-5bc99e07b356------2</link>
            <guid isPermaLink="false">https://medium.com/p/a744e9419c7a</guid>
            <category><![CDATA[digital-fashion]]></category>
            <category><![CDATA[amazon-bedrock]]></category>
            <category><![CDATA[generative-ai-solution]]></category>
            <category><![CDATA[aws-cloud]]></category>
            <category><![CDATA[ai-fashion-stylist]]></category>
            <dc:creator><![CDATA[Ima Miri]]></dc:creator>
            <pubDate>Sun, 13 Oct 2024 10:01:22 GMT</pubDate>
            <atom:updated>2024-11-10T02:33:03.494Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gvxgZFA-qGvDbh9E9W53Wg.png" /><figcaption>Build an AI Stylist with Amazon Bedrock</figcaption></figure><p>In this article, we explore how generative AI can transform the fashion industry by building an AI stylist using Amazon Bedrock. This AI stylist allows users to get personalized style suggestions based on their gender and the occasion they’re attending. Before diving into the code, let’s first understand what Amazon Bedrock is and how it simplifies the process of building generative AI solutions.</p><p>The video tutorial of this blog is available <a href="https://youtu.be/5zNEIBHcNnc">here</a>.</p><h3>What is Amazon Bedrock?</h3><p>Amazon Bedrock is a powerful platform that simplifies the integration of generative AI models into applications. It provides access to foundation models from various providers through a unified API, making it easy to incorporate state-of-the-art AI models without dealing with the complexities of model management and scaling. Bedrock supports a range of tasks, including text generation, image generation, and more.</p><p>For this AI stylist, we used two foundation models available through Amazon Bedrock:</p><ol><li><strong>Anthropic Claude-v2:1</strong>: A large language model (LLM) specializing in text-based tasks, such as generating conversational recommendations.</li><li><strong>Stability Stable Diffusion XL-v1</strong>: A model designed for text-to-image tasks, which excels at generating high-quality images from text descriptions.</li></ol><h3>How the AI Stylist Works</h3><p>The AI stylist uses a combination of text and image generation models to create personalized fashion recommendations.</p><p>Here’s how each model contributes:</p><h4>Step 1: Text-Based Recommendations with Anthropic Claude-v2:1</h4><p>The first part of the AI stylist involves generating a text-based outfit suggestion using the Anthropic Claude-v2:1 model. This LLM is ideal for handling conversational inputs and generating detailed responses.</p><ul><li><strong>Input Prompt</strong>: The AI receives a prompt asking for outfit suggestions based on the user’s gender and the occasion they are attending. For example:</li><li><em>“Human: I am attending a wedding, and I am female. What outfit would you suggest?”</em></li><li><strong>Model Call</strong>: This prompt is sent to the Claude model, which generates a detailed response. The response might suggest specific clothing items, such as a “floral dress” or “comfortable shoes.”</li><li><strong>Text Processing</strong>: The response is then split into individual parts, such as sentences or specific clothing recommendations, which will later be used to generate corresponding images.</li></ul><h4>Step 2: Image Generation with Stability Stable Diffusion XL-v1</h4><p>Once the text suggestions are generated, the Stability Stable Diffusion XL-v1 model is used to create images based on each part of the text. This model is designed to transform text descriptions into detailed, high-quality images, making it ideal for visualizing outfit ideas.</p><ul><li><strong>Input Prompt</strong>: For each part of the text suggestion (e.g., “a floral dress”), a new prompt is constructed for the image generation model. For example:</li><li><em>“An outfit that matches the following description: a floral dress.”</em></li><li><strong>Model Call</strong>: This prompt is sent to the Stable Diffusion model, which generates an image that visually represents the clothing item. Parameters like cfg_scale, steps, and seed are used to control the quality and diversity of the generated images.</li><li><strong>Image Handling</strong>: The response from the model includes a base64-encoded image. This image is decoded and displayed, giving users a visual representation of the outfit suggestion.</li></ul><p>The function:</p><pre><br>  # Call Claude model for text-based recommendation<br>  suggestion_response = bedrock.invoke_model(<br>      modelId=&#39;anthropic.claude-v2:1&#39;,<br>      contentType=&#39;application/json&#39;,<br>      body=json.dumps(payload).encode(&#39;utf-8&#39;)<br>  )<br><br>  # Extract the outfit suggestion text<br>  suggestion = suggestion_response[&#39;body&#39;].read().decode(&#39;utf-8&#39;)</pre><pre># Call Stable Diffusion for each image generation<br>image_response = bedrock.invoke_model(<br>    modelId=&#39;stability.stable-diffusion-xl-v1&#39;,<br>    contentType=&#39;application/json&#39;,<br>    body=json.dumps(stable_diffusion_payload)<br>)<br><br>image_result = json.loads(image_response[&#39;body&#39;].read())</pre><h3>Putting It All Together</h3><p>The final output of the AI stylist is a combination of text-based recommendations and matching images. The user gets a complete style suggestion, including both a written description and a visual representation of the outfit, tailored to their specific occasion and gender.</p><p>For example, if you ask for an outfit suggestion for a wedding, you might receive a text recommendation like “a floral dress paired with comfortable shoes” along with corresponding images of each item.</p><h3>Example Code</h3><p>The full code for the AI stylist uses Amazon Bedrock to interact with both the Anthropic Claude-v2:1 model for text suggestions and the Stability Stable Diffusion XL-v1 model for image generation. Here’s how the process is structured:</p><figure><img alt="Technical Architecture" src="https://cdn-images-1.medium.com/max/400/1*1caikeEL-CZrtRtsSFpaUA.png" /><figcaption>Technical Architecture</figcaption></figure><ol><li><strong>User Input</strong>: The user provides the occasion and their gender.</li><li><strong>Text Suggestion</strong>: The code sends a prompt to the Claude model to generate a detailed fashion recommendation.</li><li><strong>Image Generation</strong>: The recommendation is split into parts, and each part is used to generate a corresponding image with Stable Diffusion.</li><li><strong>Display</strong>: The text suggestions and images are displayed to the user.</li></ol><h3>Conclusion</h3><p>Amazon Bedrock makes it incredibly easy to integrate generative AI models into applications, as demonstrated with this AI stylist. By leveraging foundation models like Anthropic Claude-v2:1 and Stability Stable Diffusion XL-v1, we can create a seamless experience that combines personalized text recommendations with high-quality images. This approach has exciting potential not just in fashion but in any industry where personalization and creativity are key.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a744e9419c7a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an AI Agent from Scratch]]></title>
            <link>https://ima-miri.medium.com/agent-building-an-ai-agent-from-scratch-16c22e0e22ec?source=rss-5bc99e07b356------2</link>
            <guid isPermaLink="false">https://medium.com/p/16c22e0e22ec</guid>
            <category><![CDATA[generative-ai-solution]]></category>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Ima Miri]]></dc:creator>
            <pubDate>Mon, 07 Oct 2024 23:53:18 GMT</pubDate>
            <atom:updated>2024-10-13T09:58:30.106Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ym2DTzqJZ3A4BcOMLhlBrQ.png" /></figure><p>In this blog, we’ll dive into <strong>AI agents</strong> — smart assistants that understand and act based on conversations, gaining popularity in 2024 due to models like GPT-4.</p><p>The video tutorial of this article is available <a href="https://youtu.be/ZtlcCrJ9-yE?si=N7gVcSRNRzLaCd5f">here</a>.</p><h4>What is an AI Agent?</h4><p>An AI agent is like a knowledgeable friend that listens to your questions and takes action based on them. With advanced models, these agents offer natural, helpful responses, revolutionizing industries by handling repetitive tasks, increasing efficiency, and seamlessly integrating with modern platforms.</p><h4>Why AI Agents?</h4><p>In 2024, AI agents are more accessible and customizable. Businesses can now easily build solutions that meet specific needs, such as automating customer service, managing lead generation, or providing instant information. These capabilities make AI agents indispensable for improving business processes and engagement.</p><h4>Building an AI Agent</h4><p>In this tutorial, we built an AI agent using <strong>Python</strong>, the <strong>OpenAI API</strong>, and a structured loop of <strong>Thought, Action, Pause, Response</strong>. Here’s the simplified flow:</p><ul><li><strong>Thought</strong>: The agent processes the query.</li><li><strong>Action</strong>: It executes an action, like finding a city’s population.</li><li><strong>PAUSE</strong>: The action is completed.</li><li><strong>Response</strong>: The agent provides the final result.</li></ul><p>We also used a simple Python script, which helps break tasks into manageable steps, integrating external data sources.</p><h4>The Code</h4><p>Here’s an overview of the code we used:</p><ul><li>We initialized the OpenAI API and created an agent class.</li></ul><pre>prompt = &quot;&quot;&quot;<br>You will follow a loop of Thought, Action, PAUSE and Response. The goal is to get an answer at the end.<br>   Thought: Think about how to answer the question.<br>   Action: Use one of the available actions to get information - then return PAUSE.<br>   Response: After the action is complete, respond with the result.<br><br>Available actions:<br>   lookup_capital: Finds the capital city of a country. Example: lookup_capital: France, Result: Returns the capital city (e.g., &quot;Paris&quot;).<br>   find_population: Looks up the population of a country or city. Example: find_population: Tokyo, Result: Returns the population (e.g., &quot;14 million&quot;).<br><br>Example conversation:<br><br>Question: What is the population of capital city of France?<br><br>   Thought: I should find the capital city of France.<br>   Action: lookup_capital: Paris<br>   PAUSE<br><br>Then you&#39;ll find the population of the capital city:<br><br>Response: The population of Paris is approximately 2.1 million people.<br><br>Finally, respond:<br><br>Answer: The population of Paris is approximately 2.1 million people.<br>&quot;&quot;&quot;.strip()</pre><p>2. The agent uses predefined functions like <strong>lookup_capital</strong> (finding a city) and <strong>find_population</strong> (returning population data).</p><pre>def lookup_capital(country):<br>  capitals = {<br>  &quot;France&quot;: &quot;Paris&quot;,<br>  &quot;Germany&quot;: &quot;Berlin&quot;,<br>  &quot;Japan&quot;: &quot;Tokyo&quot;,<br>  &quot;USA&quot;: &quot;Washington, D.C.&quot;,<br>  &quot;Canada&quot;: &quot;Ottawa&quot;,<br>     }<br>  return capitals.get(country, &quot;Capital not found for {}&quot;.format(country))<br><br>def find_population(city):<br>  populations = {<br>  &quot;Tokyo&quot;: &quot;14 million&quot;,<br>  &quot;New York&quot;: &quot;8.3 million&quot;,<br>  &quot;London&quot;: &quot;9 million&quot;,<br>  &quot;Paris&quot;: &quot;2.1 million&quot;,<br>  &quot;Sydney&quot;: &quot;5.3 million&quot;,<br>     }<br>  return populations.get(city, &quot;Population data not found for {}&quot;.format(city))<br>  <br>known_actions = {<br>  &quot;find_population&quot;: find_population,<br>  &quot;lookup_capital&quot;: lookup_capital<br>}<br><br>action_re = re.compile(&#39;^Action: (\w+): (.*)$&#39;)  </pre><p>3. The conversation flows in a loop until the agent reaches a conclusion or stops when no further questions are posed.</p><p><strong>Example:</strong></p><p>You ask, “What’s the population of Tokyo?” The agent first identifies the capital city and then returns, “The population of Tokyo is approximately 14 million.” It continues until the user exits.</p><p><strong>Sample output</strong></p><p><em>Welcome to the Custom AI agent! Type ‘exit’ to end the conversation.</em></p><p><em>You: What’s the population of the capital city of Japan?</em></p><p><em>AI: Thought: I need to find the capital city of Japan first.</em></p><p><em>Action: lookup_capital: Japan</em></p><p><em>PAUSE AI: Response: The capital city of Japan is Tokyo.</em></p><p><em>Thought: Now I need to find the population of Tokyo.</em></p><p><em>Action: find_population: Tokyo</em></p><p><em>PAUSE</em></p><p><em>AI: Response: The population of Tokyo is approximately 14 million people.</em></p><p><em>Answer: The population of Tokyo is approximately 14 million people.</em></p><p><em>If you have any more questions or need further assistance, feel free to ask.</em></p><p><em>You: What’s the population of capital city of France?</em></p><p><em>AI: Thought: I need to find the capital city of France first.</em></p><p><em>Action: lookup_capital: France</em></p><p><em>PAUSE</em></p><p><em>AI: Response: The capital city of France is Paris.</em></p><p><em>Thought: Now I need to find the population of Paris.</em></p><p><em>Action: find_population: Paris</em></p><p><em>PAUSE</em></p><p><em>AI: Response: The population of Paris is approximately 2.1 million people.</em></p><p><em>Answer: The population of Paris is approximately 2.1 million people.</em></p><p><em>If you have any more questions or need further assistance, feel free to ask.</em></p><p><em>You: exit</em></p><p><em>Goodbye!</em></p><h4>Benefits and Future Potential</h4><p>As AI agents evolve, they will become even more advanced and useful in various sectors, automating complex tasks, improving customer service, and driving business innovation. By leveraging platforms like Python and OpenAI, creating your AI agent is now more accessible than ever.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=16c22e0e22ec" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build a Digital Wardrobe with Amazon Rekognition]]></title>
            <link>https://ima-miri.medium.com/build-a-digital-wardrobe-with-amazon-rekognition-6af64509a3c9?source=rss-5bc99e07b356------2</link>
            <guid isPermaLink="false">https://medium.com/p/6af64509a3c9</guid>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[system-design-project]]></category>
            <category><![CDATA[system-design-concepts]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Ima Miri]]></dc:creator>
            <pubDate>Sun, 06 Oct 2024 00:39:21 GMT</pubDate>
            <atom:updated>2024-11-10T02:31:17.417Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1fJE-M2eb_u5seM7H8l8Eg.png" /><figcaption>Digital Wardrobe</figcaption></figure><p>In this blog, we’ll walk through how to build a <strong>digital wardrobe</strong> where users can upload images of their clothing, allowing for virtual storage and enhanced shopping experiences. This feature boosts customer engagement and retention by offering personalized outfit suggestions and styling ideas based on new trends.</p><p>The video tutorial of this article is available <a href="https://youtu.be/e3UcgN6y30s?si=BkrA05HzGzyu8cIA">here</a>.</p><h3>Components Used:</h3><p>Here are the components that we used to build the digital wardrobe.</p><ul><li><strong>Amazon S3</strong> for storing images.</li><li><strong>EventBridge</strong> to trigger Lambda functions when new images are uploaded.</li><li><strong>AWS Lambda</strong> for the core logic.</li><li><strong>Amazon Rekognition</strong> to detect and label clothing items in images.</li><li><strong>DynamoDB</strong> to store image metadata like detected labels.</li><li><strong>IAM Roles</strong> to manage permissions.</li><li><strong>CloudWatch</strong> for monitoring.</li></ul><h3>How it Works:</h3><p>The following steps describe how this functionality works.</p><ol><li><strong>User Image Upload</strong>: Users upload their clothing images via a mobile or web app to an S3 bucket, where images are stored.</li><li><strong>Triggering Event</strong>: When an image is uploaded, EventBridge triggers a Lambda function to analyze it.</li><li><strong>Label Detection</strong>: The Lambda function reads the image from S3 and sends it to <strong>Amazon Rekognition</strong>. Rekognition then detects labels (e.g., dress, shoes) with confidence scores.</li><li><strong>Data Storage</strong>: Labels with confidence scores above 90% are saved in <strong>DynamoDB</strong>. The image metadata is now accessible for styling recommendations and personalized notifications.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*2RZPc8o1WbvyLZLXzpyBDA.jpeg" /><figcaption>Technical Architecture</figcaption></figure><h3>Step-by-Step AWS Setup:</h3><ol><li><strong>Create S3 Bucket</strong>: Start by creating an S3 bucket, e.g., “digital-wardrobe-bucket” to store images.</li><li><strong>Create DynamoDB Table</strong>: Set up a DynamoDB table (e.g., “virtual-wardrobe”) to store image labels and metadata.</li><li><strong>Set Up Lambda Function</strong>: Develop the Lambda function that integrates <strong>Amazon Rekognition</strong>. The core logic includes reading images from S3, analyzing them, and storing detected labels in DynamoDB.</li><li><strong>Configure IAM Roles</strong>: Assign appropriate permissions for S3, DynamoDB, and Rekognition. Follow AWS best practices by using least privilege principles.</li><li><strong>Test the Workflow</strong>: Upload images, trigger the Lambda function, and review the results in DynamoDB.</li></ol><h3>AWS Code Overview:</h3><p>Here’s a high-level look at the core functionality:</p><ul><li><strong>Get Image Labels</strong>: Calls the Rekognition API to detect clothing items. It filters labels with confidence scores greater than 90%.</li><li><strong>Store Clothing Data</strong>: Saves the detected labels into DynamoDB.</li><li><strong>Main Lambda Handler</strong>: Orchestrates the flow, processing images from S3, detecting clothing labels, and storing the results.</li></ul><pre><br># Detect labels in the image<br>labels = rekognition.detect_labels(Image={&#39;Bytes&#39;: image_bytes}, MaxLabels=3)<br><br># Extract the top labels and their confidence levels<br>top_labels = []<br>for l in labels[&#39;Labels&#39;]:<br>    if l[&#39;Confidence&#39;] &gt; 90:<br>        top_labels.append(l[&#39;Name&#39;])<br>        <br>return top_labels<br>    </pre><h3>Personalization and Customer Engagement:</h3><p>Once set up, this digital wardrobe allows users to store, organize, and receive styling suggestions for their clothes. E-commerce platforms can also leverage this to offer personalized notifications for new arrivals that match a user’s wardrobe.</p><p><strong>Use Cases</strong>:</p><ul><li><strong>E-Commerce Fashion Sites</strong>: Personalize the shopping experience.</li><li><strong>Retail</strong>: Offer virtual wardrobes that integrate with physical stores.</li><li><strong>Styling Apps</strong>: Suggest outfits based on current trends and user preferences.</li></ul><h3>Understanding Amazon Rekognition:</h3><p>Amazon Rekognition is a powerful machine learning service that identifies objects, scenes, and activities in images. Here’s how it works:</p><ul><li><strong>Image Processing</strong>: The uploaded image is analyzed using pre-trained machine learning models to detect patterns and recognize objects.</li><li><strong>Labeling</strong>: Rekognition assigns labels to the detected objects, such as “shirt”, “dress”, or “handbag”, each with a confidence score. The higher the score, the more confident the system is about its accuracy. In our case, we filter for confidence scores above 90% to ensure precise labeling.</li><li><strong>Scenes and Objects</strong>: Apart from clothing items, Rekognition can also detect broader scenes like “outdoors” or “office environment”, making it versatile for various use cases.</li></ul><p>Amazon Rekognition can be enhanced with <strong>custom models</strong>, allowing you to train domain-specific recognition for items not covered in the general label set. For example, in healthcare applications, custom models can be trained to recognize specific objects like medical instruments.</p><h3>Conclusion:</h3><p>By leveraging AWS services like Lambda, Rekognition, and DynamoDB, you can build a scalable digital wardrobe that enhances user engagement through personalized experiences. This solution is perfect for e-commerce fashion sites looking to innovate and retain customers.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6af64509a3c9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to Prompt Engineering]]></title>
            <link>https://ima-miri.medium.com/introduction-to-prompt-engineering-0cb0824c78d2?source=rss-5bc99e07b356------2</link>
            <guid isPermaLink="false">https://medium.com/p/0cb0824c78d2</guid>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[generative-ai]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Ima Miri]]></dc:creator>
            <pubDate>Wed, 02 Oct 2024 11:51:35 GMT</pubDate>
            <atom:updated>2024-10-07T23:43:20.675Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Introduction to Prompt Engineering" src="https://cdn-images-1.medium.com/max/1024/1*OSBcJu41ECwquIaysZD7TA.png" /></figure><p>Prompt engineering refers to the method of interacting with pre-trained language models, like OpenAI’s GPT, by crafting precise inputs (prompts). These inputs guide the model in generating useful responses. In this article, we’ll create a basic chatbot by sending text prompts to the OpenAI API and letting the model return natural language responses in a conversational style.</p><p>The video tutorial of this article can be found <a href="https://youtu.be/tFaTA_vwOVo?si=qzNjPK5F0qzKNyZ-">here</a>.</p><h3>Prerequisites</h3><p>For initial setup, you need the following:</p><ol><li><strong>OpenAI API account</strong>: Sign up at <a href="https://platform.openai.com/">platform.openai.com</a> to create an API key.</li><li><strong>Python and Jupyter Notebook</strong>: If you haven’t already installed Python and Jupyter Notebook, check out our earlier <a href="https://youtu.be/bbH0GtBvyss?si=ZUHHXTr_ZfCjYGM3">video</a> for a step-by-step installation guide.</li></ol><h3>Setting Up the Environment</h3><ol><li><strong>Check Python Installation</strong>: Open your terminal and type:</li></ol><pre>python3 - version</pre><p>Ensure you have Python installed. If not, use brew install python on Mac or download it from <a href="https://www.python.org/">python.org</a>.</p><p><strong>2. Install Jupyter Notebook</strong>: Once Python is installed, run Jupyter Notebook by typing:</p><pre>jupyter notebook</pre><p>You’ll be able to interactively code and execute commands in real time, which is why Jupyter is perfect for prototyping prompt engineering examples.</p><h3>Implementing OpenAI API in Python</h3><p>Now that your environment is set up, let’s install the OpenAI API package. In your terminal, run:</p><pre>pip install openai</pre><p>Once installed, import it in your Jupyter notebook.</p><h3>Creating the API Connection</h3><p>Now, let’s write a function to interact with the OpenAI API. Here’s how to proceed:</p><ol><li><strong>Obtain Your API Key</strong>:</li></ol><ul><li>Go to <a href="https://platform.openai.com/">platform.openai.com</a>.</li><li>In the <strong>Profile</strong> section, find <strong>API Keys</strong>.</li><li>Generate a new key, copy it, and store it securely, as you can only view it once.</li></ul><p><strong>2. Define the Function to Interact with OpenAI</strong>: Here’s a sample Python code to define a function for sending prompts and receiving responses from the chatbot:</p><pre>import openai<br>def get_response_from_chatbot(prompt):<br>  openai.api_key = &quot;your_api_key_here&quot;<br>  response = openai.ChatCompletion.create(<br>  model=&quot;gpt-3.5-turbo&quot;, # Choose the latest available model<br>  messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}]<br>  )<br>  return response[&#39;choices&#39;][0][&#39;message&#39;][&#39;content&#39;]</pre><h3>Testing the Chatbot</h3><p>You can test the chatbot by passing a prompt like:</p><pre>prompt = &quot;You are an amazing bot!&quot;<br>response = get_response_from_chatbot(prompt)<br>print(response)</pre><p>Running this will generate a response directly in the notebook, making it an excellent way to prototype and refine prompts interactively.</p><h3>Example: Chatbot Prompt</h3><p>Let’s try another example. We’ll ask the chatbot a simple question: <strong>“How do I make a cup of coffee?”</strong> and see how it responds:</p><pre>prompt = &quot;How do I make a cup of coffee?&quot;<br>response = get_response_from_chatbot(prompt)<br>print(response)</pre><p>The model will return a step-by-step guide to making coffee, showcasing its ability to understand and provide detailed answers based on simple prompts.</p><h3>Avoid Exposing Your API Key</h3><p>One important note: always keep your API key secure. Do not share it publicly, as anyone with access can use your credits, and you may end up paying for unintended usage.</p><h3>Conclusion</h3><p>Building an AI-powered chatbot using prompt engineering is simpler than you might think! With just a few lines of code, you can create a responsive and conversational agent to enhance your workflows or systems.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0cb0824c78d2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Real-time Image Processing Architecture using AWS]]></title>
            <link>https://ima-miri.medium.com/real-time-image-processing-architecture-using-aws-71c20f04f05f?source=rss-5bc99e07b356------2</link>
            <guid isPermaLink="false">https://medium.com/p/71c20f04f05f</guid>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[system-design-concepts]]></category>
            <category><![CDATA[aws-cloud]]></category>
            <dc:creator><![CDATA[Ima Miri]]></dc:creator>
            <pubDate>Thu, 08 Feb 2024 23:13:27 GMT</pubDate>
            <atom:updated>2024-10-07T23:42:45.772Z</atom:updated>
            <content:encoded><![CDATA[<p>In<strong> </strong>this example, a social media marketing platform allow users to upload multiple images for their social media campaign. The system automatically analyzes the images to store them, performs tasks like object detection, image classification to categorize images, and generating captions for the marketing campaigns. Here are different aspects of system design architecture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*01F0B7k6YRXuhPgXsmMVEA.png" /><figcaption>Technical Architecture for real time image processing</figcaption></figure><p><strong>1. Frontend:</strong></p><ul><li>Users upload images through a web application hosted on Amazon Simple Storage Service (S3).</li><li>Amazon CloudFront delivers the web application content globally with low latency.</li></ul><p><strong>2. Image Processing Pipeline:</strong></p><ul><li>Upon upload, S3 triggers an Amazon Lambda function that performs initial processing like resizing and format conversion.</li><li>The function publishes the processed image to an Amazon Simple Queue Service (SQS) queue.</li><li>A fleet of Amazon ECS containers, managed by Auto Scaling, continuously listen to the SQS queue.</li><li>Upon receiving an image, each container uses a pre-trained machine learning model (hosted on Amazon SageMaker) to analyze the image.</li><li>The analysis results are stored in Amazon DynamoDB for high-performance retrieval.</li><li>Amazon CloudWatch monitors the entire pipeline for performance and errors.</li></ul><p><strong>3. Result Delivery:</strong></p><ul><li>Another Lambda function reads results from DynamoDB and generates captions for marketing campaign using Amazon Comprehend.</li><li>The function stores the captions in S3 and also pushes them to the user’s web application via Amazon SNS for real-time notification.</li></ul><p><strong>Benefits of the above design:</strong></p><ul><li><strong>Scalability:</strong> The serverless architecture automatically scales based on workload, handling traffic spikes efficiently.</li><li><strong>Cost-Effectiveness:</strong> You only pay for resources used, making it cost-efficient for varying workloads.</li><li><strong>High Availability:</strong> Data redundancy and distributed processing ensure availability even if individual components fail.</li><li><strong>Real-time Processing:</strong> SQS and Lambda enable near-instantaneous image analysis and notification.</li></ul><p>The key goal here is to design a system that is <strong>secure, scalable, cost-effective, and meets specific functional requirements</strong>. By following best practices and leveraging the diverse capabilities of AWS services, you can build robust and efficient systems for your needs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=71c20f04f05f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>