<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vinayak Arannil on Medium]]></title>
        <description><![CDATA[Stories by Vinayak Arannil on Medium]]></description>
        <link>https://medium.com/@vinayak-arannil?source=rss-4f310ccec56e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*maOO1NvOtFKB0yqnO120CA.jpeg</url>
            <title>Stories by Vinayak Arannil on Medium</title>
            <link>https://medium.com/@vinayak-arannil?source=rss-4f310ccec56e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 19:16:15 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vinayak-arannil/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[An Introduction to Asynchronous Programming: The Key to Efficient Generative AI Applications]]></title>
            <link>https://vinayak-arannil.medium.com/an-introduction-to-asynchronous-programming-the-key-to-efficient-generative-ai-applications-871e76d1d02c?source=rss-4f310ccec56e------2</link>
            <guid isPermaLink="false">https://medium.com/p/871e76d1d02c</guid>
            <category><![CDATA[parallel-computing]]></category>
            <category><![CDATA[asynchronous-programming]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[asyncawait]]></category>
            <category><![CDATA[amazon-bedrock]]></category>
            <dc:creator><![CDATA[Vinayak Arannil]]></dc:creator>
            <pubDate>Fri, 11 Oct 2024 21:02:43 GMT</pubDate>
            <atom:updated>2024-10-23T17:20:12.885Z</atom:updated>
            <content:encoded><![CDATA[<h3>An Introduction to Asynchronous Programming in Python: The Key to Efficient Generative AI Applications</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gbxx44RO1CjmfrA_BU5KiQ.jpeg" /></figure><p>In the rapidly evolving landscape of generative AI, the most powerful and sophisticated language models are typically hosted and maintained by their providers. Companies like OpenAI and Anthropic have made their state-of-the-art models accessible through API services, enabling developers to leverage these AI capabilities without the need for extensive infrastructure or expertise in model deployment.</p><p><a href="https://aws.amazon.com/bedrock/">Amazon Bedrock</a> is an excellent example of such a service, providing a unified API interface to access a variety of large language models (LLMs) from different providers. This allows developers to easily experiment with and integrate different models into their applications without managing multiple API integrations.</p><h3>The Need for Asynchronous Processing in AI Applications</h3><p>As these models become more sophisticated and widely used, the ability to efficiently manage multiple API calls becomes crucial. When building applications that integrate with AI models like GPT-4, Claude, or those available through Amazon Bedrock, developers often need to make multiple API calls in quick succession.</p><p>These calls can be time-consuming, especially when dealing with complex prompts, large amounts of data, or when chaining multiple AI operations together. In a synchronous environment, each API call would block the execution of the next, leading to slow and unresponsive applications. This is particularly problematic in scenarios where an application needs to process multiple user requests simultaneously or perform a series of AI-powered operations in parallel.</p><p>For example:</p><ul><li>A customer service bot might need to handle several customer queries concurrently.</li><li>A synthetic data generation solution that generates large amount of synthetic data using LLMs</li></ul><p>In these cases, waiting for each API call to complete before starting the next would result in unacceptable latency and a poor user experience.</p><p>This is where asynchronous programming shines, offering a way to build responsive and scalable applications that leverage the power of AI without compromising performance. Asynchronous programming allows these API calls to be made concurrently, significantly improving the overall performance and user experience of your application. In this blog, we explain how to implement this in practice. Furthermore, we’ll explore how to combine asynchronous programming with multithreading using Python’s ThreadPoolExecutor. This powerful combination allows for even greater parallelism, enabling applications to efficiently manage a large number of concurrent API calls while utilizing multiple CPU cores.</p><h3>Basics of Asynchronous Processing</h3><p>Here’s a simple example of how asynchronous operation works compared to sequential.</p><pre>import asyncio<br>import time<br><br>def fetch_data_sync(delay):<br>    print(f&quot;Starting sync fetch for delay {delay}&quot;)<br>    time.sleep(delay)  # Simulating a blocking I/O operation<br>    print(f&quot;Finished sync fetch for delay {delay}&quot;)<br>    return f&quot;Data for delay {delay}&quot;<br><br>async def fetch_data_async(delay):<br>    print(f&quot;Starting async fetch for delay {delay}&quot;)<br>    await asyncio.sleep(delay)  # Simulating a non-blocking I/O operation<br>    print(f&quot;Finished async fetch for delay {delay}&quot;)<br>    return f&quot;Data for delay {delay}&quot;<br><br>def sequential_fetch():<br>    print(&quot;Starting sequential fetch&quot;)<br>    start = time.time()<br>    <br>    result1 = fetch_data_sync(1)<br>    result2 = fetch_data_sync(2)<br>    result3 = fetch_data_sync(3)<br>    <br>    end = time.time()<br>    print(f&quot;Sequential fetch completed in {end - start:.2f} seconds&quot;)<br>    return [result1, result2, result3]<br><br>async def concurrent_fetch():<br>    print(&quot;Starting concurrent fetch&quot;)<br>    start = time.time()<br>    <br>    tasks = [<br>        asyncio.create_task(fetch_data_async(1)),<br>        asyncio.create_task(fetch_data_async(2)),<br>        asyncio.create_task(fetch_data_async(3))<br>    ]<br>    results = await asyncio.gather(*tasks)<br>    <br>    end = time.time()<br>    print(f&quot;Concurrent fetch completed in {end - start:.2f} seconds&quot;)<br>    return results<br><br>async def main():<br>    seq_results = sequential_fetch()<br>    print(&quot;Sequential results:&quot;, seq_results)<br>    <br>    print(&quot;\n&quot; + &quot;=&quot;*50 + &quot;\n&quot;)<br>    <br>    conc_results = await concurrent_fetch()<br>    print(&quot;Concurrent results:&quot;, conc_results)<br><br>asyncio.run(main())</pre><p>If you execute this code, it will print</p><pre>Starting sequential fetch<br>Starting sync fetch for delay 1<br>Finished sync fetch for delay 1<br>Starting sync fetch for delay 2<br>Finished sync fetch for delay 2<br>Starting sync fetch for delay 3<br>Finished sync fetch for delay 3<br>Sequential fetch completed in 6.00 seconds<br>Sequential results: [&#39;Data for delay 1&#39;, &#39;Data for delay 2&#39;, &#39;Data for delay 3&#39;]<br><br>==================================================<br>Starting concurrent fetch<br>Starting async fetch for delay 1<br>Starting async fetch for delay 2<br>Starting async fetch for delay 3<br>Finished async fetch for delay 1<br>Finished async fetch for delay 2<br>Finished async fetch for delay 3<br>Concurrent fetch completed in 3.00 seconds<br>Concurrent results: [&#39;Data for delay 1&#39;, &#39;Data for delay 2&#39;, &#39;Data for delay 3&#39;]</pre><p>Now, let’s break down how this works:</p><ol><li>We define two functions to simulate I/O operations:</li><li>fetch_data_sync: A synchronous function using time.sleep().</li><li>fetch_data_async: An asynchronous function using asyncio.sleep().</li><li>The sequential_fetch function calls fetch_data_sync three times in sequence. Each call blocks until it completes before moving to the next. Total time is the sum of all delays (1 + 2 + 3 = 6 seconds).</li><li>The concurrent_fetch function uses asyncio.create_task() to schedule three fetch_data_async calls. asyncio.gather(*tasks) waits for all tasks to complete. Tasks run concurrently, so total time is about the longest delay (3 seconds).</li><li>In the main function we call sequential_fetch() directly as it&#39;s a regular function. We use await concurrent_fetch() as it&#39;s an asynchronous function.</li><li>Finally, asyncio.run(main()) at the end creates an event loop and runs the main() coroutine until it&#39;s complete. The event loop manages the execution of all asynchronous tasks.</li></ol><p>The async keyword is used to define a coroutine function, which can be paused and resumed, allowing other code to run in the meantime. A coroutine is a special type of function that can be suspended and resumed during its execution. It&#39;s the building block of asynchronous programming in Python, enabling concurrent execution of multiple tasks without using multiple threads. The await keyword is used inside an async function to pause execution until an asynchronous operation completes, yielding control back to the event loop without blocking the entire program. To use them, define functions with async def for asynchronous operations, and use await before calling these functions or other async operations within an async context. To run a coroutine from synchronous code you need to use the event loop (e.g. asyncio.run(some_coroutine()) )</p><h3>Applying Asynchronous Processing to Amazon Bedrock</h3><p>Let’s look at how we can apply these concepts to make asynchronous calls to an Amazon Bedrock model. We’ll use the boto3 library to interact with AWS services:</p><pre>import asyncio<br>import boto3<br>import json<br><br>bedrock_runtime =  boto3.client(service_name=&quot;bedrock-runtime&quot;, region_name=&#39;us-east-1&#39;)<br><br>async def call_bedrock_model(prompt, model_id=&quot;anthropic.claude-3-haiku-20240307-v1:0&quot;):<br>    body = json.dumps(<br>    {<br>        &quot;messages&quot;: [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: [{&quot;type&quot;: &quot;text&quot;, &quot;text&quot;: prompt}]}],<br>        &quot;anthropic_version&quot;: &quot;bedrock-2023-05-31&quot;,<br>        &quot;max_tokens&quot;: 8000,<br>        &quot;temperature&quot;: 0.7,<br>        &quot;top_p&quot;: 0.95,<br>        &quot;top_k&quot;: 250,<br>    }<br>    )<br>    modelid = model_id<br>    accept = &quot;application/json&quot;<br>    contenttype = &quot;application/json&quot;<br>    response = bedrock_runtime.invoke_model(<br>    body=body, modelId=modelid, accept=accept, contentType=contenttype<br>    )<br>    return json.loads(response.get(&quot;body&quot;).read()).get(&quot;content&quot;)[0][&quot;text&quot;]<br><br>async def process_prompts(prompts):<br>    tasks = [call_bedrock_model(prompt) for prompt in prompts]<br>    results = await asyncio.gather(*tasks)<br>    return results<br><br>async def main():<br>    prompts = [<br>        &quot;Explain quantum computing in simple terms&quot;,<br>        &quot;Write a short poem about artificial intelligence&quot;,<br>        &quot;Describe the future of space travel in a few sentences&quot;,<br>        &quot;What are the main principles of machine learning?&quot;,<br>        &quot;How does blockchain technology work?&quot;,<br>        &quot;Explain the concept of neural networks in AI&quot;<br>    ]<br>    start = time.time()<br>    results = await process_prompts(prompts)<br>    for prompt, result in zip(prompts, results):<br>        print(f&quot;Prompt: {prompt}\nResponse: {result}\n&quot;)<br>    end = time.time()<br>    print(f&quot;Completed in {end - start:.2f} seconds&quot;)<br>    <br>asyncio.run(main())</pre><p>This code allows multiple API calls to be made concurrently, significantly reducing the total time required to process all prompts. It just took around <strong>33 seconds</strong> to get completions on all prompts.</p><h3>Asynchronous Programming vs. Multithreading and Multiprocessing</h3><p>Asynchronous programming, multithreading, and multiprocessing are all powerful tools in a developer’s arsenal, each with its own strengths and ideal use cases. Understanding when and how to use each approach is crucial for building efficient and scalable applications.</p><p>Asynchronous programming excels in handling I/O-bound tasks, such as network requests or API calls. It allows for concurrent execution within a single thread, efficiently switching between tasks while waiting for I/O operations to complete. This results in better resource utilization and improved overall performance, especially when dealing with a high volume of network requests or API calls. Asynchronous code is often more readable and easier to reason about than complex multi-threaded programs.</p><p>Multithreading, on the other hand, enables true parallelism by utilizing multiple CPU cores. It’s particularly useful for CPU-bound tasks that can be divided into smaller, independent units of work. While it introduces some complexity in managing shared state and avoiding race conditions, modern threading libraries and patterns have made it easier to work with threads safely and effectively.</p><p>Multiprocessing takes parallelism a step further by spawning multiple processes, each with its own memory space. This approach is ideal for CPU-intensive tasks that require significant computational power and can be parallelized. While it comes with higher overhead in terms of memory usage and inter-process communication, multiprocessing can dramatically speed up certain types of workloads.</p><p>Each approach has its strengths:</p><ol><li>Asynchronous programming: Ideal for I/O-bound tasks, efficient resource utilization, and maintaining responsiveness in applications with many concurrent operations.</li><li>Multithreading: Well-suited for parallelizing CPU-bound tasks within a single process, allowing for shared memory and faster inter-thread communication.</li><li>Multiprocessing: Excellent for CPU-intensive tasks that can be split into independent units, providing true parallelism and isolation between processes.</li></ol><h3>Combining Asynchronous Processing with Multithreading</h3><p>While each parallelization approach has its strengths, we can in fact benefit from a hybrid approach, combining these techniques to leverage the best of both worlds. One particularly powerful combination is the use of multithreading with asynchronous programming. This hybrid approach allows you to:</p><ol><li>Utilize multiple CPU cores efficiently through multithreading.</li><li>Handle I/O-bound operations asynchronously within each thread.</li><li>Balance the load across multiple threads while maintaining the efficiency of asynchronous operations.</li></ol><p>By using a thread pool to manage multiple asynchronous tasks, you can control the level of concurrency and prevent overwhelming system resources or exceeding API rate limits.</p><p>Here’s an example using Python’s <a href="https://docs.python.org/3/library/concurrent.futures.html">concurrent.futures</a> module along with asyncio for the above Bedrock code.</p><pre>import asyncio<br>import boto3<br>import json<br>import time<br>from concurrent.futures import ThreadPoolExecutor<br>import threading<br><br>bedrock_runtime = boto3.client(service_name=&quot;bedrock-runtime&quot;, region_name=&#39;us-east-1&#39;)<br><br>async def call_bedrock_model(prompt, model_id=&quot;anthropic.claude-3-haiku-20240307-v1:0&quot;):<br>    thread_id = threading.get_ident()<br>    print(f&quot;Processing prompt in thread {thread_id}: {prompt[:30]}...&quot;)<br>    <br>    body = json.dumps(<br>    {<br>        &quot;messages&quot;: [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: [{&quot;type&quot;: &quot;text&quot;, &quot;text&quot;: prompt}]}],<br>        &quot;anthropic_version&quot;: &quot;bedrock-2023-05-31&quot;,<br>        &quot;max_tokens&quot;: 8000,<br>        &quot;temperature&quot;: 0.7,<br>        &quot;top_p&quot;: 0.95,<br>        &quot;top_k&quot;: 250,<br>    }<br>    )<br>    modelid = model_id<br>    accept = &quot;application/json&quot;<br>    contenttype = &quot;application/json&quot;<br>    <br>    loop = asyncio.get_running_loop()<br>    response = await loop.run_in_executor(<br>        None,<br>        lambda: bedrock_runtime.invoke_model(<br>            body=body, modelId=modelid, accept=accept, contentType=contenttype<br>        )<br>    )<br>    return json.loads(response.get(&quot;body&quot;).read()).get(&quot;content&quot;)[0][&quot;text&quot;]<br><br>async def process_prompts(prompts, max_workers=3):<br>    with ThreadPoolExecutor(max_workers=max_workers) as executor:<br>        loop = asyncio.get_running_loop()<br>        tasks = [<br>            loop.run_in_executor(executor, asyncio.run, call_bedrock_model(prompt))<br>            for prompt in prompts<br>        ]<br>        results = await asyncio.gather(*tasks)<br>    return results<br><br>async def main():<br>    prompts = [<br>        &quot;Explain quantum computing in simple terms&quot;,<br>        &quot;Write a short poem about artificial intelligence&quot;,<br>        &quot;Describe the future of space travel in a few sentences&quot;,<br>        &quot;What are the main principles of machine learning?&quot;,<br>        &quot;How does blockchain technology work?&quot;,<br>        &quot;Explain the concept of neural networks in AI&quot;<br>    ]<br>    start = time.time()<br>    results = await process_prompts(prompts)<br>    for prompt, result in zip(prompts, results):<br>        print(f&quot;Prompt: {prompt}\nResponse: {result}\n&quot;)<br>    end = time.time()<br>    print(f&quot;Completed in {end - start:.2f} seconds&quot;)<br><br>if __name__ == &quot;__main__&quot;:<br>    asyncio.run(main())</pre><p>As you might notice the total run time has now reduced to around <strong>10 seconds.</strong></p><p>In this implementation, we’ve integrated threading with asyncio to optimize the handling of API calls to the Bedrock model. The ThreadPoolExecutor is used to manage a pool of worker threads, allowing us to control the number of concurrent API requests. Within the process_prompts function, each call to call_bedrock_model is executed in a separate thread from the pool. Importantly, each thread operates asynchronously within itself, using await to yield control when waiting for I/O operations, allowing the event loop to efficiently manage multiple concurrent tasks across different threads. The max_workers parameter in the ThreadPoolExecutor allows us to fine-tune the number of concurrent threads, which is crucial for managing system resources and respecting any API rate limits. This approach allows us to process a large number of prompts efficiently by combining the benefits of asynchronous I/O and multithreading.</p><h3>Adding Retries with Exponential Backoff</h3><p>When working with external APIs, especially in cloud environments, it’s crucial to implement robust error handling and retry mechanisms. Network issues, temporary service outages, or rate limiting can cause API calls to fail intermittently. Implementing a retry strategy with exponential backoff can significantly improve the reliability of your application.</p><p>Exponential backoff is a technique where the wait time between retries increases exponentially. This approach helps to reduce the load on the API during periods of high traffic or when the service is experiencing issues, while still allowing your application to recover from temporary failures.</p><p>The backoff library in Python provides a simple way to implement exponential backoff for function calls. It offers decorators that can be applied to functions, automatically handling retries with configurable backoff strategies.</p><pre>import asyncio<br>import boto3<br>import json<br>import time<br>from concurrent.futures import ThreadPoolExecutor<br>import threading<br>import backoff<br>from botocore.exceptions import ClientError<br><br>bedrock_runtime = boto3.client(service_name=&quot;bedrock-runtime&quot;, region_name=&#39;us-east-1&#39;)<br><br>def backoff_hdlr(details):<br>    print(f&quot;Backing off {details[&#39;wait&#39;]:0.1f} seconds after {details[&#39;tries&#39;]} tries&quot;)<br><br>@backoff.on_exception(backoff.expo, ClientError, max_tries=5, on_backoff=backoff_hdlr)<br>def invoke_bedrock_model(body, modelid, accept, contenttype):<br>    return bedrock_runtime.invoke_model(<br>        body=body, modelId=modelid, accept=accept, contentType=contenttype<br>    )<br><br>async def call_bedrock_model(prompt, model_id=&quot;anthropic.claude-3-haiku-20240307-v1:0&quot;):<br>    thread_id = threading.get_ident()<br>    print(f&quot;Processing prompt in thread {thread_id}: {prompt[:30]}...&quot;)<br>    <br>    body = json.dumps(<br>    {<br>        &quot;messages&quot;: [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: [{&quot;type&quot;: &quot;text&quot;, &quot;text&quot;: prompt}]}],<br>        &quot;anthropic_version&quot;: &quot;bedrock-2023-05-31&quot;,<br>        &quot;max_tokens&quot;: 8000,<br>        &quot;temperature&quot;: 0.7,<br>        &quot;top_p&quot;: 0.95,<br>        &quot;top_k&quot;: 250,<br>    }<br>    )<br>    modelid = model_id<br>    accept = &quot;application/json&quot;<br>    contenttype = &quot;application/json&quot;<br>    <br>    loop = asyncio.get_running_loop()<br>    response = await loop.run_in_executor(<br>        None,<br>        lambda: invoke_bedrock_model(body=body, modelid=modelid, accept=accept, contenttype=contenttype)<br>    )<br>    return json.loads(response.get(&quot;body&quot;).read()).get(&quot;content&quot;)[0][&quot;text&quot;]<br><br>async def process_prompts(prompts, max_workers=3):<br>    with ThreadPoolExecutor(max_workers=max_workers) as executor:<br>        loop = asyncio.get_running_loop()<br>        tasks = [<br>            loop.run_in_executor(executor, asyncio.run, call_bedrock_model(prompt))<br>            for prompt in prompts<br>        ]<br>        results = await asyncio.gather(*tasks)<br>    return results<br><br>async def main():<br>    prompts = [<br>        &quot;Explain quantum computing in simple terms&quot;,<br>        &quot;Write a short poem about artificial intelligence&quot;,<br>        &quot;Describe the future of space travel in a few sentences&quot;,<br>        &quot;What are the main principles of machine learning?&quot;,<br>        &quot;How does blockchain technology work?&quot;,<br>        &quot;Explain the concept of neural networks in AI&quot;<br>    ]<br>    start = time.time()<br>    results = await process_prompts(prompts)<br>    for prompt, result in zip(prompts, results):<br>        print(f&quot;Prompt: {prompt}\nResponse: {result}\n&quot;)<br>    end = time.time()<br>    print(f&quot;Completed in {end - start:.2f} seconds&quot;)<br><br>if __name__ == &quot;__main__&quot;:<br>    asyncio.run(main())</pre><p>Let’s modify our Bedrock code to include retries with exponential backoff using the backoff library:</p><p>In this updated version, we&#39;ve made the following changes:</p><ol><li>Added the backoff library import.</li><li>Created a new function invoke_bedrock_model that wraps the actual API call.</li><li>Applied the @backoff.on_exception decorator to invoke_bedrock_model. This decorator will retry the function with exponential backoff if it raises a ClientError.</li><li>The backoff is configured to retry up to 5 times, with an exponential delay between retries.</li><li>Added a backoff_hdlr function to log information about retries.</li><li>Updated the call_bedrock_model function to use the new invoke_bedrock_model function.</li></ol><p>This implementation provides several benefits:</p><ol><li><strong>Improved Reliability</strong>: The code will automatically retry failed API calls, handling temporary network issues or service disruptions.</li><li><strong>Exponential Backoff</strong>: The wait time between retries increases exponentially, reducing the load on the API during periods of high traffic or instability.</li><li><strong>Configurable</strong>: The retry behavior can be easily adjusted by modifying the decorator parameters.</li><li><strong>Logging</strong>: The backoff_hdlr function provides visibility into the retry process, which can be helpful for debugging and monitoring.</li></ol><p>By implementing retries with exponential backoff, your application becomes more resilient to transient failures, improving overall reliability and user experience. This is particularly important when working with cloud services and external APIs, where network conditions and service availability can vary.</p><p>Remember to install the backoff library using pip:</p><pre>pip install backoff</pre><h3>Conclusion</h3><p>Asynchronous programming is a powerful tool for developers building applications that leverage AI models through APIs. By allowing concurrent execution of API calls, it significantly improves the performance and responsiveness of these applications. Whether you’re using public AI APIs or specialized services like Amazon Bedrock, incorporating asynchronous processing into your development workflow can lead to more efficient and scalable generative AI applications. Furthermore, the integration of retries with exponential backoff adds an extra layer of robustness to your applications. This approach not only enhances reliability by automatically handling transient failures but also demonstrates considerate behavior towards the API services by backing off during periods of high load or instability</p><p>As the field of AI continues to advance, the ability to efficiently manage and process large numbers of API calls will become increasingly important. By mastering asynchronous programming techniques, developers can stay ahead of the curve and build cutting-edge applications that fully harness the power of generative AI. However, it’s crucial to note that while these techniques can dramatically improve efficiency, developers must be mindful of API rate limits and usage thresholds. Many AI service providers implement strict limits to ensure fair usage and maintain service quality for all users.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=871e76d1d02c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a real time Face Recognition system using pre-trained FaceNet model]]></title>
            <link>https://vinayak-arannil.medium.com/building-a-real-time-face-recognition-system-using-pre-trained-facenet-model-f1a277a06947?source=rss-4f310ccec56e------2</link>
            <guid isPermaLink="false">https://medium.com/p/f1a277a06947</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[face-recognition]]></category>
            <category><![CDATA[facedetection]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Vinayak Arannil]]></dc:creator>
            <pubDate>Wed, 08 Nov 2017 11:36:19 GMT</pubDate>
            <atom:updated>2018-07-09T10:14:45.696Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/847/1*Ejjus4-wp3W5LJ0mGAtScA.jpeg" /></figure><p>Code at: <a href="https://github.com/vinayakkailas/face_recognition">https://github.com/vinayakkailas/face_recognition</a></p><p><strong>F</strong>ace <strong>R</strong>ecognition is becoming a new trend in the security authentication systems. Modern FR systems can even detect, if the person is real(live) or not while doing face recognition, preventing the systems being hacked by showing the picture of a real person. I am sure, everyone wondered when Facebook implemented the auto-tagging technique. It identifies the person and tag him/her when ever you upload a picture. It is so efficient that, even when the person’s face is occluded or the picture is taken in darkness, it tags accurately. All these successful face recognition systems are the results of recent advancements in the field of computer vision, which is backed by powerful deep learning algorithms. Let us explore one of such algorithms and see how we can implement a real time face recognition system.</p><p>Face recognition can be done in two ways. Imagine you are building a face recognition system for an enterprise. One way of doing this is by training a neural network model (preferably a ConvNet model) , which can classify faces accurately. As you know for a classifier to be trained well, it needs millions of input data. Collecting that many images of employees, is not feasible. So this method seldom works. The best way of solving this problem is by opting one-shot learning technique. One-shot learning aims to learn information about object categories from one, or only a few, training images. The model still needs to be trained on millions of data, but the dataset can be any, but of the same domain. Let me explain this more clearly. In one shot way of learning, you can train a model with any face datasets and use it for your own data which is very less in number. There are many publicly available face datasets like <a href="http://www.cbsr.ia.ac.cn/english/CASIA-WebFace-Database.html">CASIA-WebFace</a>, <a href="https://www.microsoft.com/en-us/research/project/ms-celeb-1m-challenge-recognizing-one-million-celebrities-real-world/">MS-Celeb-1M</a>, AT&amp;T faces dataset,<strong> </strong>etc which are having millions of face images. Now the question is, what kind of model is required for building a face recognition system.</p><p>One-shot learning can be implemented using a Siamese network. As the name indicates, its nothing but, two identical neural networks with exact same weights, but taking two distinct inputs. These networks are optimised based on the contrastive loss between their outputs. This loss will be small when the inputs to the networks are similar and large when inputs differ from each other. So in this way, optimised Siamese networks can differentiate between their inputs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*4YaomGBr6KoUXJqxMzlxKA.png" /><figcaption>Siamese network</figcaption></figure><p><a href="https://arxiv.org/pdf/1503.03832.pdf">FaceNet </a>is a one-shot model, that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity. Once this space has been produced, tasks such as face recognition, verification and clustering can be easily implemented using standard techniques with FaceNet embeddings as feature vectors(<em>from the original paper</em>).To train, they used triplets of roughly aligned matching / non-matching face patches. A triplet is nothing but a collection one anchor image, one matching image to the anchor image and one non-matching image to the anchor image. So the triplet loss minimises the distance between an anchor and a positive, both of which have the same identity, and maximises the distance between the anchor and a negative of a different identity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/792/1*RneQkRhw5o86gR-eFzRgQg.png" /><figcaption>Triplet loss before and after training(taken from facenet paper)</figcaption></figure><p>There are many pre-trained facenet models available which are trained on different deep learning frameworks. One such implementation is <a href="https://github.com/davidsandberg/facenet">https://github.com/davidsandberg/facenet</a> by David Sandberg using tensorflow. He has published the weights at <a href="https://drive.google.com/file/d/0B5MzpY9kBtDVZ2RpVDYwWmxoSUk">https://drive.google.com/file/d/0B5MzpY9kBtDVZ2RpVDYwWmxoSUk</a>. Let us see how we can use a pre-trained model for our use case. Here are the steps.</p><ol><li>Collect the images of all employees.</li><li>Align the faces using <a href="https://kpzhang93.github.io/papers/spl.pdf">MTCNN</a> (Multi-task Cascaded Convolutional Neural Networks), d<a href="http://blog.dlib.net/2017/02/high-quality-face-recognition-with-deep.html">lib</a> or <a href="http://docs.opencv.org/master/tutorial_py_face_detection.html">Opencv</a>. These methods identify, detect and align the faces by making eyes and bottom lip appear in the same location on each image.</li><li>Use the pre-trained facenet model to represent (or embed) the faces of all employees on a 128-dimensional unit hyper sphere.</li><li>Store the embeddings with respective employee names on disc.</li></ol><p>Now your face recognition system is ready !!. Let us see how we can recognise faces, with what all we have done above. Now you have with you, the corpus of 128-dimensional embeddings with corresponding employee names. When ever an employee faces your detection camera, the image being captured will be ran through the pre-trained network to create the 128-dimensional embedding which will then be compared to the stored embeddings using euclidean(L2) distance. If the lowest distance between the captured embedding and the stored embeddings is less than a threshold value, the system can recognise that person as the employee corresponding to that lowest distant embedding.</p><p>That’s all…. You have built a simple, but efficient face recognition system. Have a nice day…!!</p><p><strong>Note</strong>: If you have 1000s of classes with good number of images per class, better train a classifier on top of the features extracted through a pre-trained facenet model than calculating the embedding distances.</p><p>References:</p><ol><li>Deepface paper <a href="https://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr14.pdf">https://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr14.pdf</a></li><li>Facenet paper <a href="https://arxiv.org/pdf/1503.03832.pdf">https://arxiv.org/pdf/1503.03832.pdf</a></li><li>Pre-trained facenet model <a href="https://github.com/davidsandberg/facenet">https://github.com/davidsandberg/facenet</a></li><li>training a classifier using extracted features <a href="https://github.com/davidsandberg/facenet/wiki/Train-a-classifier-on-own-images">https://github.com/davidsandberg/facenet/wiki/Train-a-classifier-on-own-images</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f1a277a06947" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Significance of transfer learning in the domain space of artificial intelligence]]></title>
            <link>https://vinayak-arannil.medium.com/significance-of-transfer-learning-in-the-domain-space-of-artificial-intelligence-1ebd7a1298f2?source=rss-4f310ccec56e------2</link>
            <guid isPermaLink="false">https://medium.com/p/1ebd7a1298f2</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Vinayak Arannil]]></dc:creator>
            <pubDate>Sat, 11 Mar 2017 16:23:06 GMT</pubDate>
            <atom:updated>2017-11-21T07:05:00.828Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vez_ToyOzzZ1_ZMnnOzFvA.jpeg" /></figure><blockquote><strong>“Transfer Learning leads to Industrialization” — Andrew Ng</strong></blockquote><p>With the emergence of better GPUs and computational powers, two fields started accelerating with their full thrust. Obviously, the first one is the gaming industry and the other is the field of artificial intelligence. AI is more of a general term. What actually implements AI, is machine learning or better say deep learning. How does GPUs assist deep learning? To understand this, one should dig deeper into artificial neural networks. ANNs are implemented using matrix multiplications. For deep neural networks the number of layers will be many, and so does the number of computations. This cannot be handled with simple CPUs. It needs some sort of parallel computing to achieve its goals. That’s were cuda enabled GPUs provides enormous help. But, as you know these GPUs are very costly. Not every researcher can afford these.</p><p>Training the model is the major task in deep learning. When the model becomes complex, cost of training becomes high. Researchers go for complex models when they don’t get enough results with simple models. A complex deep learning model can be developed only if you have years of experience in neural network architectures. Then how do you expect the current software engineers, with limited expertise, to develop great models and interesting AI applications? This where transfer learning approach is so advantageous. Transfer learning is nothing but transfer of knowledge. As described in one of my previous articles, a perfect example of transfer learning is the relation between a teacher and students. Teacher transfers the knowledge he/she gained over time, to the students by which students become intelligent. Same approach can be adopted in deep learning also. For example, if we design a model to detect rotten tomatoes from a bag of tomatoes, the same model can be used for detecting rotten apples also as some of the features are same in both set of data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RyC6n0tnFFoiFn9I.png" /></figure><p>Now what is transfer learning in technical words? Lets take the case of image recognition. As many of you know, the major part of image recognition is the feature extraction. How well you extract the features, that better will be your predicted result. Convolutional neural networks are considered to be the best ways to extract features from images. This again depends on how you design your convolutional layers. This design sense comes from expertise. A beginner level researcher will never be able to design an exceptionally well image recognition model as many of them are not aware of the complete theoretical stuffs behind neural networks like optimizing the network to get rid of over fitting, under fitting, etc. The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) is a competition conducted every year to evaluate the best model on image classification. If you have seen the models which won the competition in recent years, you can understand how complex the models are turning out, to get better results. Below diagram shows the architecture of a well known model called Inception-V3 developed by Google research team, which achieved a top five error rate of 3.6 in the ILSVRC challenge. Parallel convolutional layers and feedback loops make this model highly complex.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XQnM39Wmg0tSvB6E.png" /><figcaption><em>credits:Google research team</em></figcaption></figure><p>From the diagram, it is clear that, except the last softmax layer, the rest of the model is complete convolutional, pooling and fully connected neural network layers which does the major part of feature extraction. Once the training is done, this part of the model can extract features of any input image, which is of similar distribution as of the training data set. The softmax layers determines the output labels. If we have a model trained on a particular data set we can reuse the same convolutional layers for doing feature extractions in a different dataset(having similar distribution). We just need to retrain the last softmax layer so that, the model can predicts the classes from the new dataset. Technically this is what transfer learning, in the context of deep neural networks. One main advantage of this approach is that, you dont need a high cost GPU for retraining the model. A simple CPU is enough for this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9ed_DdYlsnPRz0PV.png" /><figcaption>credits:Hvass-Labs/TensorFlow-Tutorials</figcaption></figure><p>There are many pre-trained models available in public like VGG-16, AlexNet, Inception V3, etc which gave very good results on image-net dataset which is an academic dataset of around 1000 classes. AI engineers can download these models and do transfer learning for their own datasets so that they dont need to worry about creating complex models and use of costly hardware configurations. Tensorflow website provides a tutorial on how to retrain the Inception v3 model for your own dataset. Below is the link to the site.</p><p><a href="https://www.tensorflow.org/tutorials/image_retraining"><strong>https://www.tensorflow.org/tutorials/image_retraining</strong></a></p><p>In summary these are the steps followed in the retrain.py script(available in the tensorflow repository), for retraining the inception model:</p><ol><li>Download and set up the pre-trained graph. The pretrained graph is available at ‘http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz&#39;</li><li>Create the list of all the images.</li><li>Apply distortions if required.</li><li>Calculate the ‘bottleneck’ image summaries and cache them on disk(bottleneck value is nothing but the feature vector you get, when you run an image till the bottleneck tensor).</li><li>Add the new layer that we’ll be training(softmax layer).</li><li>Feed the bottlenecks and ground truths into the graph, and run the training</li><li>Write out the trained graph and labels with the weights, stored as constants. You may use GraphDef() functionality or tf.train.Saver() to save the graph.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1ebd7a1298f2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sentiment Analysis powered by deep learning: An automated way to know your customer’s feelings]]></title>
            <link>https://vinayak-arannil.medium.com/sentiment-analysis-powered-by-deep-learning-an-automated-way-to-know-your-customers-feelings-a8418dc09763?source=rss-4f310ccec56e------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8418dc09763</guid>
            <category><![CDATA[sentiment-analysis]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Vinayak Arannil]]></dc:creator>
            <pubDate>Fri, 03 Mar 2017 04:44:52 GMT</pubDate>
            <atom:updated>2017-11-21T07:05:21.053Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/412/1*swwhO7kHKg_D3uTlMDm7DQ.jpeg" /><figcaption>©2017 Clarabridge®. All rights reserved.</figcaption></figure><p>A machine predicting your emotions from what you talk/write. Truly an amazing concept…!!! Imagine you have 1 million customers for your product. 1000’s of them write reviews on the product. As you know, theses reviews are the best ways to understand, how your product is being accepted in the market. Reading all these reviews and understanding the feelings of customers is a tedious task. You need to have separate human resources to identify which among theses reviews are, positive and negative. Some reviews might be neutral as well. In simple words this is what a sentiment analysis is all about.</p><p>Sentiment analysis is not a new technique. Its been part of market analysis for several years now. 15 years ago it was more of a manual task. But with the advent of accurate machine learning algorithms and tools, now it has been considered as an automated task. Question arises here.Which is the best machine learning algorithm for doing sentiment analysis? This was a relevant query until past 3–4 years in all sentiment analytics conferences. Now we are in the era of deep learning. The same question is now being changed to, which neural network architecture is the best fit for sentiment analysis.This change has come up since, deep learning outperforms even a complex machine learning nlp algorithm.</p><p>How does deep learning classifies the sentiments so efficiently? There are multiple reasons behind it. The most difficult part of natural language processing is the representation of words. Earlier data scientists were using simple bag of words model for word representations which doesn’t provide any semantic relations between words. Deep learning has shown better ways to represent words as meaning full vectors. Word2vec models (by mikolov) like skip gram or continuous bag of words can meaningfully represent any words in a vector space. Once you overcome this difficult part of representing words, now its quiet an easy task. You can either use deep learning again or use any machine learning algorithms for classifying the text to positive or negative sentiments. Simple linear regression to complex stacked LSTMs can be used for this purpose. The accuracy increases when you use neural networks as it can extract features much better.</p><p>Features on images are easily understandable. When it comes to sentences, people get confused whether text also has features or not. Let’s understand this by taking two sentences like, “it is an awesome movie” and “it is an awful movie”. It so evident that, the polarity of these sentences are determined by the words like ‘awesome’ and ‘awful’. These kind of words are literally called the features of a sentence. They give the actual meaning to the sentences. Better you extract these features, better will be your accuracy. For a machine learning way of doing sentiment analysis, feature extraction is not an easy task. This particular task dependent feature engineering part is completely skipped in a deep learning model, except a few pre-processing steps. The model extracts the features from sentences by itself and predicts the polarity.</p><p>How to create a sentiment analysis model using deep learning? Steps are as follows:-</p><ol><li>Split the text data into sentences.</li><li>Append sentence_start and sentence_end tags to each sentence.</li><li>Tokenize the sentences into words( You can use NLTK’s wordtokenizer)</li><li>Count the word frequencies.</li><li>Create a dictionary of words and word word indexes.</li><li>Convert each word into word embeddings using word2vec model(You can even use google’s pre-trained word embeddings for this).</li><li>Feed the embedding vectors to a CNN, RNN or LSTM network (even a simple DNN can also be used)</li><li>Train the neural network using any back propagation algorithm.</li><li>Save the model and use it for predictions.</li></ol><p>Getting an accurate sentiment analysis model is not that easy. There are so many challenges which makes it a difficult task. Some are listed below.</p><ul><li><strong>Sarcasm</strong>: This is a type of talk which even humans fail to understand. Then how do you expect a machine to do it better ?!</li><li><strong>Negation</strong>: Sentences like “it was not that good” conveys a negative sentiment though it contains positive features like the word “good”. There are several papers in which they explained how to overcome this challenge. Even a broader data set with such sentences solves this problem.</li><li><strong>Miss spelling</strong>: use of messaging language like ‘gr8’, ‘gud’, ‘evng’, etc are not complete words.( Can still be solved with enough such words in the data set)</li><li><strong>Anaphora resolution</strong>: The problem of resolving what a pronoun, or a noun phrase refers to.”We watched the movie and went to dinner; it was awful.” What does “It” refer to?</li><li><strong>Rare words: </strong>Words which are very rare can create confusions.<strong> </strong>“Really <em>Shoddy</em>service”.</li></ul><p>Even with these challenges, existing deep learning models give very good accuracy and is being applied on multiple domains. Sentiment analysis is being used extensively in quality improvements, recommendation systems, policy/decision making, marketing research, etc</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8418dc09763" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Recommendation systems using image similarity powered by deep learning]]></title>
            <link>https://vinayak-arannil.medium.com/recommendation-systems-based-on-deep-learning-9e6802c04b28?source=rss-4f310ccec56e------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e6802c04b28</guid>
            <category><![CDATA[recommendation-system]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[image-recognition]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Vinayak Arannil]]></dc:creator>
            <pubDate>Fri, 03 Mar 2017 04:40:21 GMT</pubDate>
            <atom:updated>2018-07-14T15:26:09.843Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*PNAUtKhqIbmdnIVT_CDlFg.jpeg" /></figure><p>Code at: <a href="https://github.com/vinayakkailas/image_similarity">https://github.com/vinayakkailas/image_similarity</a></p><p>Pre-read:Here i talk only about recommendations based on image similarity. Not the traditional recommendation systems.</p><p>A good recommendation system will always boost your sales. When it comes to apparels or footwear, the use of better recommendations using image contents are always a matter of prime importance. Currently recommendation systems are implemented using traditional machine learning algorithms like matrix factorization, collaborative filtering,etc. Algorithms like ‘Nearest Neighbour’ provides easy way to implement recommendations based on image similarity. But have you ever verified how accurate are these recommendations? In a competitive e-commerce market, a simple recommendation system wont serve your purpose. That’s where deep learning comes into picture. Researches and experiments have already proven that deep learning has achieved better results than normal machine learning learning algorithms on image retrieval tasks.</p><p>What is deep learning? is it an advanced form of machine learning? or is it more closer towards the so called ‘artificial Intelligence’ than machine learning? Getting the answers for these questions are mandatory to dive into the field of artificial intelligence. Deep learning is not an advanced machine learning technique. Both are separate ways to solve AI problems. Deep learning will never replace machine learning. Both will co-exist and solve the AI problems in its own ways. Some problems can be solved easily with machine learning where as deep learning fits better for some other. The main difference between these two, comes in the feature engineering part. The accuracy of machine learning algorithm heavily depends up on the feature engineering done on the data set,while deep learning does this part by itself.</p><p>When it comes to image recognition, deep learning always stand one step ahead. Thanks to Yann Lecun for the contributions on Convolutional Neural Networks. CNNs provide better feature extractions than a simple deep neural network. Many models have been developed based on CNN architectures and found to give recognition accuracy almost closer to humans. Now its the prefect time to think and decide, how to make use of this magical technique for boosting your business. With in a couple of years, deep learning is going to rule the e-commerce industry.</p><p>Many people are reluctant to get into deep learning because of the complex mathematical calculations and probability theories behind it. They will somehow try making some models and fail to get enough results out of them and quit henceforth. One thing they are unaware of, is how to make use of the existing models, which are having proven accuracy, for their own use. In technical terms, its called “transfer learning”. It means we are transferring the knowledge or intelligence we gained for task to a separate but similar task. A perfect example of transfer learning is the relation between a teacher and students. Teacher transfers the knowledge he/she gained over time, to the students by which students become intelligent. Same approach can be adapted in deep learning also to achieve your goals. There are many open sourced highly accurate models which can be transfer learned for your business purpose.</p><p>We have tried out some research using some of the best existing models and got amazing results for image retrieval tasks. We have explored transfer learning technique along with distance calculations of feature vectors and obtained perfect results for multiple classes. Here are few examples:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gL9UZQFdwFk-9smY.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ziXWbDIrdW_qJMCL.png" /></figure><p>These results are obtained by doing simple feature matching using deep learning. Complex algorithms can be built using feature vectors along with class labels and other attributes of the images like colour, pattern, etc. So in nutshell, visual recognition based image retrieval and recommendation systems are going to rule the e-commerce industry in coming years. So make yourself prepared to fight against the competitors with highly efficient deep learning models.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e6802c04b28" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>