<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ashwin N on Medium]]></title>
        <description><![CDATA[Stories by Ashwin N on Medium]]></description>
        <link>https://medium.com/@ashwinnaidu1991?source=rss-1c823c9f4438------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*qX5ArWXS9xGMVJh9Cq6L8A.png</url>
            <title>Stories by Ashwin N on Medium</title>
            <link>https://medium.com/@ashwinnaidu1991?source=rss-1c823c9f4438------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 07:08:34 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ashwinnaidu1991/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[I Built an AI Chief of Staff with Gemini That Triages My Inbox Every Morning]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://blog.devgenius.io/i-built-an-ai-chief-of-staff-with-gemini-that-triages-my-inbox-every-morning-787fb3dedeb3?source=rss-1c823c9f4438------2"><img src="https://cdn-images-1.medium.com/max/1024/1*2KQKEJFjS2PKM8t9LywsNg.png" width="1024"></a></p><p class="medium-feed-snippet">No Claude Code. No subscriptions. Just Python, a free Gemini API key, and 45 minutes.</p><p class="medium-feed-link"><a href="https://blog.devgenius.io/i-built-an-ai-chief-of-staff-with-gemini-that-triages-my-inbox-every-morning-787fb3dedeb3?source=rss-1c823c9f4438------2">Continue reading on Dev Genius »</a></p></div>]]></description>
            <link>https://blog.devgenius.io/i-built-an-ai-chief-of-staff-with-gemini-that-triages-my-inbox-every-morning-787fb3dedeb3?source=rss-1c823c9f4438------2</link>
            <guid isPermaLink="false">https://medium.com/p/787fb3dedeb3</guid>
            <category><![CDATA[gemini]]></category>
            <category><![CDATA[google-ai]]></category>
            <category><![CDATA[gemini-api]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Ashwin N]]></dc:creator>
            <pubDate>Mon, 20 Apr 2026 22:49:56 GMT</pubDate>
            <atom:updated>2026-04-21T14:27:51.808Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[The Transformative Impact of Large Language Models in Financial Services: Market Growth and…]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@ashwinnaidu1991/the-transformative-impact-of-large-language-models-in-financial-services-market-growth-and-0fcb73c47036?source=rss-1c823c9f4438------2"><img src="https://cdn-images-1.medium.com/max/1920/1*rld-poYQtDbfm3YbXGf5Yw.png" width="1920"></a></p><p class="medium-feed-snippet">The financial industry is experiencing a transformative shift driven by Large Language Models (LLMs) and generative AI technologies. This&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/@ashwinnaidu1991/the-transformative-impact-of-large-language-models-in-financial-services-market-growth-and-0fcb73c47036?source=rss-1c823c9f4438------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@ashwinnaidu1991/the-transformative-impact-of-large-language-models-in-financial-services-market-growth-and-0fcb73c47036?source=rss-1c823c9f4438------2</link>
            <guid isPermaLink="false">https://medium.com/p/0fcb73c47036</guid>
            <dc:creator><![CDATA[Ashwin N]]></dc:creator>
            <pubDate>Fri, 04 Apr 2025 14:04:49 GMT</pubDate>
            <atom:updated>2025-04-04T14:04:49.901Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Mastering CycleGAN: An In-Depth Analysis with Real-World Applications]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@ashwinnaidu1991/mastering-cyclegan-an-in-depth-analysis-with-real-world-applications-938c57613c59?source=rss-1c823c9f4438------2"><img src="https://cdn-images-1.medium.com/max/1024/0*Lr8Noh9hshNSlW3E" width="1024"></a></p><p class="medium-feed-snippet">CycleGAN is a state-of-the-art deep learning model that has gained significant attention in the field of computer vision.</p><p class="medium-feed-link"><a href="https://medium.com/@ashwinnaidu1991/mastering-cyclegan-an-in-depth-analysis-with-real-world-applications-938c57613c59?source=rss-1c823c9f4438------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@ashwinnaidu1991/mastering-cyclegan-an-in-depth-analysis-with-real-world-applications-938c57613c59?source=rss-1c823c9f4438------2</link>
            <guid isPermaLink="false">https://medium.com/p/938c57613c59</guid>
            <category><![CDATA[adversarial-network]]></category>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[cyclegan]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[image-to-image]]></category>
            <dc:creator><![CDATA[Ashwin N]]></dc:creator>
            <pubDate>Fri, 20 Oct 2023 15:23:06 GMT</pubDate>
            <atom:updated>2023-10-20T15:37:58.113Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Redefining Human Resources: How AI is Shaping Recruitment, Management, and Ethics.]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/analytics-vidhya/redefining-human-resources-how-ai-is-shaping-recruitment-management-and-ethics-c3c56dea656e?source=rss-1c823c9f4438------2"><img src="https://cdn-images-1.medium.com/max/1120/1*ajVOI-pSgmkWdWUHVpLNNA.jpeg" width="1120"></a></p><p class="medium-feed-snippet">Introduction</p><p class="medium-feed-link"><a href="https://medium.com/analytics-vidhya/redefining-human-resources-how-ai-is-shaping-recruitment-management-and-ethics-c3c56dea656e?source=rss-1c823c9f4438------2">Continue reading on Analytics Vidhya »</a></p></div>]]></description>
            <link>https://medium.com/analytics-vidhya/redefining-human-resources-how-ai-is-shaping-recruitment-management-and-ethics-c3c56dea656e?source=rss-1c823c9f4438------2</link>
            <guid isPermaLink="false">https://medium.com/p/c3c56dea656e</guid>
            <category><![CDATA[human-resources]]></category>
            <category><![CDATA[recruiting]]></category>
            <category><![CDATA[future-of-work]]></category>
            <category><![CDATA[ethics-in-technology]]></category>
            <category><![CDATA[ai-recruiting]]></category>
            <dc:creator><![CDATA[Ashwin N]]></dc:creator>
            <pubDate>Sun, 13 Aug 2023 18:33:11 GMT</pubDate>
            <atom:updated>2023-08-13T18:33:11.282Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Unraveling the Mystique of Variational Autoencoders: A Comprehensive Guide to Understanding and…]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@ashwinnaidu1991/unraveling-the-mystique-of-variational-autoencoders-a-comprehensive-guide-to-understanding-and-10cf71925b20?source=rss-1c823c9f4438------2"><img src="https://cdn-images-1.medium.com/max/772/1*nocBSRodPFu3XD5UXfTwbw.png" width="772"></a></p><p class="medium-feed-snippet">Autoencoders are a type of artificial neural network used to learn efficient representations of data, typically for the purpose of&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/@ashwinnaidu1991/unraveling-the-mystique-of-variational-autoencoders-a-comprehensive-guide-to-understanding-and-10cf71925b20?source=rss-1c823c9f4438------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@ashwinnaidu1991/unraveling-the-mystique-of-variational-autoencoders-a-comprehensive-guide-to-understanding-and-10cf71925b20?source=rss-1c823c9f4438------2</link>
            <guid isPermaLink="false">https://medium.com/p/10cf71925b20</guid>
            <category><![CDATA[sensors]]></category>
            <category><![CDATA[variational-autoencoder]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[tensorflow]]></category>
            <dc:creator><![CDATA[Ashwin N]]></dc:creator>
            <pubDate>Thu, 10 Aug 2023 14:53:17 GMT</pubDate>
            <atom:updated>2023-08-10T14:53:17.465Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Unleashing the Magic of MLflow: An Enchanting Guide to Machine Learning and MLOps ]]></title>
            <link>https://medium.com/analytics-vidhya/unleashing-the-magic-of-mlflow-an-enchanting-guide-to-machine-learning-and-mlops-2a710592839?source=rss-1c823c9f4438------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a710592839</guid>
            <category><![CDATA[experiment-tracking]]></category>
            <category><![CDATA[mlflow]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[model-deployment-flask]]></category>
            <category><![CDATA[mlops]]></category>
            <dc:creator><![CDATA[Ashwin N]]></dc:creator>
            <pubDate>Tue, 25 Jul 2023 10:23:32 GMT</pubDate>
            <atom:updated>2023-08-10T14:55:50.870Z</atom:updated>
            <content:encoded><![CDATA[<h3>MLflow: Where Spells and Data Collide! 🧙‍♂️✨🔮</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*ikLWQ2hTwuxAd7Zrovs2Yw.jpeg" /><figcaption>MLflow can help for collaboration and reproduction in a simplest way here!</figcaption></figure><h3>Introduction</h3><p>Welcome to the mystical realm of machine learning, where sorcery meets science, and data transforms into powerful spells! In this enchanting guide, we’ll embark on an exciting journey to discover the magic of MLflow. MLflow is an open-source platform that empowers machine learning practitioners to track experiments, share projects, and deploy models with ease. So, ready your wands (keyboards) and let’s begin our magical adventure.</p><h3>The Three Enchanting Pillars of MLflow</h3><p>Prepare to be enchanted by the three pillars of MLflow: Tracking, Projects, and Models. Each one plays a vital role in shaping the magical landscape of machine learning. Let’s explore these mystical components one by one:</p><h4>1. MLflow Tracking: The Sorcerer’s Journal</h4><p>At the heart of MLflow lies the powerful Tracking component — the sorcerer’s journal that diligently logs every aspect of our machine learning experiments. Think of it as your enchanted diary, keeping track of parameters, metrics, and artifacts with ease. With MLflow Tracking, we gain invaluable insights into the performance of different model spells and unlock the secrets of successful incantations!</p><p>Example:</p><pre>import mlflow<br><br># Start an MLflow tracking run<br>mlflow.start_run()<br><br># Log parameters<br>mlflow.log_param(&quot;hidden_units&quot;, 128)<br>mlflow.log_param(&quot;learning_rate&quot;, 0.001)<br><br># Train the model and log metrics<br># ...<br><br># Save the model as an artifact<br># ...<br><br># End the run<br>mlflow.end_run()</pre><p><em>Learn more about MLflow Tracking: </em><a href="https://www.mlflow.org/docs/latest/tracking.html"><em>Official MLflow Tracking Documentation</em></a></p><h4>2. MLflow Projects: The Gateway to Reproducibility</h4><p>Step into the enchanted world of MLflow Projects — your gateway to reproducibility and collaboration. MLflow Projects helps us organize our magical code, data, and dependencies, ensuring that fellow enchanters can recreate our spells with ease. Together, we create a harmonious ecosystem of machine learning wonders!</p><p>Example:</p><pre>hand_digits_project/<br>│   MLproject<br>│   conda.yaml<br>│   train.py<br>│<br>└───data<br>│   │   digits.csv<br>│   │   ...<br>│   <br>└───models<br>    │   ...</pre><p><em>Learn more about MLflow Projects: </em><a href="https://www.mlflow.org/docs/latest/projects.html"><em>Official MLflow Projects Documentation</em></a></p><h4>3. MLflow Models: The Artifacts of Enchantment</h4><p>The final pillar, MLflow Models, allows us to turn our machine learning models into precious artifacts. With versioning and deployment support, we ensure that our magical creations stand the test of time and bring wonders to the world beyond our spellcasting chamber!</p><p>Example:</p><pre>import mlflow.sklearn<br>from sklearn.ensemble import RandomForestClassifier<br><br># Load the enchanted dataset<br># ...<br><br># Create and train the model<br>model = RandomForestClassifier(n_estimators=100, random_state=42)<br>model.fit(X_train, y_train)<br><br># Save the model as an MLflow artifact<br>with mlflow.start_run() as run:<br>    mlflow.log_param(&quot;n_estimators&quot;, 100)<br>    mlflow.sklearn.log_model(model, &quot;hand_digits_model&quot;)</pre><p><em>Learn more about MLflow Models: </em><a href="https://www.mlflow.org/docs/latest/models.html"><em>Official MLflow Models Documentation</em></a></p><h3>Unveiling the Enchantment: Hand Digits Recognition with MLflow</h3><p>Now, let’s immerse ourselves in the real enchantment of MLflow by creating a magical spell — hand digits recognition! Our mission is to train a model capable of deciphering handwritten digits, turning scribbles into meaningful predictions. Fear not; with MLflow by our side, we shall succeed!</p><p>Example:</p><pre>import mlflow<br>import mlflow.tensorflow<br>import tensorflow as tf<br>import plotly.express as px<br>import matplotlib.pyplot as plt<br>from tensorflow.keras import layers, models<br>from tensorflow.keras.datasets import mnist<br>import numpy as np<br><br># Load the MNIST dataset and normalize the pixel values<br>(train_images, train_labels), (test_images, test_labels) = mnist.load_data()<br>train_images, test_images = train_images / 255.0, test_images / 255.0<br><br># Define the Deep Neural Network (DNN) model using TensorFlow<br>def create_model():<br>    model = models.Sequential([<br>        layers.Flatten(input_shape=(28, 28)),<br>        layers.Dense(128, activation=&#39;relu&#39;),<br>        layers.Dense(64, activation=&#39;relu&#39;),<br>        layers.Dense(10, activation=&#39;softmax&#39;)<br>    ])<br>    return model<br><br># Training the DNN model<br>def train(model):<br>    model.compile(optimizer=&#39;adam&#39;,<br>                  loss=&#39;sparse_categorical_crossentropy&#39;,<br>                  metrics=[&#39;accuracy&#39;])<br>    history = model.fit(train_images, train_labels, epochs=5, batch_size=64, validation_data=(test_images, test_labels))<br>    return history<br><br># Save learning curve plot using Plotly<br>def save_learning_curve(history):<br>    # Create a Plotly figure for the learning curve plot<br>    fig = px.line(history.history, y=[&#39;accuracy&#39;, &#39;val_accuracy&#39;], labels={&#39;index&#39;: &#39;Epoch&#39;, &#39;value&#39;: &#39;Accuracy&#39;},<br>                  title=&#39;Learning Curve - Accuracy&#39;)<br>    # Save the plot as an HTML file<br>    fig.write_html(&quot;learning_curve.html&quot;)<br>    # Log the plot as an MLflow artifact<br>    mlflow.log_artifact(&quot;learning_curve.html&quot;)<br><br># Save the DNN model and its metrics using MLflow<br>def save_to_mlflow():<br>    with mlflow.start_run() as run:<br>        # Log the hyperparameters for the run<br>        mlflow.log_param(&quot;epochs&quot;, 5)<br>        mlflow.log_param(&quot;batch_size&quot;, 64)<br>        mlflow.log_param(&quot;learning_rate&quot;, 0.001)<br><br>        # Create and train the DNN model<br>        model = create_model()<br>        history = train(model)<br><br>        # Log the learning curve plot using Plotly and save it as an artifact<br>        save_learning_curve(history)<br><br>        # Log the training and validation accuracy metrics<br>        mlflow.log_metric(&quot;final_train_accuracy&quot;, history.history[&#39;accuracy&#39;][-1])<br>        mlflow.log_metric(&quot;final_val_accuracy&quot;, history.history[&#39;val_accuracy&#39;][-1])<br><br>        # Log the TensorFlow model<br>        mlflow.tensorflow.log_model(model, &quot;hand_digits_dnn_model&quot;)<br><br># Save the DNN model, parameters, and metrics using MLflow and Plotly plots<br>save_to_mlflow()</pre><p><em>TensorFlow Documentation: </em><a href="https://www.tensorflow.org/"><em>TensorFlow Official Documentation</em></a></p><p><em>MNIST Dataset Documentation: </em><a href="https://www.tensorflow.org/datasets/catalog/mnist"><em>MNIST Dataset on TensorFlow</em></a></p><p>In this code, we’ve used TensorFlow to create the Deep Neural Network (DNN) model for hand digits recognition, using the popular MNIST dataset. The training and evaluation processes are performed using TensorFlow APIs. MLflow is used to log the training parameters, such as the number of epochs, batch size, and learning rate. Additionally, the test accuracy metric is recorded during the model evaluation. The trained DNN model is saved as an MLflow artifact, enabling easy access and deployment. MLflow’s integration with TensorFlow makes it effortless to track, manage, and share TensorFlow models and their associated metadata, enhancing the MLOps experience.</p><h3>Tracking and Managing Experiments Remotely</h3><p>MLflow enables us to manage our experiments not only locally but also remotely! With MLflow Tracking, we can easily log our experiments to a remote tracking server, making it accessible from anywhere. This allows for seamless collaboration with other enchanters or sharing progress with curious observers!</p><p>To run MLflow with remote tracking, set the MLFLOW_TRACKING_URI environment variable to the desired tracking server URL:</p><pre>export MLFLOW_TRACKING_URI=http://your-remote-server:5000</pre><p>Now, when you start your MLflow runs, they will be tracked on the remote server:</p><pre>import mlflow<br><br># Start an MLflow tracking run (tracked remotely)<br>mlflow.start_run()<br><br># Log parameters and metrics<br># ...<br><br># End the run (results are now available on the remote server)<br>mlflow.end_run()</pre><p>With this magical capability, you can collaborate with other enchanters across the mystical realm of machine learning and share the wonders of your experiments!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2d0vfFWS4bC3kP4QyNK34A.png" /><figcaption>Parameters and Metrics got saved</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*07zVb_eJ7NEpkqQdRnBzKg.png" /><figcaption>Provides options for inferences</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rYzzpwC9qs93_S-7jfk-Kg.png" /><figcaption>Learning Curve Accuracies</figcaption></figure><p><em>Learn more about MLflow Tracking with Remote Servers: </em><a href="https://www.mlflow.org/docs/latest/tracking.html#remote-server-setup"><em>Remote Server Setup in MLflow</em></a></p><h3>Embracing MLOps with MLflow</h3><p>MLOps, the art of managing machine learning operations, is an enchanting practice where MLflow truly shines. By integrating MLflow into your MLOps workflow, you can enchant your machine learning pipelines, enabling seamless collaboration, reproducibility, and model deployment.</p><h4>1. Experiment Tracking in MLOps</h4><p>With MLflow Tracking, you can monitor, compare, and evaluate your experiments during the MLOps lifecycle. Track model performance, hyperparameters, and artifacts to understand the effectiveness of your machine learning spells and make informed decisions.</p><h4>2. Collaboration and Reproducibility</h4><p>MLflow Projects empower enchanters to collaborate seamlessly by organizing code, data, and dependencies. Embrace version control and ensure reproducibility across different environments, ensuring consistent results throughout the MLOps journey.</p><h4>3. Effortless Model Deployment</h4><p>Enchant the world by deploying your machine learning models effortlessly with MLflow Models. Package your models as artifacts, version them, and deploy them to production or any other environment with a wave of your wand.</p><h3>The Grand Finale: Unleashing the Power of MLflow and MLOps 🌟</h3><p>Congratulations, fellow enchanters! You have now mastered the art of MLflow and experienced its magic firsthand. With MLOps at your side, you wield the power to streamline machine learning operations, collaborate effectively, and deploy models with ease. Armed with these enchanting powers, you are ready to conquer the mystical realm of machine learning and create wonders that will leave a lasting impact!</p><h3>Unravel More Secrets of MLflow and MLOps</h3><p>To continue your magical journey with MLflow and MLOps, visit the official website at <a href="https://mlflow.org/">https://mlflow.org/</a>. There, you will find extensive resources, documentation, and tutorials to become a true sorcerer of MLOps.</p><p>MLflow GitHub Repository: <a href="https://github.com/mlflow/mlflow">MLflow on GitHub</a></p><p>MLOps Resources: <a href="https://www.mlflow.org/docs/latest/mlops.html">MLOps Resources</a></p><p>MLflow YouTube Channel: <a href="https://www.youtube.com/channel/UCR1-GEpyOPzT2AO4D_eifdw">MLflow YouTube Channel</a></p><p>May your machine learning endeavors be filled with endless enchantment and remarkable discoveries! Happy spell-casting and may your data forever be insightful and your models, truly magical! 🧙‍♂️✨</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a710592839" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/unleashing-the-magic-of-mlflow-an-enchanting-guide-to-machine-learning-and-mlops-2a710592839">Unleashing the Magic of MLflow: An Enchanting Guide to Machine Learning and MLOps 🚀🔮</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The ‘Nosy Neighbor’ Algorithm: kNN — Get Ready for k-Nearest Snoops (kNS)!]]></title>
            <link>https://medium.com/analytics-vidhya/the-nosy-neighbor-algorithm-knn-get-ready-for-k-nearest-snoops-kns-c5ae19663095?source=rss-1c823c9f4438------2</link>
            <guid isPermaLink="false">https://medium.com/p/c5ae19663095</guid>
            <category><![CDATA[k-nearest-neighbours]]></category>
            <category><![CDATA[nearest-neighbors]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[supervised-learning]]></category>
            <category><![CDATA[knn-algorithm]]></category>
            <dc:creator><![CDATA[Ashwin N]]></dc:creator>
            <pubDate>Mon, 24 Jul 2023 16:06:22 GMT</pubDate>
            <atom:updated>2023-07-24T16:06:22.050Z</atom:updated>
            <content:encoded><![CDATA[<h3>The ‘Nosy Neighbor’ Algorithm: kNN — Get Ready for k-Nearest Snoops (kNS)!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*-JLfewL0hicsMZ2TNBBqYg.jpeg" /><figcaption>Unfortunately, it’s not that kind of friendly neighbor! :)</figcaption></figure><p>Hey there, curious minds! Today, we’re diving into the world of machine learning with a twist — introducing the hilarious alternative name for k-Nearest Neighbors (kNN) algorithm: the ‘Nosy Neighbor’ algorithm, a.k.a. k-Nearest Snoops (kNS)!</p><h4>Scenario:</h4><p>Imagine living in a wacky neighborhood filled with quirky characters, where everyone is always poking their noses into each other’s business! 🕵️‍♀️</p><h4>The ‘Nosy Neighbor’ Algorithm Unveiled:</h4><p>Just like those nosy neighbors who know everything happening around them, k-Nearest Snoops (kNS) can’t help but peep into the lives of their closest buddies — the ‘k’ nearest neighbors! They use this juicy information to make smart predictions and decisions! 🕵️‍♂️📖</p><p><em>Step 1: The Snooping Begins!</em><br>Imagine these k-Nearest Snoops knocking on the doors of their closest neighbors to gather gossip — who’s cooking what, who’s painting their walls, and who’s hosting wild parties! They collect all the neighborhood secrets in a flash! 🚪🗣️</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*Xvv4JsfhSD_QIE665EVvhg.jpeg" /><figcaption>Snooping Around</figcaption></figure><p><em>Step 2: Decisions Based on Snooping Intel!</em><br>With all the scandalous data in their hands, these snoopers decide where you should hang out, which restaurants serve the best dishes, or even predict your movie preferences based on what your neighbors love! It’s like having a personalized neighborhood guide! 🍔🍿🎬</p><h4><strong>Advantages of k-Nearest Snoops (kNS):</strong></h4><p>1. <em>The Quickest Detectives:</em> Just like your nosy neighbors, kNS are super fast at getting their answers — no waiting around for them to find out the juicy details!<br>2. <em>No Private Data Required:</em> kNS doesn’t need any private information; they work solely based on what’s out in the open — just like those chatty neighbors in your favorite sitcom! 📺🍿<br>3. <em>Neighborhood Bonding:</em> Just like your neighborly friendships, kNS creates bonds with the ‘k’ closest data points, building a strong sense of community in the data world! 🤝💻</p><h4>Why k-Nearest Snoops Can’t Always Snoop Around:</h4><p>1. <em>Loud and Chatty:</em> Sometimes, those nosy neighbors can be too loud, causing a ruckus in the neighborhood. Similarly, kNS might not work well with noisy or irrelevant data.<br>2. <em>Limited Knowledge:</em> While kNS can gather local intel, they don’t have a broader perspective of the entire neighborhood, which can lead to biased predictions.<br>3. <em>Scared of Change: </em>kNS might struggle to handle drastic changes in the neighborhood, just like those neighbors who can’t handle a surprise renovation next door! 😱</p><p>While we’ve had a bit of fun, let’s now dive into some serious notes about kNN and understand its strengths and weaknesses.</p><h4>Serious Strengths:</h4><ol><li><em>Intuitive and Simple:</em> kNN’s simplicity allows easy implementation and interpretation, making it an excellent starting point for newcomers in machine learning.</li><li><em>Non-parametric Approach:</em> kNN doesn’t assume any underlying data distribution, making it versatile and applicable to various types of data.</li><li><em>Adaptable to New Data:</em> Since it relies on local information from nearby data points, kNN can adapt to changes in the dataset, making it suitable for dynamic environments.</li></ol><h4>Serious Limitations:</h4><ol><li><em>Computationally Demanding:</em> As the dataset grows, kNN’s computational cost increases significantly, affecting its scalability for larger datasets.</li><li><em>Sensitive to Noise and Irrelevant Features:</em> Noisy data or irrelevant features can distort the results, impacting the accuracy of kNN’s predictions.</li><li><em>No Insights into the Data: </em>kNN doesn’t provide insights into the underlying relationships between features, limiting its interpretability.</li></ol><h3>In Conclusion:</h3><p>So there you have it — the ‘Nosy Neighbor’ algorithm, a.k.a. k-Nearest Snoops (kNS)! It’s a fun and powerful tool to explore the data neighborhood and make predictions based on what your closest data buddies are up to! Remember, in the world of machine learning, sometimes a touch of humor can help us unravel the mysteries of complex algorithms! Happy snooping, I mean, happy machine learning adventures! 🏠👀🔍🤖</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5ae19663095" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/the-nosy-neighbor-algorithm-knn-get-ready-for-k-nearest-snoops-kns-c5ae19663095">The ‘Nosy Neighbor’ Algorithm: kNN — Get Ready for k-Nearest Snoops (kNS)!</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Boosting Neural Network Training: The Power of Batch Normalization]]></title>
            <link>https://medium.com/analytics-vidhya/boosting-neural-network-training-the-power-of-batch-normalization-362d0d88b4b1?source=rss-1c823c9f4438------2</link>
            <guid isPermaLink="false">https://medium.com/p/362d0d88b4b1</guid>
            <category><![CDATA[gradient-descent]]></category>
            <category><![CDATA[batch-normalization]]></category>
            <category><![CDATA[deep-neural-networks]]></category>
            <category><![CDATA[keras]]></category>
            <dc:creator><![CDATA[Ashwin N]]></dc:creator>
            <pubDate>Fri, 21 Jul 2023 09:50:33 GMT</pubDate>
            <atom:updated>2023-07-21T09:58:40.401Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/248/1*Bu8uRS7VrMFQu_SVXHUMAQ.jpeg" /></figure><p><strong>Batch normalization</strong> is a technique used in neural networks to improve the training process and overall performance of the model. It addresses the internal covariate shift problem, which occurs during the training of deep neural networks. The internal covariate shift refers to the change in the distribution of layer inputs as the model learns, which can slow down the training process and make it more difficult for the model to converge.</p><h4>Why Batch Normalization?</h4><blockquote>To relate this to a daily example, let’s consider a chef preparing a meal for a large party. The chef needs to ensure that each dish is cooked perfectly, which can be challenging when cooking for a large number of guests. In this analogy, the neural network is the chef, and the guests represent the training data.</blockquote><blockquote>Imagine the chef starts cooking with a recipe that hasn’t been adjusted for the number of guests. As the chef progresses and prepares each dish, the ingredients’ proportions may not be suitable for the current number of guests, leading to inconsistencies in the taste and quality of the dishes. This situation can be time-consuming, and the chef might have to keep adjusting the recipe after every new batch of guests arrives.</blockquote><p>In the context of neural networks, without batch normalization, each layer’s input distribution changes with each batch of data during training. This means that the neural network has to constantly adapt to the varying input distributions, making the training process less stable and slower.</p><blockquote>Now, let’s introduce batch normalization into the cooking scenario. Batch normalization for the chef would be like having a kitchen assistant who continuously ensures that the ingredient proportions for each dish are appropriately adjusted based on the total number of guests. With batch normalization, the chef can maintain consistent dish quality and reduce the need for constant adjustments.</blockquote><p>Similarly, in neural networks, batch normalization normalizes the input for each layer with respect to the mean and standard deviation calculated over each mini-batch of data. This helps to stabilize the learning process by keeping the input distributions consistent and reducing internal covariate shift. As a result, the neural network can train more efficiently, converge faster, and perform better on the test data.</p><blockquote>This is similar to how the chef can efficiently cook a large meal with consistent quality using batch normalization.</blockquote><h4>Exploding Gradient Problem</h4><p>When training a deep neural network, a common issue is the <em>exploding gradient problem</em>, which occurs when the weights of the network become too large. This can cause wild fluctuations in weight values, and in extreme cases, the loss function may return <em>NaN</em>, indicating an overflow error.</p><p>To prevent the exploding gradient problem, one of the reasons for scaling input data is to ensure a stable start to training. By scaling the input, such as pixel values from 0–255 to between -1 and 1, we avoid the immediate creation of huge activation values that could lead to exploding gradients. However, as the network trains and the weights move away from their initial values, a phenomenon called <em>covariate shift </em>can occur.</p><p>Covariate shift is akin to carrying a tall pile of books and getting hit by gusts of wind. As you compensate for the wind’s force, the pile becomes slightly more unstable with each gust. Similarly, in neural networks, each layer assumes consistent input distributions from the layer beneath during updates. But if the activation distributions shift significantly in a certain direction, this can lead to runaway weight values and a collapse of the network.</p><p>In summary, scaling input data helps prevent exploding gradients at the start of training, but covariate shift can still occur during later iterations, potentially causing instability in weight values and overall network collapse. Addressing covariate shift is essential to maintain a stable and well-performing neural network.</p><h4>Batch Normalization Process</h4><p>Let’s go through a step-by-step example of the batch normalization process with a simple neural network.</p><p>Suppose we have a neural network with two hidden layers and an output layer. The architecture is as follows:</p><blockquote>Input Layer (4 features) -&gt; Hidden Layer 1 (6 neurons) -&gt; Hidden Layer 2 (4 neurons) -&gt; Output Layer (3 classes)</blockquote><p>1. Forward Pass:<br>Let’s assume we have a mini-batch of 8 samples as input to the neural network. During the forward pass, the input data flows through the network, and activations are calculated for each layer.</p><p>a. Input Layer:<br>The mini-batch of 8 samples is fed into the input layer, and the activations for the hidden layers are computed.</p><p>b. Hidden Layer 1:<br>The activations from the input layer are multiplied by the corresponding weights for Hidden Layer 1. Then, the bias is added to each neuron, and the output is passed through an activation function (e.g., ReLU).</p><p>c. Hidden Layer 2:<br>Similar to Hidden Layer 1, the activations from Hidden Layer 1 are multiplied by the corresponding weights for Hidden Layer 2. The bias is added, and the output is passed through an activation function.</p><p>d. Output Layer:<br>Finally, the activations from Hidden Layer 2 are multiplied by the weights for the Output Layer, and the bias is added. The output is then processed through the activation function (e.g., softmax) to get the probabilities for each class.</p><p>2. Batch Normalization:<br>Now, before calculating the loss and updating the weights during backpropagation, we apply batch normalization to stabilize and improve the training process.</p><p>a. Calculate Batch Mean and Variance:<br>For each neuron in the hidden layers, we calculate the mean and variance of the activations for the entire mini-batch.</p><p>b. Normalize the Activations:<br>Using the batch mean and variance, we normalize the activations for each neuron by subtracting the mean and dividing by the square root of the variance. This centers the activations around zero and scales them.</p><p>c. Scale and Shift (Gamma and Beta):<br>To allow the model to learn the optimal scale and shift for each normalized activation, we introduce two learnable parameters: gamma (scale) and beta (shift). These parameters are learned during training.</p><p>d. Apply Gamma and Beta:<br>The normalized activations are multiplied by the gamma parameter and then added to the beta parameter. This step reintroduces flexibility into the network and allows it to represent the original activations better.</p><p>3. Backward Pass:<br>After the batch normalization step, we continue with the backward pass, calculating the gradients and updating the weights using techniques like gradient descent.</p><p>By applying batch normalization, the network can train more effectively and converge faster. It reduces the internal covariate shift, making it easier to find an appropriate learning rate and avoid the exploding gradient problem, leading to better and more stable training of the neural network.</p><h4>References:</h4><p>Sergey Ioffe and Christian Szegedy, “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,” 11 February 2015, <a href="https://arxiv.org/abs/1502.03167.">https://arxiv.org/abs/1502.03167.</a></p><p>#AI #DeepLearning #BatchNormalization #NeuralNetworks #Tech #MachineLearning</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=362d0d88b4b1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/boosting-neural-network-training-the-power-of-batch-normalization-362d0d88b4b1">Boosting Neural Network Training: The Power of Batch Normalization</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fine Tuning BERT with CoLA Dataset]]></title>
            <link>https://medium.com/analytics-vidhya/fine-tuning-bert-1d0b1937a172?source=rss-1c823c9f4438------2</link>
            <guid isPermaLink="false">https://medium.com/p/1d0b1937a172</guid>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[bert]]></category>
            <category><![CDATA[cola]]></category>
            <dc:creator><![CDATA[Ashwin N]]></dc:creator>
            <pubDate>Tue, 17 Jan 2023 15:50:06 GMT</pubDate>
            <atom:updated>2023-01-17T16:02:04.259Z</atom:updated>
            <content:encoded><![CDATA[<p>Think of the original Transformer as a model built with LEGO® bricks. The construction set contains bricks such as encoders, decoders, embedding layers, positional encoding methods, multi-head attention layers, masked multi-head attention layers, post-layer normalization, feed-forward sub-layers, and linear output layers. The bricks come in various sizes and forms. You can spend hours building all sorts of models using the same building kit! Some constructions will only require some of the bricks. Other constructions will add a new piece, just like when we obtain additional bricks for a model built using LEGO® components.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*060mgzrqhdp1pL4p.png" /><figcaption>Figure 1: BERT Peak</figcaption></figure><p>BERT added a new piece to the Transformer building kit: a bidirectional multi-head attention sub-layer. Because BERT is a departure from the LSTM-based approaches to NLP, hence we can place BERT at the top of the Transformer and eventually calling it as “<strong>BERT Peak</strong>”.</p><p>When we humans are having problems understanding a sentence, we do not just look at the past words. BERT, like us, looks at all the words in the same sentence at the same time.</p><p>In this chapter, we will first explore the architecture of Bidirectional Encoder Representations from Transformers (BERT). BERT only uses the blocks of the encoders of the Transformer in a novel way and does not use the decoder stack.</p><p>Then we will fine-tune a pretrained BERT model. The BERT model we will fine-tune was trained by a third party and uploaded to Hugging Face. Transformers can be pretrained. Then, a pretrained BERT, for example, can be fine-tuned on several NLP tasks. We will go through this fascinating experience of downstream Transformer usage using Hugging Face modules.</p><p>This article will cover following topics:</p><ul><li>Preparing the pretraining environment</li><li>Defining pretraining encoder layers</li><li>Defining fine-tuning</li><li>Downstream multitasking</li><li>Building a fine-tuning BERT model</li><li>Loading an accessibility judgement dataset</li><li>Creating attention masks</li><li>BERT model configuration</li><li>Measuring the performance of the fine-tuned model</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*unYMZTWPF-ok40FvmtHlRg.jpeg" /></figure><h4>CoLA Dataset</h4><p>We will fine-tune a BERT model to predict the downstream task of <em>Acceptability Judgements</em> and measure the predictions with the <strong>Matthews Correlation Coefficient (MCC)</strong>. The BERT model we will fine-tune will be trained on <strong>The Corpus of Linguistic Acceptability (CoLA)</strong>.</p><h3>Activating the GPU</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/889/0*W6MdLe0DRYD6ngst.jpg" /></figure><p>Pretraining a multi-head attention transformer model requires the parallel processing GPUs can provide.</p><pre>import tensorflow as tf<br>device_name = tf.test.gpu_device_name()<br>if device_name != ‘/device:GPU:0’:<br> raise SystemError(‘GPU device not found’)<br>print(‘Found GPU at: {}’.format(device_name)) <br><br># Found GPU at: /device:GPU:0</pre><h3>Installing the Hugging Face PyTorch interface for BERT</h3><p>Hugging Face provides a pretrained BERT model. Hugging Face developed a base class named <strong>PreTrainedModel</strong>. By installing this class, we can load a model from a pretrained model configuration.</p><p>Hugging Face provides modules in TensorFlow and PyTorch. I recommend that a developer feels comfortable with both environments.</p><p>We will install the modules required as follows:</p><pre># !pip install -q transformers<br>import transformers</pre><pre>import torch<br>import datetime<br>from torch.utils.data import TensorDataset, DataLoader, RandomSampler, <br>SequentialSampler<br>from keras.preprocessing.sequence import pad_sequences<br>from sklearn.model_selection import train_test_split<br>from transformers import BertTokenizer, BertConfig, BertModel<br>from transformers import AdamW, BertForSequenceClassification, <br>get_linear_schedule_with_warmup<br>from tqdm import tqdm, trange</pre><p>Let us import other common libraries</p><pre>import pandas as pd<br>import io<br>import numpy as np<br>import matplotlib.pyplot as plt<br>import plotly.express as px<br>import plotly.graph_objects as go</pre><h4>Specifying CUDA as the device for torch</h4><p>We will now specify that torch uses the <strong>Compute Unified Device Architecture (CUDA)</strong> to put the parallel computing power of the NVIDIA card to work for our multi-head attention model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/269/1*xpiA_osWV-j6l8xwZRGzIQ.jpeg" /></figure><h3>1. Loading Dataset</h3><p>We will now load the <strong>CoLA</strong> based on the <strong>Warstadt et al. (2018)</strong> paper. More info on this dataset would be found on <a href="https://huggingface.co/datasets/shivkumarganesh/CoLA">https://huggingface.co/datasets/shivkumarganesh/CoLA</a></p><pre>df = pd.read_csv(“/kaggle/input/cola-public/in_domain_train.tsv”, <br>delimiter=’\t’, header=None,<br>names=[‘sentence_source’, ‘label’, ‘label_notes’, ‘sentence’])<br><br>df.shape<br><br># (8551, 4)</pre><p>Each sample in the .tsv files contains four tab-separated columns:</p><ul><li><strong>sentence_source</strong>: the source of the sentence (code)</li><li><strong>label</strong>: the label (0=unacceptable, 1=acceptable)</li><li><strong>label_notes</strong>: the label annotated by the author</li><li><strong>sentence</strong>: the sentence to be classified</li></ul><p>Creating sentences, label lists, and adding BERT tokens:</p><pre># Adding special tokens to the sentences<br>sentences = [“[CLS] “ + sentence + “ [SEP]” for sentence in df.sentence.values]<br>labels = df.label.values</pre><h3>2. Activating the BERT Tokenizer</h3><p>In this section, we will initialize a pretrained BERT tokenizer. This will save the time it would take to train it from scratch.</p><p>The program selects an uncased tokenizer, activates it, and displays the first tokenized sentence:</p><pre>tokenizer = BertTokenizer.from_pretrained(‘bert-base-uncased’, <br> do_lower_case=True)<br>tokenized_texts = [tokenizer.tokenize(sent) for sent in sentences]<br><br>print (&quot;Tokenize the first sentence:&quot;)<br>print (tokenized_texts[0])<br><br># Tokenize the first sentence:<br># [&#39;[CLS]&#39;, &#39;our&#39;, &#39;friends&#39;, &#39;won&#39;, &quot;&#39;&quot;, &#39;t&#39;, &#39;buy&#39;, &#39;this&#39;, &#39;analysis&#39;, &#39;,&#39;, &#39;let&#39;, &#39;alone&#39;, &#39;the&#39;, &#39;next&#39;, &#39;one&#39;, &#39;we&#39;, &#39;propose&#39;, &#39;.&#39;, &#39;[SEP]&#39;]</pre><h3>3. Processing the data</h3><pre># Now let us find the longest sentence in the dataset<br># and use that as the maximum length for padding<br>max_len = 0<br>for sent in tokenized_texts:<br> if len(sent) &gt; max_len:<br> max_len = len(sent)<br> <br>print(“Maximum length is: {}”.format(max_len))<br><br># Maximum length is: 47</pre><p>The longest sequence in our training set is <strong>47</strong>, but we’ll leave room on the end anyway. Set maximum length of a sequence to <em>128 </em>and the sequences are padded.</p><h4>Let’s convert tokens to IDs:</h4><pre>max_len = 128<br><br># Use the BERT tokenizer to convert the tokens to their index numbers <br># in the BERT vocabulary<br>input_ids = [tokenizer.convert_tokens_to_ids(x) for x in tokenized_texts]<br><br># Pad our input tokens<br>input_ids = pad_sequences(input_ids, maxlen=max_len, dtype=&quot;long&quot;, <br> truncating=&quot;post&quot;, padding=&quot;post&quot;)</pre><h4>Creating attention masks</h4><p>Now comes a tricky part of the process. We padded the sequences in the previous cell. But we want to prevent the model from performing attention on those padded tokens!</p><p>The idea is to apply a mask with a value of 1 for each token, which will be followed by 0s for padding:</p><pre># Create attention masks<br>attention_masks = []<br><br># Create a mask of 1s for each token followed by 0s for padding<br>for seq in input_ids:<br> seq_mask = [float(i&gt;0) for i in seq]<br> attention_masks.append(seq_mask)</pre><h4>Splitting and converting to Torch Tensors</h4><pre>train_inputs, validation_inputs, train_labels, validation_labels = train_test_split(input_ids, <br> labels, random_state=42, test_size=0.1)<br><br>train_masks, validation_masks, _, _ = train_test_split(attention_masks, <br> input_ids, random_state=42, test_size=0.1)</pre><p>The data is ready to be trained, but it still needs to be adapted to torch.</p><pre># Convert all of our data into torch tensors, the required datatype for our model<br>train_inputs = torch.tensor(train_inputs)<br>validation_inputs = torch.tensor(validation_inputs)<br>train_labels = torch.tensor(train_labels)<br>validation_labels = torch.tensor(validation_labels)<br>train_masks = torch.tensor(train_masks)<br>validation_masks = torch.tensor(validation_masks)</pre><h4>Selecting a batch size and creating an iterator</h4><p>Now, the program selects a batch size and creates an iterator. The iterator is a clever way of avoiding a loop that would load all the data in memory. The iterator, coupled with the torch DataLoader, can batch train huge datasets without crashing the memory of the machine.</p><p>In this model, the batch size is 32:</p><pre># Select a batch size for training. For fine-tuning BERT on a specific task,<br># the authors recommend a batch size of 16 or 32<br>batch_size = 32<br><br># Create an iterator of our data with torch DataLoader.<br># This helps save on memory during training because, unlike a for loop,<br># with an iterator the entire dataset does not need to be loaded into memory<br>train_data = TensorDataset(train_inputs, train_masks, train_labels)<br>train_sampler = RandomSampler(train_data)<br>train_dataloader = DataLoader(train_data, sampler=train_sampler, <br> batch_size=batch_size)<br><br>validation_data = TensorDataset(validation_inputs, <br> validation_masks, validation_labels)<br><br>validation_sampler = SequentialSampler(validation_data)<br><br>validation_dataloader = DataLoader(validation_data, <br> sampler=validation_sampler,<br> batch_size=batch_size)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/318/0*uL4z938uIF-vesqx" /></figure><p>The data has been processed and is all set. The program can now load and configure the BERT model.</p><h3>4. Loading the Hugging Face BERT uncased base model</h3><pre># Load BertForSequenceClassification, the pretrained BERT model.<br>model = BertForSequenceClassification.from_pretrained(“bert-base-uncased”, <br>num_labels=2).to(device)<br># model.cuda()</pre><h4>Optimizer grouped parameters</h4><p>Initialize the optimizer for the model’s parameters. Finetuning a model begins with initializing the pretrained model parameter values (not their names).</p><p>The parameters of the optimizer include a weight decay rate to avoid overfitting, and some parameters are filtered.</p><p>The goal is to prepare the model’s parameters for the training loop:</p><pre># Optimize model parameters<br>param_optimizer = list(model.named_parameters())<br>no_decay = [‘bias’, ‘LayerNorm.weight’]<br><br># Separate the ‘weight’ parameters from the ‘bias’ parameters.<br># — For the ‘weight’ parameters, this specifies a ‘weight_decay_rate’<br># of 0.01.<br># — For the ‘bias’ parameters, the ‘weight_decay_rate’ is 0.0.<br>optimizer_grouped_parameters = [<br> # Filter for all parameters which are not bias or LayerNorm.<br> {‘params’: [p for n, p in param_optimizer if not any(nd in n for nd in no_decay)],<br> ‘weight_decay_rate’: 0.1},<br> # Filter for parameters which are bias or LayerNorm.<br> {‘params’: [p for n, p in param_optimizer if any(nd in n for nd in no_decay)],<br> ‘weight_decay_rate’: 0.0}<br>]</pre><h4>The hyperparameters for the training loop</h4><p>The hyperparameters for the training loop are critical, though they seem innocuous. Adam will activate weight decay and also go through a warm-up phase, for example.</p><p>The learning rate (<em>lr</em>) and warm-up rate (<em>warmup</em>) should be set to a very small value early in the optimization phase and gradually increase after a certain number of iterations. This avoids large gradients and overshooting the optimization goals.</p><p>Some researchers argue that the gradients at the output level of the sub-layers before layer normalization do not require a warm-up rate. Solving this problem requires many experimental runs.</p><pre>epochs = 10<br><br>optimizer = AdamW(optimizer_grouped_parameters,<br> lr=5e-5,<br> eps=1e-8)<br><br># Total number of training steps is number of batches * number of epochs.<br>total_steps = len(train_dataloader) * epochs<br><br># Create the learning rate scheduler.<br>scheduler = get_linear_schedule_with_warmup(optimizer,<br> num_warmup_steps=0, <br> num_training_steps=total_steps)</pre><h4>Accuracy measurement helper function</h4><pre>def flat_accuracy(preds, labels):<br> pred_flat = np.argmax(preds, axis=1).flatten()<br> labels_flat = labels.flatten()<br> return np.sum(pred_flat == labels_flat) / len(labels_flat)<br><br># Function to calculate the time elapsed<br>def format_time(elapsed):<br> # Round to the nearest second.<br> elapsed_rounded = int(round((elapsed)))<br> # Format as hh:mm:ss<br> return str(datetime.timedelta(seconds=elapsed_rounded))</pre><p>The data is ready, the parameters are ready. It’s time to activate the training loop!</p><h4>The training loop</h4><p>The training loop follows standard learning processes. The number of epochs is set to 4, and there is a measurement for loss and accuracy, which will be plotted. The training loop uses the dataloader load and train batches. The training process is measured and evaluated.</p><p>The code starts by initializing the <em>train_loss_set</em>, which will store the loss and accuracy, which will be plotted. It starts training its epochs and runs a standard training loop, as shown in the following excerpt:</p><pre>#The Training Loop<br>t = []<br><br># Store our loss and accuracy for plotting<br>train_loss_set = []<br><br># trange is a tqdm wrapper around the normal python range<br>for _ in trange(epochs, desc=”Epoch”):<br> <br> # Training<br> <br> # Set our model to training mode (as opposed to evaluation mode)<br> model.train()<br> <br> # Tracking variables<br> tr_loss = 0<br> nb_tr_examples, nb_tr_steps = 0, 0<br> <br> # Train the data for one epoch<br> for step, batch in enumerate(train_dataloader):<br> # Add batch to GPU<br> batch = tuple(t.to(device) for t in batch)<br># batch = tuple(t for t in batch)<br> # Unpack the inputs from our dataloader<br> b_input_ids, b_input_mask, b_labels = batch<br> # Clear out the gradients (by default they accumulate)<br> optimizer.zero_grad()<br> # Forward pass<br> outputs = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask, labels=b_labels)<br> loss = outputs[‘loss’]<br> train_loss_set.append(loss.item()) <br> # Backward pass<br> loss.backward()<br> # Update parameters and take a step using the computed gradient<br> optimizer.step()<br><br># Update the learning rate.<br> scheduler.step()<br><br># Update tracking variables<br> tr_loss += loss.item()<br> nb_tr_examples += b_input_ids.size(0)<br> nb_tr_steps += 1<br><br>print(&quot;Train loss: {}&quot;.format(tr_loss/nb_tr_steps))<br> <br># Validation<br># Put model in evaluation mode to evaluate loss on the validation set<br>model.eval()<br><br># Tracking variables <br>eval_loss, eval_accuracy = 0, 0<br>nb_eval_steps, nb_eval_examples = 0, 0<br><br># Evaluate data for one epoch<br>for batch in validation_dataloader:<br> # Add batch to GPU<br> batch = tuple(t.to(device) for t in batch)<br># Unpack the inputs from our dataloader<br> b_input_ids, b_input_mask, b_labels = batch<br> # Telling the model not to compute or store gradients, saving memory and speeding up validation<br> with torch.no_grad():<br> # Forward pass, calculate logit predictions<br> logits = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask)<br> <br> # Move logits and labels to CPU<br> logits = logits[&#39;logits&#39;].detach().cpu().numpy()<br> label_ids = b_labels.to(&#39;cpu&#39;).numpy()<br><br> tmp_eval_accuracy = flat_accuracy(logits, label_ids)<br><br> eval_accuracy += tmp_eval_accuracy<br> nb_eval_steps += 1<br><br>print(&quot;Validation Accuracy: {}&quot;.format(eval_accuracy/nb_eval_steps))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/196/1*HWSIdIK3l801-cXWFr-L3A.jpeg" /></figure><pre>###<br><br>Epoch:   0%|          | 0/10 [00:00&lt;?, ?it/s]<br>Train loss: 0.4866324247660973<br>Epoch:  10%|█         | 1/10 [01:31&lt;13:47, 91.95s/it]<br>Validation Accuracy: 0.8032407407407407<br>Train loss: 0.2726411976198438<br>Epoch:  20%|██        | 2/10 [03:02&lt;12:09, 91.13s/it]<br>Validation Accuracy: 0.8063271604938271<br>Train loss: 0.15303814302186997<br>Epoch:  30%|███       | 3/10 [04:32&lt;10:35, 90.81s/it]<br>Validation Accuracy: 0.8097993827160493<br>Train loss: 0.08949965936483437<br>Epoch:  40%|████      | 4/10 [06:03&lt;09:03, 90.65s/it]<br>Validation Accuracy: 0.8171296296296297<br>Train loss: 0.05705505336878273<br>Epoch:  50%|█████     | 5/10 [07:33&lt;07:32, 90.55s/it]<br>Validation Accuracy: 0.8206018518518519<br>Train loss: 0.043270988243968224<br>Epoch:  60%|██████    | 6/10 [09:04&lt;06:02, 90.51s/it]<br>Validation Accuracy: 0.8182870370370371<br>Train loss: 0.02780074113690105<br>Epoch:  70%|███████   | 7/10 [10:34&lt;04:31, 90.46s/it]<br>Validation Accuracy: 0.8179012345679012<br>Train loss: 0.023910069168657372<br>Epoch:  80%|████████  | 8/10 [12:04&lt;03:00, 90.44s/it]<br>Validation Accuracy: 0.8240740740740741<br>Train loss: 0.013467846731983774<br>Epoch:  90%|█████████ | 9/10 [13:35&lt;01:30, 90.42s/it]<br>Validation Accuracy: 0.8225308641975309<br>Train loss: 0.008648735464669386<br>Epoch: 100%|██████████| 10/10 [15:05&lt;00:00, 90.56s/it]<br>Validation Accuracy: 0.8213734567901234<br>###</pre><p>The model is trained. We can now display the training evaluation.</p><h4>Training Evaluation</h4><p>The loss and accuracy values were stored in <em>train_loss_set </em>as defined at the beginning of the training loop.</p><pre>plt.figure(figsize=(15, 8))<br>plt.title(“Training Loss”)<br>plt.xlabel(“Loss per Batch”)<br>plt.ylabel(“Loss”)<br>plt.plot(train_loss_set)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/884/1*_mESZ2qJ8CwEicZLarCqww.png" /></figure><p>The model has been fine-tuned. We can now run predictions.</p><h3>4. Prediction and Evaluation on Hold-out dataset</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TeHlZTa7r_2kl_BV.gif" /></figure><p>The BERT downstream model was trained with the <em>in_domain_train.tsv</em> dataset. The program will now make predictions using the holdout dataset contained in the <em>out_of_domain_dev.tsv</em> file. The goal is to predict whether sentence is grammatically correct.</p><pre>test_df = pd.read_csv(“/kaggle/input/cola-public/out_of_domain_dev.tsv”, <br>delimiter=”\t”, header=None, <br> names = [‘sentence_source’, ‘label’, ‘label_notes’, ‘sentence’])<br><br># Create sentence and label lists<br>sentences = test_df.sentence.values<br><br># We need to add special tokens at the beginning and end of each<br># sentence for BERT to work properly<br>sentences = [&quot;[CLS] &quot; + sentence + &quot; [SEP]&quot; for sentence in sentences]<br>labels = test_df.label.values<br>tokenized_texts = [tokenizer.tokenize(sent) for sent in sentences]<br><br>../...</pre><p>The program then runs batch predictions using the <em>dataloader</em>:</p><pre># Put model in evaluation mode<br>model.eval()<br><br># Tracking variables <br>predictions, true_labels = [], []<br><br># Predict<br>for batch in prediction_dataloader:<br> # Add batch to GPU<br> batch = tuple(t.to(device) for t in batch)<br> <br> # unpack the inputs from dataloader<br> b_input_ids, b_input_mask, b_labels = batch<br> <br> with torch.no_grad():<br> logits = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask)<br> <br> # Move logits and labels to CPU<br> logits = logits[&#39;logits&#39;].detach().cpu().numpy()<br> label_ids = b_labels.to(&#39;cpu&#39;).numpy()<br><br> # Store predictions and true labels<br> predictions.append(logits)<br> true_labels.append(label_ids)</pre><h4>Evaluating using Matthews Correlation Coefficient</h4><p>The <strong>Matthews Correlation Coefficient (MCC)</strong> was initially designed to measure the quality of binary classifications and can be modified to be a multi-class correlation coefficient. A two-class classification can be made with four probabilities at each prediction:</p><ul><li>TP = True Positive</li><li>TN = True Negative</li><li>FP = False Positive</li><li>FN = False Negative</li></ul><p><em>Brian W. Matthews</em>, a biochemist, designed it in 1975, inspired by his predecessors’ phi function. Since then it has evolved into various formats such as the following one:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/428/1*Svy9rM78vpfhTGQ2oZ4qoA.png" /></figure><p>The value produced by MCC is between -1 and +1. +1 is the maximum positive value of a prediction. -1 is an inverse prediction. 0 is an average random prediction.<br>GLUE evaluates Linguistic Acceptability with MCC.</p><p>MCC is imported from sklearn.metrics:</p><pre># Import and evaluate each test batch using Matthew’s correlation <br># coefficient<br>from sklearn.metrics import matthews_corrcoef<br><br>matthews_set = []<br><br>for i in range(len(true_labels)):<br> matthews = matthews_corrcoef(true_labels[i], <br> np.argmax(predictions[i], axis=1).flatten())<br> matthews_set.append(matthews)<br><br>matthews_set<br><br>###<br>[0.049286405809014416,<br>-0.050964719143762556,<br>0.4040950971038548,<br>0.30508307783296046,<br>0.2321726094326961,<br>0.5510387687779837,<br>0.4879500364742666,<br>0.6831300510639733,<br>0.9229582069908973,<br>0.7704873741021288,<br>1.0,<br>0.8333333333333334,<br>0.8150678894028793,<br>0.7948717948717948,<br>0.4547940268270977,<br>0.6457765999379483,<br>0.0]<br>###</pre><p>The output produces MCC values between -1 and +1 as expected. Almost all the MCC values are positive, which is good news. Let’s see what the evaluation is for the whole dataset.</p><pre># Flatten the predictions and true values for aggregate Matthew’s<br># evaluation on the whole dataset<br><br>flat_predictions = [item for sublist in predictions for item in sublist]<br>flat_predictions = np.argmax(flat_predictions, axis=1).flatten()<br>flat_true_labels = [item for sublist in true_labels for item in sublist]<br><br>matthews_corrcoef(flat_true_labels, flat_predictions)<br><br># 0.5657382497020534</pre><p>The output confirms that the MCC is positive, which shows that there is a correlation for this model and dataset.</p><h3>5. Summary</h3><p>BERT brings bidirectional attention to transformers. Predicting sequences from left to right and masking the future tokens to train a model has serious limitations. If the masked sequence contains the meaning we are looking for, the model will produce errors. BERT attends to all of the tokens of a sequence at the same time.</p><p>We explored the architecture of BERT, which only uses the encoder stack of transformers. BERT was designed as a two-step framework. The first step of the framework is to pretrain a model. The second step is to fine-tune the model. We built a fine-tuning BERT model for an Acceptability Judgement downstream task. The finetuning process went through all phases of the process. First, we loaded the dataset and loaded the necessary pretrained modules of the model. Then the model was trained, and its performance measured.</p><p>Fine-tuning a pretrained model takes fewer machine resources than training downstream tasks from scratch. Fine-tuned models can perform a variety of tasks. BERT proves that we can pretrain a model on two tasks only, which is remarkable in itself. But producing a multitask fine-tuned model based on the trained parameters of the BERT pretrained model is extraordinary.</p><h3>6. References</h3><ol><li>The complete implementation will be found in <a href="https://www.kaggle.com/ashwinnaidu/finetuningbertwithcoladataset">https://www.kaggle.com/ashwinnaidu/finetuningbertwithcoladataset</a></li><li>For few other info: <a href="https://github.com/PacktPublishing">https://github.com/PacktPublishing</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1d0b1937a172" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/fine-tuning-bert-1d0b1937a172">Fine Tuning BERT with CoLA Dataset</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating a TF-IDF Model from Scratch in Python]]></title>
            <link>https://medium.com/analytics-vidhya/creating-a-tf-idf-model-from-scratch-in-python-71047f16494e?source=rss-1c823c9f4438------2</link>
            <guid isPermaLink="false">https://medium.com/p/71047f16494e</guid>
            <category><![CDATA[tf-idf]]></category>
            <category><![CDATA[term-frequency]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Ashwin N]]></dc:creator>
            <pubDate>Tue, 26 Jul 2022 09:55:02 GMT</pubDate>
            <atom:updated>2022-07-26T09:55:02.548Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction to TF-IDF</h3><p>TF-IDF is a method of information retrieval that is used to rank the importance of words in a document. It is based on the idea that words that appear in a document more often are more relevant to the document.</p><p>TF-IDF is the product of Term Frequency and Inverse Document Frequency. Here’s the formula for TF-IDF calculation.</p><p><em>TF-IDF = Term Frequency (TF) * Inverse Document Frequency (IDF)</em></p><h4>1. What is Term Frequency?</h4><p>It is the measure of the frequency of words in a document. It is the ratio of the number of times the word appears in a document compared to the total number of words in that document.</p><h4>2. What is Inverse Document Frequency?</h4><p>It is the measure of how much information the word provides about the topic of the document. It is the log of the ratio of the number of documents to the number of documents containing the word.</p><p>We take log of this ratio because when the corpus becomes large IDF values can get large causing it to explode hence taking log will dampen this effect. We cannot divide by 0, we smoothen the value by adding 1 to the denominator.</p><p><em>idf(t) = log(N/(df + 1))</em></p><h4>Advantage of TF-IDF</h4><p>This method removes the drawbacks faced by the <a href="https://www.analyticsvidhya.com/blog/2021/08/a-friendly-guide-to-nlp-bag-of-words-with-python-example/">Bag of Words model</a>. It does not assign equal value to all the words, hence important words that occur a few times will be assigned high weights.</p><p>Bag of Words just creates a set of vectors containing the count of word occurrences in the document (reviews), while the TF-IDF model <strong>contains information on the more important words and the less important ones as well</strong>.</p><h3>Step by Step Implementation of the TF-IDF Model</h3><h3>1. Preprocess the data</h3><p>We will start with preprocessing the text data, and make a vocabulary set of the words in our training data and assign a unique index for each word in the set. But first let us consider below sample text.</p><pre>sample_text = [&#39;Topic sentences are similar to mini thesis statements. Like a thesis statement&#39;, &#39;a topic sentence has a specific main point. Whereas the thesis is the main point of the essay, the topic sentence is the main point of the paragraph.               Like the thesis statement, a topic sentence has a unifying function. <br>But a thesis statement or topic sentence alone doesn’t guarantee unity.&#39;, &#39;An essay is unified if all the paragraphs relate to the thesis,&#39;whereas a paragraph is unified if all the sentences relate to the topic sentence.&#39;]</pre><p>Now tokenize the sample text and create unique set of words.</p><pre>from nltk.tokenize import word_tokenize</pre><pre>sentences = []<br>word_set = []<br><br>for sent in sample_text:<br>    words = [word.lower() for word in word_tokenize(sent) if word.isalpha()]<br>    sentences.append(words)<br>    for word in words:<br>        if word not in word_set:<br>            word_set.append(word)</pre><pre># Set of words<br>word_set = set(word_set)<br># total documents in our corpus<br>total_docs = len(sample_text)<br>print(&#39;Total documents: &#39;, total_docs)<br>print(&#39;Total words: &#39;, len(word_set))<br>-----------------------------------------------------------<br>Total documents:  3<br>Total words:  35</pre><p>Now let us index each word from vocabulary. This will be later used to map the word to the vector.</p><pre>word_index = {}<br>for i, word in enumerate(word_set):<br>    word_index[word] = i</pre><p>Create a dictionary to keep the count of the number of documents containing the given word.</p><pre>def count_dict(sentences):<br><em>    </em>count_dict = {}<br>    for word in word_set:<br>        count_dict[word] = 0<br>    for sent in sentences:<br>        for word in sent:<br>            count_dict[word] += 1<br>    return count_dict</pre><pre>word_count = count_dict(sentences)<br>print(word_count)<br>-----------------------------------------------------------<br>{&#39;paragraphs&#39;: 1, &#39;topic&#39;: 6, &#39;thesis&#39;: 6, &#39;alone&#39;: 1, &#39;similar&#39;: 1, &#39;sentences&#39;: 2, &#39;if&#39;: 2, &#39;has&#39;: 2, &#39;of&#39;: 2, &#39;unity&#39;: 1, &#39;all&#39;: 2, &#39;main&#39;: 3, &#39;relate&#39;: 2, &#39;sentence&#39;: 5, &#39;paragraph&#39;: 1, &#39;doesn&#39;: 1, &#39;mini&#39;: 1, &#39;the&#39;: 11, &#39;unified&#39;: 2, &#39;statement&#39;: 3, &#39;specific&#39;: 1, &#39;to&#39;: 3, &#39;but&#39;: 1, &#39;or&#39;: 1, &#39;an&#39;: 1, &#39;a&#39;: 7, &#39;point&#39;: 3, &#39;unifying&#39;: 1, &#39;function&#39;: 1, &#39;whereas&#39;: 2, &#39;essay&#39;: 2, &#39;guarantee&#39;: 1, &#39;is&#39;: 4, &#39;t&#39;: 1, &#39;are&#39;: 1}</pre><p>Now let us calculate the Term Frequency (TF) of each word in the corpus</p><pre>def term_frequency(document, word):<br><em>    </em>N = len(document)<br>    occurance = len([token for token in document if token == word])<br>    return occurance / N</pre><p>For inverse document frequency (IDF) for each word in the document:</p><pre>def inverse_document_frequency(word):<br><em>    </em>try:<br>        word_occurance = word_count[word] + 1<br>    except:<br>        word_occurance = 1<br>    return np.log(total_docs / word_occurance)</pre><p>The last part is to combine both TF and IDF</p><pre>def tf_idf(sentence):<em><br>    </em>vec = np.zeros((len(word_set),))<br>    for word in sentence:<br>        tf = term_frequency(sentence, word)<br>        idf = inverse_document_frequency(word)<br>        vec[word_index[word]] = tf * idf<br>    return vec</pre><p>Now let us apply TF-IDF to our sample texts.</p><pre>vectors = []<br>for sent in sentences:<br>    vectors.append(tf_idf(sent))<br><br>print(vectors)<br>-----------------------------------------------------------<br>[ 0.0144809  -0.01027436 -0.09078191  0.          0.          0.<br>  0.         -0.14853154  0.0144809   0.          0.          0.<br>  0.          0.0144809   0.         -0.06052128  0.          0.<br> -0.01027436  0.0144809   0.         -0.02054872 -0.02475526  0.<br>  0.          0.         -0.10508885  0.          0.          0.<br>  0.          0.         -0.02054872  0.         -0.01824377]</pre><p>Now, if the model encounters an unknown word other than the vocab, it will give us a Key error as we did not account for any unknown tokens.</p><p>The purpose of this article is to demonstrate how TF-IDF actually works under the hood.</p><p>The implementation for above code is in <a href="https://github.com/ashushekar/TF-IDF-Model.git">github repository</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=71047f16494e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/creating-a-tf-idf-model-from-scratch-in-python-71047f16494e">Creating a TF-IDF Model from Scratch in Python</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>