<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dipansh on Medium]]></title>
        <description><![CDATA[Stories by Dipansh on Medium]]></description>
        <link>https://medium.com/@thepansh?source=rss-5c0d00399f5f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*fsrfUMWHzxnJgYODOV33lA.jpeg</url>
            <title>Stories by Dipansh on Medium</title>
            <link>https://medium.com/@thepansh?source=rss-5c0d00399f5f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 16:41:16 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@thepansh/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Between Miles and Miracles: What Surviving a Cardiac Arrest Taught Me as an Athlete]]></title>
            <link>https://thepansh.medium.com/between-miles-and-miracles-what-surviving-a-cardiac-arrest-taught-me-as-an-athlete-dff6ae4dd2f8?source=rss-5c0d00399f5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/dff6ae4dd2f8</guid>
            <category><![CDATA[health]]></category>
            <category><![CDATA[running]]></category>
            <category><![CDATA[life]]></category>
            <category><![CDATA[fitness]]></category>
            <category><![CDATA[sports]]></category>
            <dc:creator><![CDATA[Dipansh]]></dc:creator>
            <pubDate>Wed, 11 Jun 2025 16:29:59 GMT</pubDate>
            <atom:updated>2025-08-05T06:42:03.183Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Between Miles and Miracles: </strong>A Runner’s Second Chance</h3><p><em>“The fall is part of the journey. But so is getting back up , quietly, clumsily, courageously.”</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o-3JUhe96ywSoSYCOYksHw.png" /><figcaption>Original ECG graph from the hospital</figcaption></figure><p>In a grey, cloudy room, my body lay motionless. Everything felt hazy, like I was floating inside a lucid dream. I was lying on a hospital bed, surrounded by doctors and nurses trying to bring me back , talking to me, urging me to respond. And I was watching it all, but not from my own eyes.</p><p>My soul was somewhere else, somewhere weightless, suspended between reality and imagination. Away from my body. Detached. Somewhere in the middle — between life and death. And strangely, it felt peaceful.</p><blockquote>Was this it? Maybe this was how it ends. I felt complete. I felt ready.</blockquote><p>And then, I heard a voice. A nurse, maybe. Or something inside me.</p><blockquote>‘You’re an athlete, right? You can push through this. You’ve got this, don’t you?’</blockquote><p>And something shifted. Or maybe everything did. That’s when I realised, this wasn’t a dream. Something real had happened. Something terrifying. I didn’t know what, or how, but one thing became clear: <strong>The athlete in me wasn’t going down without a fight.</strong><br>The part of me that had run marathons. The part that had trained through pain, adversities, and self-doubt.<br>The part that didn’t know how to give up.</p><blockquote>And in that moment, every cell in my body decided — <strong>SURVIVE</strong>.</blockquote><p>For my mom, who had poured her whole life into raising me and my sister. For my sister who raised me like her own child. For the girl who was holding my hand, crying, begging me to come back. For the sport that simply taught me how to live.</p><h3>The Day Everything Changed</h3><p><strong>April 12, 2025</strong> was supposed to be just another Saturday. Easy long run, catch up with friends, recover for the next run.</p><p>The night before was quiet. I felt a little more tired than usual, but I didn’t think much of it. Normally, I would have paid attention to dinner - carbs, fuel, the usual prep. But that night, I slacked off. Smaller dinner. Slept early.</p><p>Woke up as usual. Tied my laces. Wore my favourite Nike running vest. Had my electrolytes and pre run meal. And Jogged to the starting point to meet my friends. The vibe was easy. Just another long run on the weekend.</p><p>The city was still yawning awake. My friends and I were cruising.</p><blockquote>The rhythm of our feet, the easy pace, the chatter — everything felt right.</blockquote><p>I wasn’t at my best, I’d been feeling a little off lately, not quite where I used to be , but that morning, something felt good. The easy pace felt easy again. My body felt like it was syncing back into momentum. And whatever discomfort did show up, I barely noticed it, probably drowned out by the endless stream of unsolicited wisdom from my coach running beside me. :D</p><p>Around the 15th kilometre mark, as we neared the end of the run, I remember cracking a joke.</p><blockquote>“Maybe I spoke too soon,” I laughed. “Guess I’m not quite back yet.”</blockquote><p>And then —</p><p><strong>Blank.</strong></p><p>I stopped for a breather. And everything went dark.</p><blockquote>I don’t remember falling. I don’t remember collapsing.</blockquote><blockquote>But I’ve been told I did.</blockquote><h3>Before the Fall: The Athlete Within</h3><p>That blank moment was the exact opposite of everything I’ve trained for my whole life.</p><p>But maybe, to understand what really happened that day, you need to understand who I was before it.</p><p>Raised by a single parent — my mother had always been a Super Woman. Whether it was the absence of support from my father’s family or the scarcity of resources, one thing she engraved in me was simple:</p><blockquote>“Sincere hard work never goes in vain. You might not see the reward immediately, but it always arrives.”</blockquote><p>That mindset became my foundation. <strong>Badminton</strong> was my first love. It taught me everything — discipline, grit, humility. Win or lose, I kept showing up, training harder, chasing the next match. I tried to chase it professionally, but somewhere, I lacked that final edge of obsession required to make it all the way.</p><p>Even when I pivoted to engineering, I never stopped being an athlete.</p><p>And then, a few years later, <strong><em>running</em></strong> walked into my life. 👣</p><p>The moment I ran my first race, I felt free. Unstoppable.</p><p>I ran my first 10K, then a marathon less than 6 months later. That fire — the one I felt back on the badminton court — was back. Only this time, it burned brighter.</p><blockquote>I wasn’t just chasing personal bests. I was chasing a higher version of myself. A version who defied limits daily.</blockquote><p>I ran Mumbai. Then the prestigious <strong>Berlin Marathon</strong>. All within the first year.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lF2uGz2lZYXWt_6lfvnZXg.jpeg" /><figcaption>At the finish line of the Berlin Marathon 2024</figcaption></figure><blockquote>Running became more than a sport. It became a lifestyle.</blockquote><p>My life revolved around training, mileage, and race day routines. What I built on the road — discipline, resilience, structure — I took into my professional life as a Data Scientist.</p><p>Honestly, work felt easy. Because by the time my day began, I had already faced the hard part — on the road, in the dark, alone, pushing through.</p><blockquote>Travel plans? Built around races.</blockquote><blockquote>Social events? Often skipped for long runs or recovery.</blockquote><p>Was it extreme? Maybe. But I loved it.</p><blockquote>Because for the first time in my life, I wasn’t chasing balance. I was chasing purpose.</blockquote><p><strong>I was beginning to live like a professional athlete.</strong></p><p>But even that came with its shadows. From obsessive carb loading, sugar highs, isolation from family, and placing too much of my worth into the sport, I was still learning to manage that line. And thankfully, I had people who understood me, who held space for me through all of it.</p><blockquote>A sound mind. A healthy athlete’s body. A content, purpose-driven soul.</blockquote><blockquote>What more could I ask for?</blockquote><p>And then, it all stopped.</p><blockquote>The same body that carried me through national tournaments, brutal training cycles, and race after race… gave up.</blockquote><p>No warning. No hard signs. It just… stopped.</p><h3>Why did it happen?</h3><p>I’ve run marathons. I’ve pushed through pain. But this time, I didn’t even see it coming. And I still don’t fully remember the moment my life nearly slipped away.</p><blockquote>The more I try to question it, the fewer answers I get. Maybe because there are none. Nothing clear. Nothing concrete I can point to.</blockquote><p>This wasn’t supposed to happen.</p><p>Not to someone like me. Not to someone who had spent years treating their body like a temple, pushing limits, living clean, showing up.</p><p>But <em>maybe that’s the irony.</em></p><p>Maybe that’s why it hit so hard.</p><h3>The Hands That Pulled Me Back: A Story of Miracles</h3><blockquote>From this point on, the story stops being mine — <br>and becomes the story of those who refused to let me go.</blockquote><p>It was just another Saturday long run. The roads were quiet, so we chose to run on the open street instead of the pavement — separated by a high railing. A small decision that would end up meaning everything.</p><p>Around 90 minutes later, I collapsed. No warning. No pulse. A cardiac arrest in the middle of the run.</p><p>One friend caught me before I hit the ground. The other friend (who’s also my coach) was recently trained inCPR and he began CPR instantly — acting on instinct. Something for which I’ll forever be grateful.</p><blockquote>In the middle of that empty road, two friends fought to keep me alive.</blockquote><p>An auto driver pulled over. Thankfully, we were on the open road, not behind the railing — so carrying my lifeless body was quicker and easier. Without a word, he asked his passengers to step out and rushed us to the nearest hospital. No money asked. Just disappeared after dropping us off.</p><blockquote>To this day, we haven’t found him.. A godsend. An angel in disguise.</blockquote><p>At the hospital, I still had no pulse. The doctors were ready to let go — until one asked, <em>“How long has it been?”</em> When they heard it had only been five minutes, they called in senior staff.</p><blockquote>A top cardiologist happened to be on-site. Another miracle.</blockquote><p>My partner arrived and saw me through the glass — blue, lifeless. She held my hand and cried. Somehow… I twitched. A flicker. <strong>She reached me before I could fully slip away.</strong></p><p>An hour later, my mother was flying back after hearing the news through a call from my phone. While she was mid-air, they got my heart beating again. But no one knew if my brain had survived.</p><p>When she arrived, the doctor said<strong> I had roughly a 20% chance</strong>. She stood beside my unconscious body, whispered my name. Begged me to respond. I didn’t open my eyes — but somehow tears rolled down my eyes.</p><blockquote>Somewhere deep inside, I could hear her.</blockquote><p>That night, a nurse asked me softly: <em>“What makes you an athlete?”</em><br> And somewhere deep inside, a voice answered:<br> <em>“It’s because I never quit.”</em></p><p>The next morning, I woke up. Fully conscious. Breathing. Alive. Like nothing had happened — and yet everything had.</p><p>They kept me under observation. My vitals were perfect. My spirit, unshaken. Even the staff I hadn’t met came by just to say:<br> <em>&quot;You’re a miracle.&quot;</em></p><p>But deep down, I knew it wasn’t just luck.</p><blockquote>It was love. It was quick thinking of my friends. It was CPR. It was an auto driver’s kindness. It was fate. And maybe, just maybe, it was a life that still had something left to do.</blockquote><h3>The Long Road Back: Confusion, Fear, and Finding a New Path</h3><p>Coming back to life sounds dramatic — but no one tells you what comes after.</p><blockquote><em>It’s not joy. It’s not even relief. It’s confusion. The haunting question: </em>What just happened?</blockquote><p>They told me I survived. That I was unconscious for nearly two days. That I had no heartbeat. That I was gone. And yet — here I was. Breathing. Awake. Alive.</p><p>But I didn’t feel victorious.</p><blockquote>I felt hollow. Like I had come back into a life that no longer fully belonged to me.</blockquote><p>The hospital discharged me with good news. <strong>No major long-term implications.</strong> I could, eventually, return to my life. Slowly. Gradually. Cautiously.</p><p>And in that moment, I felt a wave of relief — the kind that really overwhelms you.</p><blockquote>But was that really possible? Could I truly go back?</blockquote><p>In the days that followed, my body began whispering answers before my mind could form questions. My energy was gone. Every small activity left me drained. My heart would beat harder after just a few steps. My mother — my forever pillar — poured herself into taking care of me. Meals, nutrients, rest, warmth. She never left my side.</p><p>And slowly, painfully slowly, progress came.</p><p>Took me some time to be able to stand up for long. Then a walk. Then stairs without needing to take the lift.</p><p>A month later, I could walk freely. My fatigue had lessened. But so had my certainty.</p><p>Because a new kind of fear had crept in — the fear of trying again.</p><p>What if I can’t push anymore?</p><p>What if I collapse again?</p><blockquote>What if I never get to be that version of me — the one who trained like a warrior and lived like an athlete?</blockquote><p>I’ve never known another way to live.</p><p>My whole life — from my teens on a badminton court to miles on the track — has all been about giving my all. About never stopping short. About showing up on tired legs and proving to myself that limits are just illusions.</p><blockquote>It’s not ambition. It’s identity.</blockquote><p>And now, that identity feels… uncertain.</p><blockquote>But if <strong>sport</strong> has taught me anything — it’s this: <em>when the rules change, you adapt</em>. <em>When the game shifts, you adjust your strategy.</em></blockquote><p>You learn to evolve. And that’s where I am now.</p><p>I’m learning a new game.</p><p>A slower game.</p><p>A safer game.</p><p>But still — <strong>My game</strong>.</p><p>Because I’m not done.</p><p>I don’t know when I’ll run again. Or if I’ll race again.</p><p>But I do know this: my life was given back to me — not by chance, but by love. By kindness. By Medical science. By a circle of people who refused to give up.</p><p>And I owe it to them — and to myself — to find a new version of the athlete I used to be.</p><blockquote>Not broken. Not defeated. Just… changed.</blockquote><p>And somehow, I will figure it out.</p><p>Not out of pressure. Not out of fear.</p><p>But out of deep, unwavering love for life — the messy, miraculous, unpredictable thing it is.</p><h3>Final Thoughts</h3><blockquote>To every athlete, every dreamer, every person living at 110% — just remember: you’re not less for resting. You’re not weak for needing help.</blockquote><p>If you’ve ever felt like life knocked the wind out of you — maybe not physically, but emotionally, mentally — I want you to know this:</p><blockquote>You can come back. Maybe not to the same place, the same pace, the same you. But you can come back.</blockquote><p>And sometimes, what comes after might be even more powerful.</p><h3>Join ME</h3><p>Medium: <a href="https://medium.com/@thepansh">https://medium.com/@thepansh</a></p><p>Linkedin: <a href="https://www.linkedin.com/in/dipansh-iiit/">https://www.linkedin.com/in/dipansh-iiit/</a></p><p>Instagram: <a href="https://www.instagram.com/thepansh/">https://www.instagram.com/thepansh/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dff6ae4dd2f8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Best (fastest) ways to import CSV files in python  for production environments (pandas, csv, dask)]]></title>
            <link>https://medium.com/analytics-vidhya/best-fastest-ways-to-import-csv-files-in-python-for-production-environments-pandas-csv-dask-1407be42893e?source=rss-5c0d00399f5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/1407be42893e</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[csv]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[data-analysis]]></category>
            <dc:creator><![CDATA[Dipansh]]></dc:creator>
            <pubDate>Sun, 21 Jun 2020 19:14:36 GMT</pubDate>
            <atom:updated>2020-06-23T10:13:12.725Z</atom:updated>
            <content:encoded><![CDATA[<h3>Best (fastest) ways to import CSV files in python for production environments (pandas, csv, dask)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/681/1*oD9e75JjXMZ3QtHmzNK3jg.png" /></figure><p>In the life of an Engineer working around data, working on csv files is an everyday task. While dealing with analytics tasks, we sometimes choose to ignore the time consumed in importing the data, but it becomes rather important in production environments. And it becomes somewhat problematic while dealing with large files. It turns out, as it will become clear further down in this post that if you face the issue of larger input files (like several several hundred MBs or more) it is probably best to use some sort of partitioning or parallel processing for the import.</p><p>This post will not focus on super accurate benchmarking, but rather on how to perform each of these CSV import options and how to run some simple benchmarking ourselves. We can draw some conclusions afterwards on the basis of this casual analysis. This post focuses only on CSV files, as these are quite often the format of choice for tabular data.</p><p>The options that I will cover here are: <strong>csv.DictReader()</strong> , <strong>pandas.read_csv()</strong>, <strong>dask.dataframe.read_csv(). </strong>This is by no means an exhaustive list of all methods for CSV import. However, it is probably a somewhat exhaustive list of the most common methods to do that.</p><p><strong>How I measured the time efficiency</strong></p><p>Here I am interested in how much time it takes to import a CSV file using various methods. This is pretty straightforward. Get the time before and after a certain action and then get the difference in seconds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/948/1*0pfOJ7sWmhveM6gsYLmZ0Q.png" /></figure><p>The results given by this method can be very volatile. Every split second different processes run on the computer with different intensity and therefore they interfere with the results of the measurement. For the purposes of this post we can ignore this interference, since we are more interested in time efficiency in relative rather than absolute terms.</p><p>Let us move on to the main main comparison.</p><p><strong>csv.DictReader()</strong></p><p>The DictReader is a Python class which maps the data read as a dictionary, whose keys, unless specified are the first row of the CSV. All values in the subsequent rows will be dictionary values and can be accessed with the respective dictionary key. However, these values will be imported as strings. I personally do not use this method a lot for data analysis tasks, but it is valuable in certain other cases (working with the JSON format or dealing with csvs through the terminal). You would go about it in the following way:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/1*ImpOaxU8UIlZlZbiI76UcQ.png" /></figure><p>The snippet above will create the CSV object (data).</p><p><strong>pandas.read_csv()</strong></p><p>pandas is a popular library that has data structures suited for tabular data with heterogeneously-typed columns (integers, floats, strings like the random data we already created), as well as time series.Pandas are my preferred data structure for data analysis tasks as pandas are very easy to manipulate and transform. In a sense they are very similar to data.frame and data.table in R.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/676/1*Y6tz9h8-dmix7Sr1unEIkQ.png" /></figure><p>According to many opinions I have come across, the pandas package is very well optimized and thus very efficient in many tasks including basic ones as data import.</p><p>An importnat point here is that pandas.read_csv() can be run with the chunksize option. This will break the input file into chunks instead of loading the whole file into memory. This will reduce the pressure on memory for large input files and given an optimal chunksize found through trial and error, there can be significant increase in efficiency. The code below will split the input file into chunks of 100 000 lines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/1*b-hHc_ax-DMzhKKiDFQ72Q.png" /></figure><p>Another important functionality present in pandas is the <strong>usecols </strong>parameter. This allows you to read only the selected columns to be read and skip the non-relevant columns in the data. This can save us time that was earlier used to read all the unnecessary columns.</p><p>I found <a href="http://gouthamanbalaraman.com/blog/distributed-processing-pandas.html">this post</a> by Goutham Balaraman very useful, where it is also explained how separating the input file in chunks can help speed up further processing as well. Later he edited his post saying that he finds the dask.dataframe superior to this method, which brings us to the next method.</p><p><strong>dask.dataframe</strong></p><p>The <a href="http://dask.pydata.org/en/latest/dataframe.html">dask.dataframe</a> is a collection of smaller pandas data frames split by the index (the row labels used for identification of data), which can be processed in parallel on a single machine or on multiple machines on a cluster. I learnt about dask from this useful <a href="http://pythondata.com/dask-large-csv-python/">post</a> by Eric Brown.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*rvSB04ABsMAUqHo2CsDxmg.png" /></figure><p>Apparently, unlike pandas with dask the data is not fully loaded into memory, but is ready to be processed. Also certain opperations can be performed again without loading the whole dataset into memory. Another advantage is that the most functions used with pandas can be also use with dask. The differences arise from the parallel nature of dask.</p><p><strong>Benchmarking</strong></p><p>I perform the comparison for the Python methods with the following code all of which was already covered above:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XHApwbcLRxQKwd63_Myo9g.png" /></figure><p>The csv file used for the above code has around 500,000 rows and has a size of 420 MB.</p><p>The results I get are the following:</p><blockquote><strong>csv.DictReader</strong> took <strong>0.000013709068298339844 seconds</strong><br><strong>pd.read_csv</strong> took <strong>11.0141019821167 seconds</strong><br><strong>pd.read_csv</strong> with <strong>chunksize</strong> took <strong>11.9807119369507 seconds</strong><br><strong>dask.dataframe</strong> took <strong>0.079848103523254395 seconds</strong></blockquote><p>csv.DictReader is by far the fastest, but as I pointed out earlier it imports everything as strings, while the other methods try to guess the data types of each column separately and possibly do multiple other validations upon import. This means that if you use csv.DictReader for data analysis tasks you will possibly have to do a number of transformations on it before you can analyze the contents. But in cases when you know that the columns present in the data are all in string format already, csv.DictReader is the method to go for.</p><p>Ignoring csv.DictReader, dask.dataframe is by far the quickest method. This is understandable since it does not load the whole data set to memory, which the pandas methods do.</p><p><strong>Summary</strong></p><p>Pandas is very popular for data science work and has integration with a lot of other libraries. I would recommend looking into the chunksize option, if you are dealing with a large dataset and you are low on RAM. Dask is an option which gives tremendous improvements in import speed. However, I have not used it very much on data analysis tasks, but from what I understand it should be quite the same as pandas, which is promising.</p><p>Csv.DictReader remains the quickest option, but as I mentioned earlier I would not recommend it for data analysis tasks. Possibly working with the JSON format is one such area where this method will be quite useful. In production environments, when you have already analyzed the data, say using pandas and just need to use a final preprocessed data once in a while to either retrain the model or to upload the updated data to a search index like elasticsearch index, using csv.DictReader to import the data makes more sense as it is way faster than the other options.</p><p>This was a short overview of what I observed while working recently. Please reach out to me if you have any suggestions.</p><p>Cheers!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1407be42893e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/best-fastest-ways-to-import-csv-files-in-python-for-production-environments-pandas-csv-dask-1407be42893e">Best (fastest) ways to import CSV files in python  for production environments (pandas, csv, dask)</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Text Classification with BERT using Transformers for long text inputs]]></title>
            <link>https://medium.com/analytics-vidhya/text-classification-with-bert-using-transformers-for-long-text-inputs-f54833994dfd?source=rss-5c0d00399f5f------2</link>
            <guid isPermaLink="false">https://medium.com/p/f54833994dfd</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Dipansh]]></dc:creator>
            <pubDate>Sun, 31 May 2020 11:18:24 GMT</pubDate>
            <atom:updated>2021-11-26T17:30:43.242Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/610/1*SjM_gC1Qgj75xmfpNjWzaw.png" /><figcaption>Bidirectional Encoder Representations from Transformers</figcaption></figure><p>Text classification has been one of the most popular topics in NLP and with the advancement of research in NLP over the last few years, we have seen some great methodologies to solve the problem. In this blog, we will solve a text classification problem using BERT (Bidirectional Encoder Representations from Transformers). We will use the Google Play app reviews dataset consisting of app reviews, tagged with either positive or negative sentiment — i.e., how a user or customer feels about the app.</p><p>We’ll learn how to fine-tune BERT for sentiment analysis after doing the required text preprocessing (special tokens, padding, and attention masks) and then building a Sentiment Classifier using the amazing Transformers library by Hugging Face!</p><h3>WHAT IS BERT?</h3><p>BERT (introduced in <a href="https://arxiv.org/abs/1810.04805">this paper</a>) stands for Bidirectional Encoder Representations from Transformers. If you don’t know what most of that means — you’ve come to the right place! Let’s unpack the main ideas:</p><ul><li>Bidirectional — to understand the text you’re looking you’ll have to look back (at the previous words) and forward (at the next words)</li><li>Transformers — The <a href="https://arxiv.org/abs/1706.03762">Attention Is All You Need</a> paper presented the Transformer model. The Transformer reads entire sequences of tokens at once. In a sense, the model is non-directional, while LSTMs read sequentially (left-to-right or right-to-left). The attention mechanism allows for learning contextual relations between words (e.g. his in a sentence refers to Jim).</li><li>(Pre-trained) contextualized word embeddings — <a href="https://arxiv.org/abs/1802.05365v2">The ELMO paper</a> introduced a way to encode words based on their meaning/context. Nails has multiple meanings — fingernails and metal nails.</li></ul><p>BERT was trained by masking 15% of the tokens with the goal to guess them. An additional objective was to predict the next sentence. Let’s look at examples of these tasks:</p><h3>Masked LM (MLM)</h3><p>The idea here is “simple”: Randomly mask out 15% of the words in the input — replacing them with a [MASK] token — run the entire sequence through the BERT attention based encoder and then predict only the masked words, based on the context provided by the other non-masked words in the sequence. However, there is a problem with this naive masking approach — the model only tries to predict when the [MASK] token is present in the input, while we want the model to try to predict the correct tokens regardless of what token is present in the input. To deal with this issue, out of the 15% of the tokens selected for masking:</p><ul><li>80% of the tokens are actually replaced with the token [MASK].</li><li>10% of the time tokens are replaced with a random token.</li><li>10% of the time tokens are left unchanged.</li></ul><p>While training the BERT loss function considers only the prediction of the masked tokens and ignores the prediction of the non-masked ones. This results in a model that converges much more slowly than left-to-right or right-to-left models.</p><h3>Next Sentence Prediction (NSP)</h3><p>In order to understand the <em>relationship</em> between two sentences, BERT training process also uses the next sentence prediction. A pre-trained model with this kind of understanding is relevant for tasks like question answering. During training, the model gets as input pairs of sentences and it learns to predict if the second sentence is the next sentence in the original text as well.</p><p>As we have seen earlier, BERT separates sentences with a special [SEP] token. During training the model is fed with two input sentences at a time such that:</p><ul><li>50% of the time the second sentence comes after the first one.</li><li>50% of the time it is a random sentence from the full corpus.</li></ul><p>BERT is then required to predict whether the second sentence is random or not, with the assumption that the random sentence will be disconnected from the first sentence:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/454/0*x4ujVUsyHQKdNoBF" /></figure><p>To predict if the second sentence is connected to the first one or not, basically, the complete input sequence goes through the Transformer based model, the output of the [CLS] token is transformed into a 2×1 shaped vector using a simple classification layer, and the IsNext-Label is assigned using softmax.</p><p>The model is trained with both Masked LM and Next Sentence Prediction together. This is to minimize the combined loss function of the two strategies — <em>“together is better”</em>.</p><h3>BERT for long text</h3><p>One of the limitations of BERT is on the application when you have long inputs because, in BERT, the self-attention layer has a <strong>quadratic complexity</strong> O(n²) in terms of the <strong>sequence length n</strong> (see this<a href="https://arxiv.org/abs/1810.04805"> link</a>). In this post, I followed the <strong>main ideas of </strong><a href="https://arxiv.org/abs/1910.10781"><strong>this paper</strong></a> in order to know how to overcome this limitation, when you want to use BERT over long sequences of text.</p><h3>Data Preprocessing</h3><p>You might already know that Machine Learning models don’t work with raw text. You need to convert text to numbers (of some sort). BERT requires even more attention (of course!). Here are the requirements:</p><ul><li>Add special tokens to separate sentences and do classification</li><li>Pass sequences of constant length (introduce padding)</li><li>Create an array of 0s (pad token) and 1s (real token) called <em>attention mask</em></li></ul><p>The Transformers library provides a wide variety of Transformer models (including BERT). It works with TensorFlow and PyTorch! It also includes prebuilt tokenisers that do the heavy lifting for us!</p><p>Let’s check our data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/527/1*FSKzLenvaoLFnablDd8_IA.png" /></figure><blockquote><em>You can use a cased and uncased version of BERT and tokenizer. I’ve experimented with both. The cased version works better. Intuitively, that makes sense, since “BAD” might convey more sentiment than “bad”.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yHs93Cfu8-6zZPZZ3gCYpw.png" /></figure><p>Let’s load a pre-trained <a href="https://huggingface.co/transformers/model_doc/bert.html#berttokenizer">BertTokenizer</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fKOIp2PngdPjaGqPDSzBOQ.png" /></figure><blockquote><strong>tokenizer.tokenize </strong>converts the text to tokens and <strong>tokenizer.convert_tokens_to_ids</strong> converts tokens to unique integers.</blockquote><p>Some special tokens added by BERT are: [SEP], [CLS] , [PAD].</p><p>BERT understands tokens that were in the training set. Everything else can be encoded using the [UNK] (unknown) token.</p><p>To get a better understanding of the text preprocessing part and the code snippets for everything step by step, you can follow this amazing <a href="https://www.curiousily.com/posts/sentiment-analysis-with-bert-and-hugging-face-using-pytorch-and-python/">blog </a>by Venelin Valkov.</p><h3>Format the data for BERT model</h3><p>In this article as <a href="https://arxiv.org/abs/1910.10781">the paper</a> suggests, we are going to segment the input into smaller text and feed each of them into BERT, it means for each row, we will split the text in order to have some smaller text (200 words long each ), for example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*_kelyboWi0UgJOHg1366vQ.png" /></figure><p>We must split it into a chunk of 200 words each, with 50 words overlapped, just for example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/637/1*EC8yura6v7PelepmQswllQ.png" /></figure><p>The following function can be used to split the data:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*AlxKytGLdIDovThayimiwQ.png" /></figure><p>Applying this function to the review_text column of the dataset would help us get the dataset where every row has a list of string of around 200-word length.</p><p><strong>For every 200-length chunk, we extracted a representation vector from BERT of size 768 each</strong></p><h3>Sentiment Classification with BERT and Hugging Face</h3><p>We have all building blocks required to create a PyTorch dataset. Let’s discuss all the steps involved further.</p><ol><li><strong>Preparing the text data to be used for classification:</strong></li></ol><p>This step involves specifying all the major inputs required by BERT model which are <strong>text, input_ids, attention_mask and targets.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*mPSW2lWZvdRua8YQ7Ds62w.png" /></figure><p><strong>2. Splitting the data into train and test:</strong></p><p>It is always better to split the data into train and test datasets to evaluate the model on the test dataset in the end.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/994/1*YxX61kAHdWyifcjBs_c3mQ.png" /></figure><p><strong>3. Loading the data</strong></p><p>We also need to create a couple of data loaders and create a helper function for the same. A sample data loader function can be like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/710/1*KfcMMSPZXAnQGU2FPAAWgA.png" /></figure><p><strong>4. Loading the BERT Model</strong></p><p>There are a lot of helpers that make using BERT easy with the Transformers library. Depending on the task you might want to use <a href="https://huggingface.co/transformers/model_doc/bert.html#bertforsequenceclassification">BertForSequenceClassification</a>, <a href="https://huggingface.co/transformers/model_doc/bert.html#bertforquestionanswering">BertForQuestionAnswering</a> or something else. We’ll use the basic <a href="https://huggingface.co/transformers/model_doc/bert.html#bertmodel">BertModel</a> and build our sentiment classifier on top of it. Let’s load the model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/683/1*UmTQ_uErqbXr3ERFDpgqMg.png" /></figure><p><strong>5. Create the Sentiment Classifier model, which is adding a single new layer to the neural network that will be trained to adapt BERT to our task.</strong> Please refer to the <a href="https://github.com/dipansh-girdhar/Text-classification-for-long-text/blob/master/bert_doc_classification.py">SentimentClassifier </a>class in my GitHub repo.</p><p><strong>6. Train the model</strong></p><p>To reproduce the training procedure from the BERT paper, we’ll use the <a href="https://huggingface.co/transformers/main_classes/optimizer_schedules.html#adamw">AdamW</a> optimizer provided by Hugging Face. It corrects weight decay, so it’s similar to the original paper.</p><p>The BERT authors have some recommendations for fine-tuning:</p><ul><li>Batch size: 16, 32</li><li>Learning rate (Adam): 5e-5, 3e-5, 2e-5</li><li>Number of epochs: 2, 3, 4</li></ul><p>Note that increasing the batch size reduces the training time significantly, but gives you lower accuracy.</p><p><strong>7. Evaluating the results</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/345/1*bfEpNRFU3bYY_SEo8sJetQ.png" /></figure><p>We have achieved an accuracy of almost 90% with basic fine-tuning.</p><h3>Final Words</h3><p>The techniques for classifying long documents <strong>requires, in most cases, padding to a shorter text, however, as we saw, using BERT with masking techniques, we can still achieve such tasks.</strong></p><p><a href="https://arxiv.org/abs/1910.10781">In the paper</a>, another method has been proposed: ToBERT (transformer over BERT. This is something I’ll probably try in the future.)</p><p>The <strong>complete code</strong> can be found on this <a href="https://github.com/dipansh-girdhar/Text-classification-for-long-text/blob/master/bert_doc_classification.py">GitHub repository</a>. The project isn’t complete yet, so, I’ll be making modifications and adding more components to it. Feel free to raise an issue or a pull request if you need my help.</p><p>Thanks for reading. Cheers!</p><h3>References</h3><ul><li><a href="https://arxiv.org/abs/1810.04805">BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding</a></li><li><a href="https://www.youtube.com/watch?v=BnpB3GrpsfM">L11 Language Models — Alec Radford (OpenAI)</a></li><li><a href="https://www.curiousily.com/posts/sentiment-analysis-with-bert-and-hugging-face-using-pytorch-and-python/">Sentiment analysis with BERT and huggingface using PyTorch</a></li><li><a href="https://medium.com/@armandj.olivares/using-bert-for-classifying-documents-with-long-texts-5c3e7b04573d">Using BERT to classify documents with long texts</a></li><li><a href="https://jalammar.github.io/illustrated-bert/">The Illustrated BERT, ELMo, and co.</a></li><li><a href="https://mccormickml.com/2019/07/22/BERT-fine-tuning/">BERT Fine-Tuning Tutorial with PyTorch</a></li><li><a href="https://arxiv.org/pdf/1905.05583.pdf">How to Fine-Tune BERT for Text Classification?</a></li><li><a href="https://huggingface.co/transformers/">huggingface Transformers</a></li><li><a href="https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270">BERT Explained: State of the art language model for NLP</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f54833994dfd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/text-classification-with-bert-using-transformers-for-long-text-inputs-f54833994dfd">Text Classification with BERT using Transformers for long text inputs</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>