<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by KD on Medium]]></title>
        <description><![CDATA[Stories by KD on Medium]]></description>
        <link>https://medium.com/@kumardeepak_10246?source=rss-ab14798f4f89------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*QV-Gyry69qJ4gEA6.</url>
            <title>Stories by KD on Medium</title>
            <link>https://medium.com/@kumardeepak_10246?source=rss-ab14798f4f89------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 12:34:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kumardeepak_10246/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[BERT — Multi class Text Classification]]></title>
            <link>https://medium.com/analytics-vidhya/bert-multi-class-text-classification-e3822836ee0d?source=rss-ab14798f4f89------2</link>
            <guid isPermaLink="false">https://medium.com/p/e3822836ee0d</guid>
            <category><![CDATA[bert]]></category>
            <category><![CDATA[multiclass-text]]></category>
            <category><![CDATA[multiclass-classification]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[robert]]></category>
            <dc:creator><![CDATA[KD]]></dc:creator>
            <pubDate>Sun, 10 Nov 2019 02:59:48 GMT</pubDate>
            <atom:updated>2019-12-09T08:56:33.426Z</atom:updated>
            <content:encoded><![CDATA[<h3>BERT — Multi class Text Classification on your dataset</h3><figure><img alt="Ruins of the ancient Nalanda University in Bihar, India" src="https://cdn-images-1.medium.com/max/1024/1*ERci-1D3Q4XlJHW3OfGr8w.png" /><figcaption>Ruins of the ancient Nalanda University in Bihar, India</figcaption></figure><p>I was working on multi-class text classification for one of my clients, where I wanted to evaluate my current model accuracy against BERT sequence classification.</p><p>And that’s how all of it started for me. Given the popularity BERT enjoys I was sure that I would get some online code help to compare result against my existing classification model. But, surprisingly I could not get anything handy and end up researching about it.</p><p>So, I thought of saving time for others and decided to write this article for those who wanted to use BERT for multi-class text classification on their dataset</p><p>Thanks to “Hugging Face” for transformers, that is now available in pytorch. You can start by installing the transformers pip. The model will be get installed on the first code run.</p><p>I have used <strong><em>bert-base-uncased</em></strong> as model, so further discussion in this write-up will be about this pre-trained model.</p><p>In order to use pre-trained models you need to stick to specified sequence featurization. Read following functions in the given order to understand the vectorization steps.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f803d5b4fbb04888b07a995649ba73bf/href">https://medium.com/media/f803d5b4fbb04888b07a995649ba73bf/href</a></iframe><p>Before you actually do training exercise it is important to understand the following function, this is the function that needs to be modified as per the model you are planning to use. Also, it is important to use specific <em>tokenizer</em>, so be careful.</p><p>The following featurization code is specific to <strong><em>bert-base-uncased </em></strong>model.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/79dc74009512d58b22be3bc4a2266b82/href">https://medium.com/media/79dc74009512d58b22be3bc4a2266b82/href</a></iframe><p>Once we have the vectorized dataset, we can go to training step. All other steps like running for a certain epochs, saving the models for evaluation remains same.</p><p>In my case, I have had around 10,000 training and 2,000 validation sentences. I run 10 epochs to get accuracy improvements against my existing classification model.</p><p><strong>Running on your own data</strong></p><p>You can directly run this code or notebook on your own dataset by arranging your data as discussed below:</p><ul><li>installed dependencies like transformers, torch etc. (I have mentioned the list in repo.)</li><li>split your data into three usual three categories, “<em>train, valid, and test</em>” and store as CSV file.</li><li>The CSV file should at least have two columns, named “<strong>texts</strong>” and “<strong>labels</strong>”</li><li>You must have guessed that “<strong>texts</strong>” should contains your sentence and “<strong>labels</strong>” should contain its class / category</li></ul><p>Following are my hyper-parameters with respect to BERT evaluation. Once you are able to run with default values and evaluation, I would suggest playing around with “per_gpu_train_batch_size”, “max_seq_length” and “learning_rate” and rerun and evaluate results.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f94473cf4f192764fcd86db07e86fefb/href">https://medium.com/media/f94473cf4f192764fcd86db07e86fefb/href</a></iframe><p>My work is committed <a href="https://github.com/kumardeepak/bert-multi-class.git">here</a>, feel free to fork and comment.</p><p><strong>Running other pre-train models</strong></p><p>You need to tweak my code a bit to run other available models like XLNet, GPT2 or RoBERT etc. In case you need help, do reach out to me. However, I am sure, you need to tweak or fix into vectorization step, as mentioned above.</p><p>Please clone <a href="https://github.com/huggingface/transformers.git">transformers repository</a>, specifically for binary classification or multi-class classification, look into “run_glue.py” code.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e3822836ee0d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/bert-multi-class-text-classification-e3822836ee0d">BERT — Multi class Text Classification</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>