<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mukhtar A.Algezoli on Medium]]></title>
        <description><![CDATA[Stories by Mukhtar A.Algezoli on Medium]]></description>
        <link>https://medium.com/@mukhtaralgezoli?source=rss-fb68ff1e49d7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*Twlc4L3uAByRAqv-.jpg</url>
            <title>Stories by Mukhtar A.Algezoli on Medium</title>
            <link>https://medium.com/@mukhtaralgezoli?source=rss-fb68ff1e49d7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 11:44:44 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mukhtaralgezoli/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Preparing KALIMAT dataset for BERT]]></title>
            <link>https://medium.com/@mukhtaralgezoli/preparing-kalimat-dataset-for-bert-2d6727957106?source=rss-fb68ff1e49d7------2</link>
            <guid isPermaLink="false">https://medium.com/p/2d6727957106</guid>
            <category><![CDATA[arabic]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[pre-processing]]></category>
            <category><![CDATA[summarization]]></category>
            <category><![CDATA[pre-sum]]></category>
            <dc:creator><![CDATA[Mukhtar A.Algezoli]]></dc:creator>
            <pubDate>Mon, 23 Mar 2020 21:41:50 GMT</pubDate>
            <atom:updated>2020-03-23T21:56:28.939Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LJvUSTNNBwVKYvqTVSQT4Q.png" /></figure><p><strong>KALIMAT</strong> is a multipurpose Arabic corpus used mainly for extractive summarization, although this dataset is not that good (as it has articles in different Arabic dialects) the lack of Arabic resources makes it very important for NLP in this language.</p><p>In this article I will try to show how you can prepare <strong>KALIMAT</strong> for the summarization task using BERT, we will be building upon the preprocessing of this <a href="https://arxiv.org/abs/1908.08345">paper</a> by Yang Liu and Mirella Lapata but the final output can be used with any BERT based implementation.</p><p>First, you can download the raw dataset from <a href="https://sourceforge.net/projects/kalimat/">here</a>, each category in the dataset(culture, economy, local-news, international-news, religion, and sport) have its own file containing articles of each month in a txt file, the text on those txt files are in Latin-1 (ironically Latin-1 does not support Arabic), so to spare you the frustration I already converted it to UTF-8 and organized it in a CSV file you can download it from <a href="https://drive.google.com/file/d/1l1cz2e3USk0FY3AjdDn0TXkPTBlhbjXL/view?usp=sharing">here</a>.</p><p>Next, we will be putting those articles and their summaries together, the best way to do this is to put each article and its summary in a separate .STORY file with the summary divided into highlights and put at the end of the file, you can use <a href="https://github.com/mukhtar-algezoli/Preprocessing_KALIMAT/blob/master/arabicCSVToStories.py">this code</a>.</p><p>At this stage we can finally use the preprocessing suggested by the <strong>Presum</strong> paper in <a href="https://github.com/nlpyang/PreSumm">this repository</a>, but with some changes to make it work with Arabic:</p><ol><li>In the Standford corenlp file replace<strong> stanford-corenlp-3.9.2</strong> with <strong>stanford-arabic-corenlp-2018–10–05-models </strong>(you can download it from <a href="http://nlp.stanford.edu/software/stanford-arabic-corenlp-2018-10-05-models.jar">here</a>)<strong> </strong>and use its path as the path for Stanford corenlp tokenizer.</li><li>Replace data_builder.py script in <a href="https://github.com/nlpyang/PreSumm/blob/master/src/prepro/data_builder.py">Presum</a> with <a href="https://github.com/mukhtar-algezoli/Preprocessing_KALIMAT/blob/master/data_builder.py">this databuilder script</a>, this replaces <strong>bert-base-uncased </strong>pretrained model with <a href="https://huggingface.co/transformers/pretrained_models.html"><strong>bert-base-multilingual-uncased</strong></a><strong> </strong>(or you can use a standalone Arabic BERT-pretrained model that hopefully will be released in the future).</li></ol><p>After going through all four steps in the <strong>Presum </strong>repository you will get 7 PT files each with 2000-ish entry, each entry containing:</p><ul><li><strong>src_txt </strong>and<strong> src</strong>: those the original articles and their tokenized counterparts (tokenized using BERT_multilingual tokenizer).</li><li><strong>tgt_txt</strong> and <strong>tgt</strong>: those the original summaries and their tokenized counterparts (tokenized using BERT_multilingual tokenizer).</li></ul><p><strong>References</strong>:</p><ul><li><a href="https://arxiv.org/abs/1810.04805">BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding</a></li><li><a href="https://arxiv.org/abs/1908.08345">Text Summarization with Pretrained Encoders</a></li></ul><h3>Thank you Do Not forget to give claps.</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2d6727957106" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>