<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Bhangun Hartani on Medium]]></title>
        <description><![CDATA[Stories by Bhangun Hartani on Medium]]></description>
        <link>https://medium.com/@bhangun?source=rss-253d9aecde52------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*OSh4vt7q5XH1DdBE.jpg</url>
            <title>Stories by Bhangun Hartani on Medium</title>
            <link>https://medium.com/@bhangun?source=rss-253d9aecde52------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 15:47:16 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bhangun/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Tokenize Quranic Text (Surah Alfatihah) using NLTK]]></title>
            <link>https://medium.com/@bhangun/tokenize-quranic-text-surah-alfatihah-using-nltk-9cd6c1d5e96d?source=rss-253d9aecde52------2</link>
            <guid isPermaLink="false">https://medium.com/p/9cd6c1d5e96d</guid>
            <dc:creator><![CDATA[Bhangun Hartani]]></dc:creator>
            <pubDate>Tue, 11 Aug 2020 01:27:00 GMT</pubDate>
            <atom:updated>2020-08-16T04:58:25.981Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/761/1*mNmFXB5sXcANHSQILDLrig.png" /></figure><p>Assalamu’alaykum warahmatullahi wabarakatuh.</p><p>In a few days ago i’m looking for code to get metadata from quranic text to be implement in my big project. And then i realize Natural Language Processing (NLP) is a good option. So i’m looking for python library, because python has a good practice for ML, DL or Data Science tools, then i got <a href="https://www.nltk.org/">nltk</a>. This is may be just first to my projects to get other metadata from quran, so there would be another research which, in sha Allah, i would share again.</p><p>For this sample i tried to short surah in Alquran which is Alfatihah as below.</p><blockquote>بِسْمِ ٱللَّهِ ٱلرَّحْمَٰنِ ٱلرَّحِيمِ<br>ٱلْحَمْدُ لِلَّهِ رَبِّ ٱلْعَٰلَمِينَ<br>ٱلرَّحْمَٰنِ ٱلرَّحِيمِ<br>مَٰلِكِ يَوْمِ ٱلدِّينِ<br>إِيَّاكَ نَعْبُدُ وَإِيَّاكَ نَسْتَعِينُ<br>ٱهْدِنَا ٱلصِّرَٰطَ ٱلْمُسْتَقِيمَ<br>صِرَٰطَ ٱلَّذِينَ أَنْعَمْتَ عَلَيْهِمْ غَيْرِ ٱلْمَغْضُوبِ عَلَيْهِمْ وَلَا ٱلضَّآلِّين</blockquote><p>From this text we want to tokenize, so we can get which/how many words and frequency of each words. What we need for the first time is install the library (ignore this step if you has this library).</p><blockquote>pip install nltk</blockquote><p>And then we need to download the <a href="https://www.nltk.org/_modules/nltk/tokenize/punkt.html">punkt</a> function to tokenize the text.</p><blockquote>nltk.download(‘punkt’)</blockquote><p>for this example, the quran text above, i stored in a file and then read it, as below.</p><blockquote>f = codecs.open(‘/content/q001.txt’, ‘r’, ‘utf-8-sig’)</blockquote><blockquote>text = f.read()</blockquote><p>and now we have the data source (to be notes, this text is based on UTF-8 unicode). The next step is tokenize it as below.</p><blockquote>a = nltk.word_tokenize(text)</blockquote><blockquote>print(a)</blockquote><p>then we got the result.</p><blockquote>[‘بِسْمِ’, ‘ٱللَّهِ’, ‘ٱلرَّحْمَٰنِ’, ‘ٱلرَّحِيمِ’, ‘ٱلْحَمْدُ’, ‘لِلَّهِ’, ‘رَبِّ’, ‘ٱلْعَٰلَمِينَ’, ‘ٱلرَّحْمَٰنِ’, ‘ٱلرَّحِيمِ’, ‘مَٰلِكِ’, ‘يَوْمِ’, ‘ٱلدِّينِ’, ‘إِيَّاكَ’, ‘نَعْبُدُ’, ‘وَإِيَّاكَ’, ‘نَسْتَعِينُ’, ‘ٱهْدِنَا’, ‘ٱلصِّرَٰطَ’, ‘ٱلْمُسْتَقِيمَ’, ‘صِرَٰطَ’, ‘ٱلَّذِينَ’, ‘أَنْعَمْتَ’, ‘عَلَيْهِمْ’, ‘غَيْرِ’, ‘ٱلْمَغْضُوبِ’, ‘عَلَيْهِمْ’, ‘وَلَا’, ‘ٱلضَّآلِّين’]</blockquote><p>But what we need is not just the words, also frequency of each word in whole of surah. So we need to add this code.</p><blockquote>f = nltk.FreqDist(a)</blockquote><blockquote>print(f.most_common(100))</blockquote><p>When you run above code you will get this result. Notes, 100 above is how many words you want to expose, for this i used 100, actualy is less then 100 as you see above, but no problem if your more than what actualy is.</p><blockquote>[(‘ٱلرَّحْمَٰنِ’, 2), (‘ٱلرَّحِيمِ’, 2), (‘عَلَيْهِمْ’, 2), (‘بِسْمِ’, 1), (‘ٱللَّهِ’, 1), (‘ٱلْحَمْدُ’, 1), (‘لِلَّهِ’, 1), (‘رَبِّ’, 1), (‘ٱلْعَٰلَمِينَ’, 1), (‘مَٰلِكِ’, 1), (‘يَوْمِ’, 1), (‘ٱلدِّينِ’, 1), (‘إِيَّاكَ’, 1), (‘نَعْبُدُ’, 1), (‘وَإِيَّاكَ’, 1), (‘نَسْتَعِينُ’, 1), (‘ٱهْدِنَا’, 1), (‘ٱلصِّرَٰطَ’, 1), (‘ٱلْمُسْتَقِيمَ’, 1), (‘صِرَٰطَ’, 1), (‘ٱلَّذِينَ’, 1), (‘أَنْعَمْتَ’, 1), (‘غَيْرِ’, 1), (‘ٱلْمَغْضُوبِ’, 1), (‘وَلَا’, 1), (‘ٱلضَّآلِّين’, 1)]</blockquote><p>for complete code you can look as below.</p><blockquote>import nltk</blockquote><blockquote>import codecs</blockquote><blockquote>nltk.download(‘punkt’)</blockquote><blockquote>f = codecs.open(‘/content/q001.txt’, ‘r’, ‘utf-8-sig’)</blockquote><blockquote>text = f.read()</blockquote><blockquote>a = nltk.word_tokenize(text)</blockquote><blockquote>#print(a)</blockquote><blockquote>f = nltk.FreqDist(a)</blockquote><blockquote>print(f.most_common(500))</blockquote><p>Ok alhamdulillah, i hope you would try it too. If you have another workaround for tokenize quran text don’t hesitate to comment below. Thanks, Wassalamu’alaykum.</p><p>You can try this <a href="https://colab.research.google.com/drive/1zvdKDOY3wPLaQIT1aznjO70SNMerjTrF#scrollTo=bRYPLUWz3ePU&amp;line=10&amp;uniqifier=1">code</a> yourself .</p><p><a href="https://colab.research.google.com/drive/1zvdKDOY3wPLaQIT1aznjO70SNMerjTrF#scrollTo=bRYPLUWz3ePU&amp;line=2&amp;uniqifier=1">Google Colaboratory</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9cd6c1d5e96d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>