<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[AI Backyard - Medium]]></title>
        <description><![CDATA[sharing interesting codes and projects about Artificial Intelligence - Medium]]></description>
        <link>https://medium.com/ai-backyard?source=rss----ede5667c68fb---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>AI Backyard - Medium</title>
            <link>https://medium.com/ai-backyard?source=rss----ede5667c68fb---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 20:16:40 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/ai-backyard" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[News Classifier]]></title>
            <link>https://medium.com/ai-backyard/news-classifier-89940b01bbcb?source=rss----ede5667c68fb---4</link>
            <guid isPermaLink="false">https://medium.com/p/89940b01bbcb</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[text]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[news]]></category>
            <dc:creator><![CDATA[Bahram Mohammadpour]]></dc:creator>
            <pubDate>Fri, 24 Aug 2018 17:53:14 GMT</pubDate>
            <atom:updated>2018-08-24T17:54:40.353Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*HgXA9v1EsqlrRDaC_iORhQ.png" /></figure><h3>this is a joint affair of <a href="https://medium.com/@arminbehjati">Armin Behjati</a> and <a href="https://medium.com/@bahram.mp">Bahram Mohammadpour.</a></h3><p>Here we try to make a news classifier based on the news language model we made earlier . back then we saved the encoder to use it here as the backbone of this model .</p><p>as before we use the fantastic fastai library to make things faster and easier !</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/91db8d13dcd09a01e5c148eb7f251abc/href">https://medium.com/media/91db8d13dcd09a01e5c148eb7f251abc/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d8c85f5fb82d24df58283c7c800d8eed/href">https://medium.com/media/d8c85f5fb82d24df58283c7c800d8eed/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/599cf7da075bd3d8629a0412a9ceb167/href">https://medium.com/media/599cf7da075bd3d8629a0412a9ceb167/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7d7829380db126c94a97a1d9316c9380/href">https://medium.com/media/7d7829380db126c94a97a1d9316c9380/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/edf8e8f123de0540bde42e3960822361/href">https://medium.com/media/edf8e8f123de0540bde42e3960822361/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/329eed89406afa93c935a9427880772b/href">https://medium.com/media/329eed89406afa93c935a9427880772b/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8e4bedec497e26b4c13b4fbc2e107d87/href">https://medium.com/media/8e4bedec497e26b4c13b4fbc2e107d87/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*Edd7BaYlSf2ccI1wPWR9kw.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1ae4711f5e457ce090f796eb7af0029c/href">https://medium.com/media/1ae4711f5e457ce090f796eb7af0029c/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/06a3ef699d4b25090b75eb98d27c3091/href">https://medium.com/media/06a3ef699d4b25090b75eb98d27c3091/href</a></iframe><pre>{10: 0, 2: 1, 18: 2, 5: 3}</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/43dbc7d5ed7c4fb23f47d6e696dab8bf/href">https://medium.com/media/43dbc7d5ed7c4fb23f47d6e696dab8bf/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ec2b57c0872843be738cc2b90ef390dd/href">https://medium.com/media/ec2b57c0872843be738cc2b90ef390dd/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/294da34e5e97a0ede977168751be1070/href">https://medium.com/media/294da34e5e97a0ede977168751be1070/href</a></iframe><pre>array([2, 2, 18, ..., 18, 2, 5], dtype=object)</pre><p>We shuffle the dataset to add an element of randomness .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c13881fd36144a3dc528e6cc992376c2/href">https://medium.com/media/c13881fd36144a3dc528e6cc992376c2/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fa2bcdda27d8a4861220ced6d4c958a5/href">https://medium.com/media/fa2bcdda27d8a4861220ced6d4c958a5/href</a></iframe><p>We split the data so that we have 9000 news files as training data and the rest is for validation .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/adba5ff4ad56b074773712c8a67a9f80/href">https://medium.com/media/adba5ff4ad56b074773712c8a67a9f80/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bd57b922db9ed16eed0effec8ecf6da3/href">https://medium.com/media/bd57b922db9ed16eed0effec8ecf6da3/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/eb0a7d4228665d8a100d56a728ef943a/href">https://medium.com/media/eb0a7d4228665d8a100d56a728ef943a/href</a></iframe><pre>(1171, 2)</pre><p>We have 4 labels referring to 4 news categories .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1307081b7f8d02e17edcd3cf39eef35c/href">https://medium.com/media/1307081b7f8d02e17edcd3cf39eef35c/href</a></iframe><pre>{0, 1, 2, 3}</pre><p>As before we save our data and then tokenize it .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/084782208545093ccb0f044f103c2f88/href">https://medium.com/media/084782208545093ccb0f044f103c2f88/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/64e39d3045b98e5a3938265cd35b0755/href">https://medium.com/media/64e39d3045b98e5a3938265cd35b0755/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1b876acc9bffbaa429a5ef864be72e0d/href">https://medium.com/media/1b876acc9bffbaa429a5ef864be72e0d/href</a></iframe><pre>&#39;\xa0معاون بازاريابي فروش ايران خودرو با بيان اين كه اجراي روش هاي متنوع و جامع براي فروش محصولات يكي از مسيرهاي افزايش رضايتمندي مشتريان و فراهم كردن امكان خريد كالاي ايراني است تاكيد كرد: ايران خودرو در سال حمايت از كالاي ايراني برنامه هاي متنوعي را براي خريد محصولات به اجرا خواهد گذاشت. \xa0\xa0 \xa0\xa0به گزارش ايكوپرس، مصطفي خان كرمي با تشريح برنامه هاي فروش ايران خودرو در سال 97، اعمال تخفيف خريد كالاي ايراني را از جمله مشوق هاي خريد محصولات ايران خودرو دانست و اظهار كرد:\u200c با مطالعه و بررسي بازار و نيازسنجي از مشتريان برنامه هاي فروش را منطبق با خواست و سليقه آنان به اجرا مي گذاريم.وي افزود:\u200c طرح هاي فروش را متناسب با بودجه مشتريان در نظر مي گيريم و در اين راستا برنامه هاي فروش اقساطي و اعتباري را طرح ريزي مي كنيم تا همه گروه هاي درآمدي موفق به خريد محصولات ايراني ايران خودرو شوند.خان كرمي طرح هاي پيش فروش را از جمله طرح هاي فروش محصولات ايران خودرو نام برد و گفت: در پيش فروش هايی كه به صورت سرمايه گذاري پيشنهاد مي شود امكان مشاركت مشتريان در فرآيند توليد محصول ايجاد مي شود.وي درعين حال تصريح كرد: پرواضح است كه ايران خودرو نيز از مشاركت هاي مردمي در طرح هاي توليد بهره مند مي شود. با اين روش بازگشت سرمايه مشتريان تا موعد تحويل خودرو كاملا تضمين شده است.خان كرمي افزود:\u200c در طرح هاي پيش فروش عادي نيز امكان برنامه ريزي مالي بهتري را براي مشتريان فراهم مي كنيم.معاون بازاريابي و فروش ايران خودرو اضافه كرد: علاوه بر فروش هاي نقدي كه محصول با قيمت قطعي به دست مشتريان مي رسد، فروش اقساطي و اعتباري را نيز به اجرا مي گذاريم كه با اعمال شرايط منعطف، علاوه بر شناور بودن مبلغ پيش پرداخت، مشتريان قادر هستند زمان بازپرداخت اقساط را براساس شرايط مالي خود انتخاب كنند. | &#39;</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/06ab618ba720c20e79759e2186e8bf00/href">https://medium.com/media/06ab618ba720c20e79759e2186e8bf00/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d7b91d2f74dc3a6ced1c9cbbf971924c/href">https://medium.com/media/d7b91d2f74dc3a6ced1c9cbbf971924c/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ae531d30a05c3100789a0d36654a6cd9/href">https://medium.com/media/ae531d30a05c3100789a0d36654a6cd9/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/61b290f1464d023c249160153daf01d3/href">https://medium.com/media/61b290f1464d023c249160153daf01d3/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/24c9c1d77cff0c8f13e70518e37c4968/href">https://medium.com/media/24c9c1d77cff0c8f13e70518e37c4968/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/005e0487874964b12087e512976d79d1/href">https://medium.com/media/005e0487874964b12087e512976d79d1/href</a></iframe><pre>0<br>1<br>2<br>3<br>4<br>5<br>6<br>7<br>8<br>0<br>1</pre><p>Here we just save the tokens .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/caea12db32ec6127f3175918a5007160/href">https://medium.com/media/caea12db32ec6127f3175918a5007160/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/93c798f41bc7be7f5e7e6915b7fc1b6e/href">https://medium.com/media/93c798f41bc7be7f5e7e6915b7fc1b6e/href</a></iframe><p>Now we load itos we saved before in language model part and make a stoi function from it .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a62ca4995847cd07d0082cfad384a64f/href">https://medium.com/media/a62ca4995847cd07d0082cfad384a64f/href</a></iframe><pre>45303</pre><h4>Classifier !</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8253cd406671003e68c02d26b810b0dc/href">https://medium.com/media/8253cd406671003e68c02d26b810b0dc/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9f5ca43d9fd5c19fe14acce408827582/href">https://medium.com/media/9f5ca43d9fd5c19fe14acce408827582/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/49b97702de0786734df83d3575ddfb50/href">https://medium.com/media/49b97702de0786734df83d3575ddfb50/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fa14610e23d335ca4b25513a04fca81c/href">https://medium.com/media/fa14610e23d335ca4b25513a04fca81c/href</a></iframe><p>The hyperparameters should be same as before as we are going to load the encoder . but the dropouts and bptt can be changed .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f721529ae1c8ecd4e6aae86c46be4179/href">https://medium.com/media/f721529ae1c8ecd4e6aae86c46be4179/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cf21c13a40efe30175a5ee51d23bfe07/href">https://medium.com/media/cf21c13a40efe30175a5ee51d23bfe07/href</a></iframe><p>The first thing we need is a Dataset. So we have a Dataset passing in the documents and their labels.then we use sortish which makes it so the first things in the list are, on the whole, shorter than the things at the end, but a little bit random as well.<br>sampler parameter and sampler is a class we are going to define that tells the data loader how to shuffle. <br>For validation set, we are going to define something that actually just sorts. It just deterministically sorts it so that all the shortest documents will be at the start, all the longest documents will be at the end, and that’s going to minimize the amount of padding.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/687179fac111be0a61660b55203ada44/href">https://medium.com/media/687179fac111be0a61660b55203ada44/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1c7e32c9c5e894b9025721da5e214da5/href">https://medium.com/media/1c7e32c9c5e894b9025721da5e214da5/href</a></iframe><p>So now we can call that function as before .this timewith the head we add on, we have a few more things we can do. we can add more than one hidden layer. In layers=[em_sz*3, 50, c]<br>em_sz*3 : is the input to the classifier head <br>50: output of the first layer .<br>c : output of the last layer or the number of classes .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/69fa5af199fdaa37f69b116456270a51/href">https://medium.com/media/69fa5af199fdaa37f69b116456270a51/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/effcf0adbc3bf35e738b63d1f754c089/href">https://medium.com/media/effcf0adbc3bf35e738b63d1f754c089/href</a></iframe><p>We are going to use RNN_Learner just like before.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/91737ea52d89d82df5408949d1b47bc6/href">https://medium.com/media/91737ea52d89d82df5408949d1b47bc6/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0488ebf4b8ee431fe3ae57abf87f9ed7/href">https://medium.com/media/0488ebf4b8ee431fe3ae57abf87f9ed7/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d7dbceff74cd9ff887e369249ce2fb66/href">https://medium.com/media/d7dbceff74cd9ff887e369249ce2fb66/href</a></iframe><p>Here we use weight decay and load the encoder we have saved .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dd41322814d28951e276ddf6dd7c2223/href">https://medium.com/media/dd41322814d28951e276ddf6dd7c2223/href</a></iframe><p>Now we just train the last layer and we get 89% accuracy !</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a2b776a21a4163341314df3275361ebc/href">https://medium.com/media/a2b776a21a4163341314df3275361ebc/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/82ee9a5030f30fcc997be678d33a0a76/href">https://medium.com/media/82ee9a5030f30fcc997be678d33a0a76/href</a></iframe><pre>epoch      trn_loss   val_loss   accuracy                    <br>    0      0.367081   0.870484   0.899231</pre><pre>[array([0.87048]), 0.8992314260297103]</pre><p>Here we train the last three layers and we get the 92% accuracy !</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d819f6d9c5f544f1347cb68ebb1b928e/href">https://medium.com/media/d819f6d9c5f544f1347cb68ebb1b928e/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a1b3f209818a2313efea07d278d2f6c6/href">https://medium.com/media/a1b3f209818a2313efea07d278d2f6c6/href</a></iframe><pre>epoch      trn_loss   val_loss   accuracy                    <br>    0      0.297732   0.232494   0.918873</pre><pre>[array([0.23249]), 0.9188727579190118]</pre><p>And here id the model architecture :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/08255938824b486388c8ba56335e1ff4/href">https://medium.com/media/08255938824b486388c8ba56335e1ff4/href</a></iframe><pre>SequentialRNN(<br>  (0): MultiBatchRNN(<br>    (encoder): Embedding(45303, 400, padding_idx=1)<br>    (encoder_with_dropout): EmbeddingDropout(<br>      (embed): Embedding(45303, 400, padding_idx=1)<br>    )<br>    (rnns): ModuleList(<br>      (0): WeightDrop(<br>        (module): LSTM(400, 1150)<br>      )<br>      (1): WeightDrop(<br>        (module): LSTM(1150, 1150)<br>      )<br>      (2): WeightDrop(<br>        (module): LSTM(1150, 1150)<br>      )<br>      (3): WeightDrop(<br>        (module): LSTM(1150, 400)<br>      )<br>    )<br>    (dropouti): LockedDropout(<br>    )<br>    (dropouths): ModuleList(<br>      (0): LockedDropout(<br>      )<br>      (1): LockedDropout(<br>      )<br>      (2): LockedDropout(<br>      )<br>      (3): LockedDropout(<br>      )<br>    )<br>  )<br>  (1): PoolingLinearClassifier(<br>    (layers): ModuleList(<br>      (0): LinearBlock(<br>        (lin): Linear(in_features=1200, out_features=50, bias=True)<br>        (drop): Dropout(p=0.2)<br>        (bn): BatchNorm1d(1200, eps=1e-05, momentum=0.1, affine=True)<br>      )<br>      (1): LinearBlock(<br>        (lin): Linear(in_features=50, out_features=4, bias=True)<br>        (drop): Dropout(p=0.1)<br>        (bn): BatchNorm1d(50, eps=1e-05, momentum=0.1, affine=True)<br>      )<br>    )<br>  )<br>)</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=89940b01bbcb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ai-backyard/news-classifier-89940b01bbcb">News Classifier</a> was originally published in <a href="https://medium.com/ai-backyard">AI Backyard</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Persian to English translation]]></title>
            <link>https://medium.com/ai-backyard/persian-to-english-translation-efa97896846?source=rss----ede5667c68fb---4</link>
            <guid isPermaLink="false">https://medium.com/p/efa97896846</guid>
            <category><![CDATA[lstm]]></category>
            <category><![CDATA[persian]]></category>
            <category><![CDATA[farsi]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[translation]]></category>
            <dc:creator><![CDATA[Armin Behjati]]></dc:creator>
            <pubDate>Thu, 23 Aug 2018 18:50:39 GMT</pubDate>
            <atom:updated>2018-08-23T18:50:39.188Z</atom:updated>
            <content:encoded><![CDATA[<h4>this is a joint affair of <a href="https://medium.com/@arminbehjati">Armin Behjati</a> and <a href="https://medium.com/@bahram.mp">Bahram Mohammadpour</a></h4><p>Here we try to make a sequence-to-sequence model . we are going to make a Persian to English translation model . we will be using TEP ,English-Persian corpus which is extracted from movie subtitles and is sentence aligned . it has 4 million tokens on each side . Total number of bilingual sentences is 612086 .Average sentence length is 7.8 words.</p><p>We are using fastai library to make things faster and easier . <br>Also thanks to <a href="https://medium.com/@hiromi_suenaga">Hiromi Suenaga</a> for explanations and notes .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8efe8fd8b683fead4a4992e15eb027e8/href">https://medium.com/media/8efe8fd8b683fead4a4992e15eb027e8/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cac85eee73269a9c920eb3f95137e422/href">https://medium.com/media/cac85eee73269a9c920eb3f95137e422/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ed8dacbf50fb60c10745869b1b8d4e06/href">https://medium.com/media/ed8dacbf50fb60c10745869b1b8d4e06/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1a9a6d3d47b8eff160baa1cf5a94b073/href">https://medium.com/media/1a9a6d3d47b8eff160baa1cf5a94b073/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/adacd23cb74c5c4495b81fcdc7c78537/href">https://medium.com/media/adacd23cb74c5c4495b81fcdc7c78537/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a5c39c2ffaf086a0f89a5598e4610842/href">https://medium.com/media/a5c39c2ffaf086a0f89a5598e4610842/href</a></iframe><pre>([(&#39;raspy breathing .\n&#39;, &#39;\ufeffصداي خر خر .\n&#39;),<br>  (&#39;dad .\n&#39;, &#39;پدر .\n&#39;),<br>  (&#39;maybe its the wind .\n&#39;, &#39;شايد صداي باد باشه .\n&#39;),<br>  (&#39;no .\n&#39;, &#39;نه .\n&#39;),<br>  (&#39;stop please stop .\n&#39;, &#39;دست نگه داريد خواهش ميکنم دست نگه داريد .\n&#39;)],<br> 612086)</pre><p>we are just going to save them for further use .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/472fb9ef18642c9c9902cf15b4e280cb/href">https://medium.com/media/472fb9ef18642c9c9902cf15b4e280cb/href</a></iframe><p>we can easily split them apart into English and Persian sentences lists .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a5e67a9aa83fdac788669db774138d16/href">https://medium.com/media/a5e67a9aa83fdac788669db774138d16/href</a></iframe><p>Here we tokenize the sentences . again we are using the spacy tokenizer and due to the lack of good persian tokenizer available we decided to try english tokenizer for persian too and it worked fine ! <br>proc_all_mp is processing every sentence across multiple processes .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/06e4f01fabed6e35963022a91ef21e86/href">https://medium.com/media/06e4f01fabed6e35963022a91ef21e86/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7569d6295f9816505e77a2080e88eebc/href">https://medium.com/media/7569d6295f9816505e77a2080e88eebc/href</a></iframe><p>Here is example of a sentence after tokenization :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fe1154a8ca6176c8e9062b9b713e0e0a/href">https://medium.com/media/fe1154a8ca6176c8e9062b9b713e0e0a/href</a></iframe><pre>([&#39;you&#39;,<br>  &#39;have&#39;,<br>  &#39;a&#39;,<br>  &#39;week&#39;,<br>  &#39;,&#39;,<br>  &#39;evans&#39;,<br>  &#39;then&#39;,<br>  &#39;well&#39;,<br>  &#39;burn&#39;,<br>  &#39;the&#39;,<br>  &#39;house&#39;,<br>  &#39;.&#39;,<br>  &#39;\n&#39;],<br> [&#39;اوانز&#39;,<br>  &#39;تو&#39;,<br>  &#39;فقط&#39;,<br>  &#39;يک&#39;,<br>  &#39;هفته&#39;,<br>  &#39;وقت&#39;,<br>  &#39;داري&#39;,<br>  &#39;وگرنه&#39;,<br>  &#39;خونتو&#39;,<br>  &#39;خواهيم&#39;,<br>  &#39;سوزوند&#39;,<br>  &#39;.&#39;,<br>  &#39;\n&#39;])</pre><p>Now we have to save the tokens .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a340974bcbf7fd503d27eee3df674525/href">https://medium.com/media/a340974bcbf7fd503d27eee3df674525/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e28d95162ada6fe8ec20fa10b0323b17/href">https://medium.com/media/e28d95162ada6fe8ec20fa10b0323b17/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fd48b22e993bfb9b9dda6637f7fd0409/href">https://medium.com/media/fd48b22e993bfb9b9dda6637f7fd0409/href</a></iframe><p>Here are the top 25 most frequent words :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/eab95773749dbe56bff31281153ef018/href">https://medium.com/media/eab95773749dbe56bff31281153ef018/href</a></iframe><pre>[(&#39;\n&#39;, 612051),<br> (&#39;.&#39;, 474293),<br> (&#39;،&#39;, 103134),<br> (&#39;من&#39;, 90505),<br> (&#39;به&#39;, 76135),<br> (&#39;را&#39;, 66759),<br> (&#39;تو&#39;, 60411),<br> (&#39;و&#39;, 60239),<br> (&#39;که&#39;, 59504),<br> (&#39;از&#39;, 55806),<br> (&#39;اين&#39;, 47409),<br> (&#39;اون&#39;, 44937),<br> (&#39;يک&#39;, 33618),<br> (&#39;در&#39;, 32070),<br> (&#39;با&#39;, 29032),<br> (&#39;ما&#39;, 28843),<br> (&#39;رو&#39;, 26067),<br> (&#39;كه&#39;, 24502),<br> (&#39;شما&#39;, 21629),<br> (&#39;هم&#39;, 19892),<br> (&#39;بايد&#39;, 18554),<br> (&#39;براي&#39;, 18230),<br> (&#39;نه&#39;, 18117),<br> (&#39;مي&#39;, 16622),<br> (&#39;بود&#39;, 16085)]</pre><p>We are going to limit the number of words to 50000 to keep things simple . We insert a few extra tokens for beginning of stream (_bos_), padding (_pad_), end of stream (_eos_), and unknown (_unk).<br>we will return 3 for the tokens we haven’t seen .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f9359958356f4e6e2a4ba7570765af21/href">https://medium.com/media/f9359958356f4e6e2a4ba7570765af21/href</a></iframe><p>Now we have a list of IDs for both Persian and English and functions to turn them to words and reverse .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f583ec46d3293dcbc92f6d05c7465c12/href">https://medium.com/media/f583ec46d3293dcbc92f6d05c7465c12/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/35a403a060c6ca1904487e93dfd7074b/href">https://medium.com/media/35a403a060c6ca1904487e93dfd7074b/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4cfd6327f7fbb992e46ff8c9bb0dc32b/href">https://medium.com/media/4cfd6327f7fbb992e46ff8c9bb0dc32b/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bccd5657a05720bd669fe3e6a22c35ad/href">https://medium.com/media/bccd5657a05720bd669fe3e6a22c35ad/href</a></iframe><pre>([&#39;پدر&#39;, &#39;.&#39;, &#39;\n&#39;, &#39;_eos_&#39;], 50004, 50004)</pre><h4>Word Vectors</h4><p>We are going to use pre-trained word vectors here but I think that transfer learning and fine-tuning pre-trained models would work better which is something wee are going to experiment more on it !<br> Here we use fasttext word vectors which has the dimension of 300 .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/254dbf02ef9e1a63ba0361dc95f43b44/href">https://medium.com/media/254dbf02ef9e1a63ba0361dc95f43b44/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/25a06d22e2f83b15addc0315fb320f05/href">https://medium.com/media/25a06d22e2f83b15addc0315fb320f05/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b3c07a79b840f6a0d935314218ea3173/href">https://medium.com/media/b3c07a79b840f6a0d935314218ea3173/href</a></iframe><p>We are going to convert them into standard python dictionary for each word .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d68b118eb7369576c20b64d16f6da2a/href">https://medium.com/media/6d68b118eb7369576c20b64d16f6da2a/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cfd11e83a083de3a16f5b0e90a936dbd/href">https://medium.com/media/cfd11e83a083de3a16f5b0e90a936dbd/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f73f3008d5410f7857ea8664d85bc82d/href">https://medium.com/media/f73f3008d5410f7857ea8664d85bc82d/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6ca2767a089956276b103a3ce5b53466/href">https://medium.com/media/6ca2767a089956276b103a3ce5b53466/href</a></iframe><pre>(300, 300)</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7d86ef5f61e80e6a8e8156c3408bf16e/href">https://medium.com/media/7d86ef5f61e80e6a8e8156c3408bf16e/href</a></iframe><pre>(0.0075652334, 0.29283327)</pre><h3>ModelData</h3><p>We are going to grab a maximum length for sentences .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/44e77eaee4a89a20a0127a5d673eff99/href">https://medium.com/media/44e77eaee4a89a20a0127a5d673eff99/href</a></iframe><pre>(21, 18)</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5c7de5dde80f2f632fc46e9cd7cb4392/href">https://medium.com/media/5c7de5dde80f2f632fc46e9cd7cb4392/href</a></iframe><p>Here we define our dataset which requiers two things : len and an indexer .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1e69b394e998730383e0af1fc1d49082/href">https://medium.com/media/1e69b394e998730383e0af1fc1d49082/href</a></iframe><p>Making training and validation set using a list of random numbers .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ac66ccdbd40a7c2dbbb4abf0dbfdf516/href">https://medium.com/media/ac66ccdbd40a7c2dbbb4abf0dbfdf516/href</a></iframe><pre>(550799, 61252)</pre><p>Now we create our Datasets .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bf81f8a465135afbeb2715a2488d5360/href">https://medium.com/media/bf81f8a465135afbeb2715a2488d5360/href</a></iframe><p>We have declare our padding index because it’s going to pad all sentences so that they have equal sizes . as before we are going to use the sampler to save some memory by putting together sentences with similar sizes .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/051343432e856fb516451c034ac833aa/href">https://medium.com/media/051343432e856fb516451c034ac833aa/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5a98343b31157c57225f718ea9ec2fe1/href">https://medium.com/media/5a98343b31157c57225f718ea9ec2fe1/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5c4428f87a68aa34b1f41de500060674/href">https://medium.com/media/5c4428f87a68aa34b1f41de500060674/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d4cced839e2053ad341ee725353b0b44/href">https://medium.com/media/d4cced839e2053ad341ee725353b0b44/href</a></iframe><pre>[(18, 21), (13, 6), (18, 15), (11, 4), (10, 4)]</pre><p>The architecture has an encoder that’s going to take a sequence of tokens then it returns a final hidden state or just a vector which we feed to the decoder RNN .</p><p>We have to make an embedding with the size of vocabsize by 300(fasttext vectors size ) .<br>we multiply by 3 because we had the std of about 0.3 and the standard is 1 .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c8040e0c5064912141738ab0d35a0cd1/href">https://medium.com/media/c8040e0c5064912141738ab0d35a0cd1/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4afb6cb7b0a62b40c53be61e13ee2d6c/href">https://medium.com/media/4afb6cb7b0a62b40c53be61e13ee2d6c/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8f70cdd5ceb16fad4efde0319f9f55b3/href">https://medium.com/media/8f70cdd5ceb16fad4efde0319f9f55b3/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/918a38503117b34cd7c406106542653d/href">https://medium.com/media/918a38503117b34cd7c406106542653d/href</a></iframe><p>We are going to go through and put it through the embedding. We are going to stick it through the RNN, dropout, and a linear layer. We will then append the output to a list which will be stacked into a single tensor and get returned. <br>the input to the embedding is the previous word that we translated. <br>outp.data.max : It looks in its tensor to find out which word has the highest probability. max in PyTorch returns two things: the first thing is what is that max probability and the second is what is the index into the array of that max probability. So we want that second item which is the word index with the largest thing.<br>dec_inp : It contains the word index into the vocabulary of the word.<br>by adding bidirectional=True to our encoder now we have a bidirectional model .<br> we also feed in the actual correct word at the early traing so that it has less difficulty to learn. that’s what we call pr-force .<br>We also used Attention here . the way it was used was that we created a little nueral network inside and we trained it end to end .<br>w2h = self.l2(h[-1])<br>u = F.tanh(w1e + w2h)<br>a = F.softmax(u @ self.V, 0)<br>We are going to take the last layer’s hidden state and we are going to stick it into a linear layer. Then we stick it into a nonlinear activation and then we are going to do a matrix multiply. Now rather than just taking the last encoder output, we have the whole tensor of all of the encoder outputs which we just weight by this neural net we created .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b6fd53bb80d8741901099b70208738fc/href">https://medium.com/media/b6fd53bb80d8741901099b70208738fc/href</a></iframe><p>If the generated sequence length is shorter than the sequence length of the target, we need to add some padding. PyTorch padding function requires a tuple of 6 to pad a rank 3 tensor (sequence length, batch size, by number of words in the vocab). Each pair represents padding before and after that dimension.<br>F.cross_entropy expects a rank 2 tensor, but we have sequence length by batch size, so we just flatten it out. That is what view(-1, …) does.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ce1f87b854bfb004c9ec11ed956f02a1/href">https://medium.com/media/ce1f87b854bfb004c9ec11ed956f02a1/href</a></iframe><p>to_gpu will not put to in the GPU if you do not have one.<br>We could just call Learner to turn that into a learner, but if we call RNN_Learner, it does add in save_encoder and load_encoder that can be handy sometimes .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c414d1aa6aacd173a42f18d3d5e422c4/href">https://medium.com/media/c414d1aa6aacd173a42f18d3d5e422c4/href</a></iframe><pre>29332 [&#39;...&#39;, &#39;کني&#39;, &#39;ميخوام&#39;, &#39;ميکني&#39;, &#39;ميدوني&#39;]<br>6363 [&quot;&#39;s&quot;, &quot;n&#39;t&quot;, &#39;...&#39;, &quot;&#39;m&quot;, &quot;&#39;re&quot;]</pre><p>as usual we use the learning rate finder to find the best learning rate .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c272c77e78e5e8f1059e85c9b0dad540/href">https://medium.com/media/c272c77e78e5e8f1059e85c9b0dad540/href</a></iframe><pre>66%|██████▌   | 2837/4304 [07:47&lt;04:01,  6.07it/s, loss=19.4]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/1*CRMY0qZ8QjcwNt6r7BcI5Q.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a2a7e70d7e475343ad9932bcc530b464/href">https://medium.com/media/a2a7e70d7e475343ad9932bcc530b464/href</a></iframe><p>we train the model .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fcddfd260e91178d174fb4e39be7b570/href">https://medium.com/media/fcddfd260e91178d174fb4e39be7b570/href</a></iframe><pre>epoch      trn_loss   val_loss                                <br>    0      1.788608   6.152387  <br>    1      2.027214   4.972059                                <br>    2      1.997908   4.378297                                <br>    3      2.163803   4.001754                                <br>    4      2.255261   3.823183                                <br>    5      2.33996    3.825839                                <br>    6      2.470269   3.503114                                <br>    7      2.553205   3.394398                                <br>    8      2.667889   3.23554                                 <br>    9      2.653444   3.195496                                <br>    10     2.792435   3.154075                                <br>    11     2.63283    3.183128                                <br>    12     2.501625   3.198065                                <br>    13     2.53512    3.153566                                <br>    14     2.659681   3.171559</pre><pre>[array([3.17156])]</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3996438b506c974f55dbbb946ae36990/href">https://medium.com/media/3996438b506c974f55dbbb946ae36990/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a9a5381200f3e420edb9746f4c2f7db8/href">https://medium.com/media/a9a5381200f3e420edb9746f4c2f7db8/href</a></iframe><h3>test</h3><p>Let’s test our translation . <br>we can also visualize our attention list and see at each time step where is the attention .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/60073f195d94b468e7d7ab83b5d27b55/href">https://medium.com/media/60073f195d94b468e7d7ab83b5d27b55/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4d9bbb70cc17caa92a5185092802ec7a/href">https://medium.com/media/4d9bbb70cc17caa92a5185092802ec7a/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/acf7aacb4817284cc267e6b9ca5df360/href">https://medium.com/media/acf7aacb4817284cc267e6b9ca5df360/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*pFyYq_i9w_oT7ekcTeJ2dg.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/90e49e493a8836d013acaec01b0ed09f/href">https://medium.com/media/90e49e493a8836d013acaec01b0ed09f/href</a></iframe><pre>من آخرين آزمايش گذشتت بودم . اگر داري به . اين گوش ميدي ، تو شکست خورده ي<br>i was your final test of forgiveness . and if you re listening to this , then you ve failed .<br>i was the last test . if you re listening to this , you you failed . <br> _eos_<br>نه ، راستش . حالم زياد خوب نيست . <br> _eos_<br>no , i � m not . you know , i do nt i do nt feel very well . <br><br>no , i m not . i m not so good . <br> _eos_<br>اون كه پرواز آخره . دوست دارم با من بياي تئاتر . <br> _eos_<br>it � s the late flight . i really want you to come to the play . please . <br> _eos_<br>he s the last one . i d like to come in to me . <br> _eos_<br>، پس با خودم مي‌گفتم اگر تو با من به تئاتر بياي واقعا كمكم ميكني . <br> _eos_<br>so i was thinking that if you came to the play , i think that would really help . <br> _eos_<br>so you can help me if you come with me to the theater . <br> _eos_<br>ميخواستي من را بكشي شريك . الانه كه وضعيت 211 بشه . <br> _eos_<br>do you want me murder , partner are you crazy , there has been a busy 211 . . <br> _eos_<br>you re going to kill me . . <br> _eos_<br>احمق ، گوش كن ، من فقط اونو برميدارم من ميدونم تو داري نگاه كني . <br> _eos_<br>silly of me . listen , i just picked this up . i know you like to look . <br> _eos_<br>idiot , listen , i just take it . i know you re watching . <br> _eos_<br>به نظر شبيه يک معماست ، چقدر احمقانه است ، چيزهاي قديمي مثل اون به درد چي ميخورن<br>must have seemed like a puzzle , what a silly , old thing like that was any good for . <br><br>it looks like a a . , its old , what what it like to do with him . <br> _eos_<br>و اگر تو اگر تو باور نداری ، جک ، اگر نمی تونی باور کنی ، ‏ .<br>_unk you if you do nt believe that , jack , if you ca nt believe that , . <br> _eos_<br>and if you do nt believe , , if you ca nt believe , , you ca nt believe , ,<br>لازم نیست که وقتی می تونه این بالا پیش تو باشه اون پایین بمونه . ‏ . <br><br>there s no need for him to be down there with me when he can be up here with you .<br>no need to be up with you when you can be up there . <br> _eos_<br>اما به همه ی اون پناهنده های بیچاره ای فکر کن که اگر من کمکشون نمی کردم اینجا<br>but think of all those poor refugees who must rot in this place if i did nt help them . <br><br>but you &#39;s all the poor refugees who if i did n&#39;t help them out . <br> _eos_</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=efa97896846" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ai-backyard/persian-to-english-translation-efa97896846">Persian to English translation</a> was originally published in <a href="https://medium.com/ai-backyard">AI Backyard</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Persian news Language model]]></title>
            <link>https://medium.com/ai-backyard/persian-news-language-model-9aed5891328e?source=rss----ede5667c68fb---4</link>
            <guid isPermaLink="false">https://medium.com/p/9aed5891328e</guid>
            <category><![CDATA[language-model]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[news]]></category>
            <dc:creator><![CDATA[Armin Behjati]]></dc:creator>
            <pubDate>Thu, 23 Aug 2018 18:50:29 GMT</pubDate>
            <atom:updated>2018-08-23T18:50:29.292Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/658/1*3Zd2BxLFq_i7wJ5EztZOGg.png" /></figure><h4>this is a joint affair of <a href="https://medium.com/@arminbehjati">Armin Behjati</a> and <a href="https://medium.com/@bahram.mp">Bahram Mohammadpour</a></h4><h4>News Crawler :</h4><p>in this news agency web site there is an “archive” page in which all published news are listed.in this page you can filter news by category and publish date.<br>if you open the link bellow you can see the archive page:</p><p><a href="http://www.entekhab.ir/fa/archive?service_id=18&amp;sec_id=0&amp;cat_id=0&amp;rpp=100&amp;from_date=1389/10/01&amp;to_date=1397/01/30&amp;p=1">http://www.entekhab.ir/fa/archive?service_id=18&amp;sec_id=0&amp;cat_id=0&amp;rpp=100&amp;from_date=1389/10/01&amp;to_date=1397/01/30&amp;p=1</a><br> <br>there are 3 important keywords in the url :<br>service_id : specifies category of news<br>rpp : specifies number of news in each search page<br>p : page number</p><p>we use bs4 to parse web pages and urllib3 to send http requests.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ee1f5b22828693c91e852d27502c6a25/href">https://medium.com/media/ee1f5b22828693c91e852d27502c6a25/href</a></iframe><p>here we define a function to fetch an archive web page given page number and category of news(service_id)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3a3332593d93c710f6f2dc8e07778062/href">https://medium.com/media/3a3332593d93c710f6f2dc8e07778062/href</a></iframe><p>the function bellow parses the output of the previous function and returns a list of news link.this function takes page number and news category(service_id) as inputs</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/edf929211ecc15ad2cdd18a2c51aa84a/href">https://medium.com/media/edf929211ecc15ad2cdd18a2c51aa84a/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c7434d8e9d7dfbe552bec21a0bfed2cb/href">https://medium.com/media/c7434d8e9d7dfbe552bec21a0bfed2cb/href</a></iframe><pre>(100, {&#39;service_id&#39;: 2, &#39;url&#39;: &#39;http://www.entekhab.ir/fa/news/389088&#39;})</pre><p>the next thing to do is to fetch data of each page and store it in mongoDB.<br>“get_data_of_page” does this job .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7bf30975eac28f8c2f866e07c9446af8/href">https://medium.com/media/7bf30975eac28f8c2f866e07c9446af8/href</a></iframe><p>“get_all_pages_data” function takes page number and service_id as inputs and thread number as an optional input and returns data of each news link in the archive page with news category = service_id</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e5fc8d962195106e490dedc1737b6af0/href">https://medium.com/media/e5fc8d962195106e490dedc1737b6af0/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b03f8337fd0e449743e2849989472eab/href">https://medium.com/media/b03f8337fd0e449743e2849989472eab/href</a></iframe><p>the next thing is to run a for loop on the above function and store the output to mongoDB.<br>bellow is a sample output of each documnet in mongoDB:</p><p>{‘url’: ‘<a href="http://www.entekhab.ir/fa/news/389088&#39;">http://www.entekhab.ir/fa/news/389088&#39;</a>,<br> ‘content’:”&lt;html&gt;…&lt;/html&gt;”,<br> ‘service_id’:2}<br> <br> <br>we collect news in these four categories</p><p>| category | service_id |<br>| — — — — — | — — — — — — |<br>|politics|2|<br>|economics|5|<br>|havades|10|<br>|art and culture|18</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bfa46e7f105c55f9ffccba8abe8f4a89/href">https://medium.com/media/bfa46e7f105c55f9ffccba8abe8f4a89/href</a></iframe><p>we change service_id and run this loop again to collect news in different categories</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9925b14a5097fb74181173b755cf9404/href">https://medium.com/media/9925b14a5097fb74181173b755cf9404/href</a></iframe><p>the next thing is to parse html data and store news data and corresponding service_id<br>so we define the news_parse function that takes html string as input and returns the text of news.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bbf2b946c25dff69cbb35607248ad1e3/href">https://medium.com/media/bbf2b946c25dff69cbb35607248ad1e3/href</a></iframe><p>we parse the data and grab text and service_id of each news.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3f7fad8bdd72b358ba6b5da52b91f9b0/href">https://medium.com/media/3f7fad8bdd72b358ba6b5da52b91f9b0/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/096c199ac367e80b3ff00e4e6889371b/href">https://medium.com/media/096c199ac367e80b3ff00e4e6889371b/href</a></iframe><pre>1000<br>2000<br>3000<br>4000<br>5000<br>6000<br>7000<br>8000<br>9000<br>10000<br>11000<br>12000<br>13000<br>14000<br>15000<br>16000<br>17000<br>18000<br>19000<br>20000<br>21000<br>22000<br>23000<br>24000<br>25000<br>26000<br>27000<br>28000</pre><p>the last step in data preparation is storing data.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/73d667c5220ffdaf010b0098ea56935a/href">https://medium.com/media/73d667c5220ffdaf010b0098ea56935a/href</a></iframe><h3>News Language model</h3><p>We are already familiar with the idea of using a pre trained network and adding some layers to the top and make it do something different in computer vision .<br>It’s the simple idea of a backbone plus a custom head that ables us almost do anything we can think about !<br>Here we try to apply the same idea to NLP .</p><p>Here we try to train a Language model on Persian news files and then use this model to train a classifier predicting the news categories .</p><p>Most of the code and models here were available thanks to fastai and Jeremy Howard .</p><p>We are going to use fastai.text here instead of torchtext which was very slow and confusing .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5f54a6c4a9cd5e25dcfbcd7ece4efcb5/href">https://medium.com/media/5f54a6c4a9cd5e25dcfbcd7ece4efcb5/href</a></iframe><p>Setting the path of our data and a path to store the language model :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/106a243b68c403f3c247c4094918915a/href">https://medium.com/media/106a243b68c403f3c247c4094918915a/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/86382391397574dcccb86f57ba85a3fc/href">https://medium.com/media/86382391397574dcccb86f57ba85a3fc/href</a></iframe><p>The news files from the crawler were stored in a Pandas dataframe named ‘news_out’.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/21a91e7434d260d03c5368566e63cfa1/href">https://medium.com/media/21a91e7434d260d03c5368566e63cfa1/href</a></iframe><p>as you can see there are some blanks here that we need get rid of .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/72b4339480aeac0544d9fcf9e3c51356/href">https://medium.com/media/72b4339480aeac0544d9fcf9e3c51356/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/452/1*jOZ9hZurLmcs8Z1ysS8eNQ.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e33550be78b4b0c868459d2aae8707d4/href">https://medium.com/media/e33550be78b4b0c868459d2aae8707d4/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e8a4660ce9ed297a5d79a8d0b49e4625/href">https://medium.com/media/e8a4660ce9ed297a5d79a8d0b49e4625/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*5U5Ny5z5uP-HaeBLmOE5_g.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a0bc8998dff22162ec14c92625bb8da5/href">https://medium.com/media/a0bc8998dff22162ec14c92625bb8da5/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0ca0cdeebec20f98e6e64029ad573159/href">https://medium.com/media/0ca0cdeebec20f98e6e64029ad573159/href</a></iframe><pre>(10171, 2)</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a490f848ba793f25f08b613932512b67/href">https://medium.com/media/a490f848ba793f25f08b613932512b67/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*I43YEb3eE75u8krGI-bz7Q.png" /></figure><p>In this part we try standardize the dataframe . each row has two columns for the news text and the news label.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/05246d0440199dd64b8458555a97677d/href">https://medium.com/media/05246d0440199dd64b8458555a97677d/href</a></iframe><pre><br>[&#39;text&#39;, &#39;label&#39;]</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1fdf536e6aa45c6d14bb7adf3215a5fd/href">https://medium.com/media/1fdf536e6aa45c6d14bb7adf3215a5fd/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/453/1*EU1jzGX72C7wx03HT1t9-g.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/01459c2a0635128b5bc2adbde8f6387a/href">https://medium.com/media/01459c2a0635128b5bc2adbde8f6387a/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*Edd7BaYlSf2ccI1wPWR9kw.png" /></figure><p>split_vals is a function we use to make validation set</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3fcddb14b1412ef6c8af4aef9f755f38/href">https://medium.com/media/3fcddb14b1412ef6c8af4aef9f755f38/href</a></iframe><p>we use 9000 news files for the train and the rest for validation</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b80a9d27dd39bb39e161be332aa13896/href">https://medium.com/media/b80a9d27dd39bb39e161be332aa13896/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/71dfce5c2e8893282b9e337554d7f664/href">https://medium.com/media/71dfce5c2e8893282b9e337554d7f664/href</a></iframe><pre>(1171, 2)</pre><p>We don’t need news labels for language model so we set’em all to zero .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1ea285c6240dfcb510b3535279a19764/href">https://medium.com/media/1ea285c6240dfcb510b3535279a19764/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d14ba11dadac3d3eeccdcc61d407d4a9/href">https://medium.com/media/d14ba11dadac3d3eeccdcc61d407d4a9/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2c2ef1bc71613771270f303ea6bbc4bc/href">https://medium.com/media/2c2ef1bc71613771270f303ea6bbc4bc/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/467/1*9DcYeMhjk3Q0xpay-vxGjg.png" /></figure><p>we should save the training and validation data for further use.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5cb6d22cdb77a0dad8c486bde60b2fc3/href">https://medium.com/media/5cb6d22cdb77a0dad8c486bde60b2fc3/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9c3ed44e790d37b71f98d96b482edcc4/href">https://medium.com/media/9c3ed44e790d37b71f98d96b482edcc4/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e0dff85c675d76c33c0a55e304b12d7d/href">https://medium.com/media/e0dff85c675d76c33c0a55e304b12d7d/href</a></iframe><pre>&#39;\xa0معاون بازاريابي فروش ايران خودرو با بيان اين كه اجراي روش هاي متنوع و جامع براي فروش محصولات يكي از مسيرهاي افزايش رضايتمندي مشتريان و فراهم كردن امكان خريد كالاي ايراني است تاكيد كرد: ايران خودرو در سال حمايت از كالاي ايراني برنامه هاي متنوعي را براي خريد محصولات به اجرا خواهد گذاشت. \xa0\xa0 \xa0\xa0به گزارش ايكوپرس، مصطفي خان كرمي با تشريح برنامه هاي فروش ايران خودرو در سال 97، اعمال تخفيف خريد كالاي ايراني را از جمله مشوق هاي خريد محصولات ايران خودرو دانست و اظهار كرد:\u200c با مطالعه و بررسي بازار و نيازسنجي از مشتريان برنامه هاي فروش را منطبق با خواست و سليقه آنان به اجرا مي گذاريم.وي افزود:\u200c طرح هاي فروش را متناسب با بودجه مشتريان در نظر مي گيريم و در اين راستا برنامه هاي فروش اقساطي و اعتباري را طرح ريزي مي كنيم تا همه گروه هاي درآمدي موفق به خريد محصولات ايراني ايران خودرو شوند.خان كرمي طرح هاي پيش فروش را از جمله طرح هاي فروش محصولات ايران خودرو نام برد و گفت: در پيش فروش هايی كه به صورت سرمايه گذاري پيشنهاد مي شود امكان مشاركت مشتريان در فرآيند توليد محصول ايجاد مي شود.وي درعين حال تصريح كرد: پرواضح است كه ايران خودرو نيز از مشاركت هاي مردمي در طرح هاي توليد بهره مند مي شود. با اين روش بازگشت سرمايه مشتريان تا موعد تحويل خودرو كاملا تضمين شده است.خان كرمي افزود:\u200c در طرح هاي پيش فروش عادي نيز امكان برنامه ريزي مالي بهتري را براي مشتريان فراهم مي كنيم.معاون بازاريابي و فروش ايران خودرو اضافه كرد: علاوه بر فروش هاي نقدي كه محصول با قيمت قطعي به دست مشتريان مي رسد، فروش اقساطي و اعتباري را نيز به اجرا مي گذاريم كه با اعمال شرايط منعطف، علاوه بر شناور بودن مبلغ پيش پرداخت، مشتريان قادر هستند زمان بازپرداخت اقساط را براساس شرايط مالي خود انتخاب كنند. | &#39;</pre><p>The next thing we need to do is to tokenize the text . unfortunately we couldn’t find any good persian tokenizer available so we used the spacy english tokenizer which worked fine for the task !<br>we pass ‘cunksize’ when we read the csv file .that means that pandas does not return a data frame, but it returns an iterator that we can iterate through chunks of a data frame.<br> That is why we don’t say tok_trn = get_text(df_trn) but instead we call get_all which loops through the data frame but actually what it’s really doing is it’s looping through chunks of the data frame so each of those chunks is basically a data frame representing a subset of the data .</p><p>Before we include the text, we have “beginning of stream” (BOS) token which we defined in the beginning.<br>So every text is going to start with ‘xbos’ because it’s often useful for the model to know when a new text is starting.</p><p>we tokenize it by doing a “process all multiprocessing” (proc_all_mp).</p><p>There is also a function called partition_by_cores which takes a list and splits it into sublists. The number of sublists is the number of cores that you have in your computer.</p><p>We make a list list of all the words that appear in some order.then we replace every word with its index into that list.the list of all the tokens, we call that the vocabulary.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f6d483f5caaaa18b509d1879f01a75c2/href">https://medium.com/media/f6d483f5caaaa18b509d1879f01a75c2/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/859c88180a39b62c6658d88ccd895f0a/href">https://medium.com/media/859c88180a39b62c6658d88ccd895f0a/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a689375dc254ee8a37f1878fc029a0d5/href">https://medium.com/media/a689375dc254ee8a37f1878fc029a0d5/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b3e686f52b57d13a95dbf7a282901a31/href">https://medium.com/media/b3e686f52b57d13a95dbf7a282901a31/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2de4a7418b7c0b69b7359f0002ad4e9f/href">https://medium.com/media/2de4a7418b7c0b69b7359f0002ad4e9f/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/08825798c5a8dc65635961c7220f5ac9/href">https://medium.com/media/08825798c5a8dc65635961c7220f5ac9/href</a></iframe><pre>0<br>1<br>2<br>3<br>4<br>5<br>6<br>7<br>8<br>0<br>1</pre><p>Here we just save the tokens .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3eed09f3f6f2c64c80024b4d503ed0e0/href">https://medium.com/media/3eed09f3f6f2c64c80024b4d503ed0e0/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c02158b2b2804897ca7ed2f7d4d0675d/href">https://medium.com/media/c02158b2b2804897ca7ed2f7d4d0675d/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bffa93c1ba2f403f6b56d045658988a4/href">https://medium.com/media/bffa93c1ba2f403f6b56d045658988a4/href</a></iframe><p>Python counter class gives us a list of unique items and their counts. Here are the 25 most common things in the vocabulary.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/15f6d69d1b14dcadb21bafbc05f54dbf/href">https://medium.com/media/15f6d69d1b14dcadb21bafbc05f54dbf/href</a></iframe><pre>[(&#39;و&#39;, 219034),<br> (&#39;در&#39;, 166423),<br> (&#39;به&#39;, 140454),<br> (&#39;از&#39;, 104388),<br> (&#39;که&#39;, 95511),<br> (&#39;این&#39;, 84752),<br> (&#39;را&#39;, 72305),<br> (&#39;با&#39;, 69094),<br> (&#39;:&#39;, 42844),<br> (&#39;است&#39;, 37288),<br> (&#39;برای&#39;, 29959),<br> (&#39;است.&#39;, 26304),<br> (&#39;می&#39;, 24125),<br> (&#39;آن&#39;, 22736),<br> (&#39;هم&#39;, 22181),<br> (&#39;یک&#39;, 19358),<br> (&#39;شده&#39;, 19201),<br> (&#39;ما&#39;, 17422),<br> (&#39;کرد&#39;, 17408),<br> (&#39;های&#39;, 17134),<br> (&#39;سال&#39;, 16955),<br> (&#39;بر&#39;, 16330),<br> (&#39;خود&#39;, 16176),<br> (&#39;گفت&#39;, 15925),<br> (&#39;کشور&#39;, 15450)]</pre><p>We use most_common, pass in the max vocab size and that’ll sort it by the frequency and if it appears less often than a minimum frequency, then leave it. That gives us itos — that’s the same name that torchtext used and it means integer-to-string. This is just the list of unique tokens in the vocab. We’ll insert two more tokens — a vocab item for unknown (_unk_) and a vocab item for padding (_pad_).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/78e312ef0dfc69ad715c3482e3801b6d/href">https://medium.com/media/78e312ef0dfc69ad715c3482e3801b6d/href</a></iframe><p><a href="https://gist.github.com/bahrammp/507004d8d310057835bc23503467a769">https://gist.github.com/bahrammp/507004d8d310057835bc23503467a769</a></p><p>stoi is a dictionary going in the opposite direction as the itos</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3dd0a27322437b10f85f87d8969ef843/href">https://medium.com/media/3dd0a27322437b10f85f87d8969ef843/href</a></iframe><pre>45303</pre><p>Now we call stoi for every word in sentences .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7f3cb4485c7b3d0aaeda7a99840a32a9/href">https://medium.com/media/7f3cb4485c7b3d0aaeda7a99840a32a9/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/08fa5aafd30f04d8c19bc41aeed50dce/href">https://medium.com/media/08fa5aafd30f04d8c19bc41aeed50dce/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3bb4be3620c39f68c0eb35f10e86f85b/href">https://medium.com/media/3bb4be3620c39f68c0eb35f10e86f85b/href</a></iframe><p>Now our vocab size is 45303 and our training language model has 9000 documents in it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d8d7e56440ac29fc3939bb7943c250df/href">https://medium.com/media/d8d7e56440ac29fc3939bb7943c250df/href</a></iframe><pre>(45303, 9000)</pre><h3>Language model</h3><p>As before we set our embedding size , number of hidden layers and number of layers .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b2fcef7aec0c407b49d5efd567162c04/href">https://medium.com/media/b2fcef7aec0c407b49d5efd567162c04/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5223b34a4423d6d6a78caae738df6ebd/href">https://medium.com/media/5223b34a4423d6d6a78caae738df6ebd/href</a></iframe><p>We take all our documents and just concatenate them back to back.we are going to try to predict the next word after these words. once we have a model data object, we can grab the model from it, so that’s going to give us a learner.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cfe1956c2cd680a87b7d4a3d6ebd6524/href">https://medium.com/media/cfe1956c2cd680a87b7d4a3d6ebd6524/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/babe8f9fc53bdc99e1aef998df81f4f2/href">https://medium.com/media/babe8f9fc53bdc99e1aef998df81f4f2/href</a></iframe><p>We grab a learner from ModelData object .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c60d9e22d8465a58afcb409fd483a901/href">https://medium.com/media/c60d9e22d8465a58afcb409fd483a901/href</a></iframe><p>we use the learning rate finder to find the best learning rate .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ce8568125d30231b170690842b92bd99/href">https://medium.com/media/ce8568125d30231b170690842b92bd99/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c6a900c1fb32dd0ae0004de41c45fea9/href">https://medium.com/media/c6a900c1fb32dd0ae0004de41c45fea9/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/395/1*97CRFCPwchmVv6EvsuI7Ww.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3e8462449cd5b618f6f92ea46852e564/href">https://medium.com/media/3e8462449cd5b618f6f92ea46852e564/href</a></iframe><p>Now we train the the model for few epochs .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/42dbfe03485608f1693e15a9c0c1bbbd/href">https://medium.com/media/42dbfe03485608f1693e15a9c0c1bbbd/href</a></iframe><pre>epoch      trn_loss   val_loss   accuracy                     <br>    0      4.148626   5.03116    0.226647  <br>    1      4.250771   5.047802   0.224713                     <br>    2      4.304603   5.040141   0.225326                     <br>    3      4.255172   5.036219   0.225684                     <br>    4      4.323801   5.016485   0.226707                     <br>    5      4.151685   5.030256   0.227471                     <br>    6      4.179784   5.024665   0.226944                     <br>    7      4.124536   5.018256   0.228459                     <br>    8      4.089547   5.018955   0.229405                     <br>    9      4.095399   5.018171   0.229185                     <br>    10     4.031234   5.021676   0.229044                     <br>    11     4.111572   5.00962    0.230145                     <br>    12     3.943506   5.016498   0.230516                     <br>    13     3.922315   5.016849   0.230894                     <br>    14     3.990867   5.018164   0.230943                     <br>    15     4.186847   5.023207   0.228214                     <br>    16     4.138712   5.062355   0.225501                     <br>    17     4.278202   5.038208   0.226603                     <br>    18     4.19066    5.041808   0.226961                     <br>    19     4.083084   5.050022   0.227127                     <br>    20     4.160091   5.035064   0.227734                     <br>    21     4.056128   5.048086   0.22812                      <br>    22     4.013799   5.045687   0.228869                     <br>    23     3.940494   5.049366   0.228921                     <br>    24     3.958642   5.04207    0.229198                     <br>    25     3.943375   5.044687   0.229329                     <br>    26     3.962328   5.038988   0.229918                     <br>    27     3.842061   5.044684   0.229866                     <br>    28     3.885955   5.042859   0.230633                     <br>    29     3.807452   5.039023   0.231055                     <br>    30     3.888656   5.08086    0.227317                     <br>    31     4.159625   5.071354   0.226402                     <br>    32     4.180851   5.059904   0.226543                     <br>    33     4.008345   5.076501   0.227153                     <br>    34     4.003859   5.071528   0.227382                     <br>    35     4.187835   5.037947   0.228434                     <br>    36     4.13018    5.037786   0.229056                     <br>    37     3.899381   5.069072   0.228455                     <br>    38     3.934846   5.065062   0.22942                      <br>    39     3.876984   5.068136   0.229285                     <br>    40     3.993476   5.054292   0.229019                     <br>    41     4.055646   5.042522   0.23002                      <br>    42     3.835831   5.06627    0.230388                     <br>    43     3.781686   5.0633     0.229969                     <br>    44     3.782634   5.061765   0.230488                     <br> 78%|███████▊  | 1700/2188 [04:06&lt;01:10,  6.90it/s, loss=3.71]</pre><h3>Generate text</h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cd4731a7e83a9186e8fb193e14798a6a/href">https://medium.com/media/cd4731a7e83a9186e8fb193e14798a6a/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0433830a069ebc95cc4782871ae86afa/href">https://medium.com/media/0433830a069ebc95cc4782871ae86afa/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5430dde90cf754fab6d51353a7ec2628/href">https://medium.com/media/5430dde90cf754fab6d51353a7ec2628/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7cf414e41505ca4b644bf03a866268bc/href">https://medium.com/media/7cf414e41505ca4b644bf03a866268bc/href</a></iframe><blockquote>کارشناس ارشد حوزه انرژی و صنایع دستی و گردشگری و … در این باره گفت : این طرح در حال حاضر در حال انجام است و امیدواریم تا پایان سال جاری این پروژه به اتمام برسد.</blockquote><p>Generated news seem fine !</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9aed5891328e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ai-backyard/persian-news-language-model-9aed5891328e">Persian news Language model</a> was originally published in <a href="https://medium.com/ai-backyard">AI Backyard</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Comment sentiment analysis]]></title>
            <link>https://medium.com/ai-backyard/comment-sentiment-analysis-702f66ceea8e?source=rss----ede5667c68fb---4</link>
            <guid isPermaLink="false">https://medium.com/p/702f66ceea8e</guid>
            <category><![CDATA[sentiment-analysis]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Armin Behjati]]></dc:creator>
            <pubDate>Mon, 06 Aug 2018 17:13:54 GMT</pubDate>
            <atom:updated>2018-08-06T17:13:53.903Z</atom:updated>
            <content:encoded><![CDATA[<p>This article is a joint affair of<strong> </strong><a href="https://medium.com/@arminbehjati"><strong>Armin Behjati</strong></a> and <a href="https://medium.com/@bahram.mp"><strong>Bahram Mohammadpour</strong></a></p><p>This project was done in March 2018</p><h3>Zoodfood comment crawler :</h3><p>crawling comments from zoodfood.</p><p>First we open the page containing restaurants list then we parse the extracted restaurant’s links there are 12 restaurants in each page. we define the function ‘get_restaurants_list’ to get list of restaurants for each page given the page number.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7a9f489200f3f53d25d749ed016777a7/href">https://medium.com/media/7a9f489200f3f53d25d749ed016777a7/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1bad7ecd905da9b0e54af276ec922df9/href">https://medium.com/media/1bad7ecd905da9b0e54af276ec922df9/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a54caf09cba1bfaf7e957535b4e123f3/href">https://medium.com/media/a54caf09cba1bfaf7e957535b4e123f3/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8d0ef840facdb41f5c79ffa4113a8e77/href">https://medium.com/media/8d0ef840facdb41f5c79ffa4113a8e77/href</a></iframe><p>zoodfood restaurants list has 288 pages so we fetch data for each page.<br>we can decrease time of excecution using threads.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/46427051478bb64ddb882ef940cb89eb/href">https://medium.com/media/46427051478bb64ddb882ef940cb89eb/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cc506e4720ef3afd28996ca6d653758c/href">https://medium.com/media/cc506e4720ef3afd28996ca6d653758c/href</a></iframe><p>we write links of restaurants to a text file so we can use them later</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b58568013ed98811b9db4bcbacc5c917/href">https://medium.com/media/b58568013ed98811b9db4bcbacc5c917/href</a></iframe><p>reading links from text file :</p><p>sample link : ‘/restaurant/menu/37j82x/kenzo/fereshteh’</p><p>we need the 3rd argument of this URL,which is called vendor name.it is an ID for the restaurant in snappfood API.</p><p>we just need the vendor name to collect comments.</p><p>we store vendor names in ‘restaurant_vendor_name’</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9826d2d8dc182f4b6954bd9e53684936/href">https://medium.com/media/9826d2d8dc182f4b6954bd9e53684936/href</a></iframe><p>url of comment API :</p><p>https://www.zoodfood.com/restaurant/comment/vendor/{vendor_id}/{page}<br> in each API call we get 10 comments and next page will get 10 next comments<br>if we request this URL : <a href="https://www.zoodfood.com/restaurant/comment/vendor/37j82x/0">https://www.zoodfood.com/restaurant/comment/vendor/37j82x/0</a><br> <br>we will get somthing like this :</p><p>{“status”:true,”data”:{“count”:435,”pageSize”:10,”comments”:[…]}}<br> <br>‘count’ shows the number of comments.<br>we should make (count/10) API calls to get all comments.</p><p>‘get_comments_for_restaurant_page’ function returns the comment data given the ‘vendor_name’ and ‘page number’.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/02effc09b6dc9a6f1bffcca0c4da7217/href">https://medium.com/media/02effc09b6dc9a6f1bffcca0c4da7217/href</a></iframe><p>we need a function to get all the work done . to simplify the process we define the function bellow but first we should connect to mongoDB.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b20c2524da7ed291f72879a118990e22/href">https://medium.com/media/b20c2524da7ed291f72879a118990e22/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ea3933181fc5289897564ac3291f8fd8/href">https://medium.com/media/ea3933181fc5289897564ac3291f8fd8/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b36cc399b2bde7a7cb9b615cd05b535d/href">https://medium.com/media/b36cc399b2bde7a7cb9b615cd05b535d/href</a></iframe><p>get all comments</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6de139e1ad812534368873ae018b4cdc/href">https://medium.com/media/6de139e1ad812534368873ae018b4cdc/href</a></iframe><p>we need a text file containing all comment texts.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dde43b9cfc865ed5a5ec2ebb84d992e6/href">https://medium.com/media/dde43b9cfc865ed5a5ec2ebb84d992e6/href</a></iframe><h4>Language Model:</h4><p>We want to make a classifier to predict the restaurant comments emotions .</p><p>all the comments are in persian and unfortunately there are no good pre-trained language models available for persian. so we have to train our own . after training the persian language model and understanding the persian language structures we will try to recognize the emotion behind each comment .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3064d275d32941efcf3d61a5a3f93fee/href">https://medium.com/media/3064d275d32941efcf3d61a5a3f93fee/href</a></iframe><p>specifying the PATH :<br>PATH_lang : language model PATH<br>PATH : labeled data for sentiment analysis PATH</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/78a891a64842698c03fce5bf5b904f3c/href">https://medium.com/media/78a891a64842698c03fce5bf5b904f3c/href</a></iframe><p>Specifying the train and validation path for both language model and labeled data :<br>TRN : path for traing labeled data<br>VAL = path for validation labeled data<br>TRN_lang :language model training data path<br>VAL_lang = language model validation data path</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4a6c078908c015b81cbda29b0259e903/href">https://medium.com/media/4a6c078908c015b81cbda29b0259e903/href</a></iframe><p>making train and validation folders containing language model training and validation files.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9128fd530c59340dbde3bde0b567e375/href">https://medium.com/media/9128fd530c59340dbde3bde0b567e375/href</a></iframe><p>counting the words in train and validation dataset :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3922315d2dbbe3e91a252cdca423048d/href">https://medium.com/media/3922315d2dbbe3e91a252cdca423048d/href</a></iframe><pre>2120494</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/620fbba445692968381144b852e027c8/href">https://medium.com/media/620fbba445692968381144b852e027c8/href</a></iframe><pre>791984</pre><p>Tokenizing the text :<br>before we can work withe the text we must first convert it to an array of tokens(or words).<br>for tokenizing the text we use an open source tokenizer “spacy” which supports persian language too . although the persian tokenizer is not very great it seemed sufficient enough for this project .though it needs some improvements.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/65178fe640609eda2a4a92e77897e9dd/href">https://medium.com/media/65178fe640609eda2a4a92e77897e9dd/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/59a5616a9ee71c71271d1d3a0fd09b79/href">https://medium.com/media/59a5616a9ee71c71271d1d3a0fd09b79/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7f72e7f68e4f1cea272e47e2195f9837/href">https://medium.com/media/7f72e7f68e4f1cea272e47e2195f9837/href</a></iframe><pre>&#39;سس اسپرینگ رول اشتباه ارسال شده بود&#39;</pre><p>checking the tokenizer on one comment from the dataset. <br>every token is seperated with a “?” mark .<br>everything seems to work fine .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8f692506a103edbd3214d2c129f571b6/href">https://medium.com/media/8f692506a103edbd3214d2c129f571b6/href</a></iframe><pre>&#39;میتونم?بگم?هیچ?ارزونی?ای?بی?علت?نیست?.?کیفیت?پایین?ولی?با?توجه?به?مثل?هر?چقدر?پول?بدی?اش?میخوری?میشه?گفت?قابل?قبول?ولی?پیتزا?پدر?خوب?نارمک?تویه?همین?رنج?قیمت?کیفیت?بهتری?داره&#39;</pre><p>we are using fastai library which works with torchtext . so we have to make a torchtext field to pass it to the LnguageModelData object .<br>here we tell it how to preprocess the text . simply making it lowercase and using spacy for tokenizing.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c557a5427c85ba9e3241b9c4f7cc9f48/href">https://medium.com/media/c557a5427c85ba9e3241b9c4f7cc9f48/href</a></iframe><p>setting the batch size and backprop through time values.<br>*bptt simply means how many layers we will backprop through.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d33e20368c63d256cea07aca0f629dd/href">https://medium.com/media/6d33e20368c63d256cea07aca0f629dd/href</a></iframe><p>after building the ModelData , ModelData object fills the attributes of the TEXT object.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c9183bedc63ecbc40a288ffcfc3b1e7d/href">https://medium.com/media/c9183bedc63ecbc40a288ffcfc3b1e7d/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/24ed0012f9ed70a182a8fac8eb1b8b79/href">https://medium.com/media/24ed0012f9ed70a182a8fac8eb1b8b79/href</a></iframe><pre>(491, 7955, 1, 2364421)</pre><p>first twelve elements of the map from integers to unique tokens:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8889e4f4acf5cc06308bf6cad89a18cd/href">https://medium.com/media/8889e4f4acf5cc06308bf6cad89a18cd/href</a></iframe><pre>[&#39;&lt;unk&gt;&#39;,<br> &#39;&lt;pad&gt;&#39;,<br> &#39;بود&#39;,<br> &#39;و&#39;,<br> &#39;غذا&#39;,<br> &#39;خوب&#39;,<br> &#39;از&#39;,<br> &#39;به&#39;,<br> &#39;خیلی&#39;,<br> &#39;هم&#39;,<br> &#39;عالی&#39;,<br> &#39;که&#39;]</pre><p>numericalize is one of TEXT attributes that handles turning tokens to integers in a text.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7211d0ef6f83540d64e4a5150e24c00a/href">https://medium.com/media/7211d0ef6f83540d64e4a5150e24c00a/href</a></iframe><pre>Variable containing:<br>  170<br>    7<br>  347<br>    0<br>  537<br>   17<br>  134<br>  454<br> 1869<br>    7<br>  491<br>    0<br>[torch.cuda.LongTensor of size 12x1 (GPU 0)]</pre><p>the LanguageModelData object creates batches of 64 columns (that’s our batch size) and squence lengths of around 75 tokens(that’s the bptt we defined ) <br>Each batch also contains the exact same data as labels, but one word later in the text since we’re trying to always predict the next word. The labels are flattened into a 1d array.<br>because we can’t shuffle the text files , torchtext automatically changes the bptt value a little bit to add the element of randomness.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ced667392dc80c1dffef6861c485d105/href">https://medium.com/media/ced667392dc80c1dffef6861c485d105/href</a></iframe><pre>(Variable containing:<br>   170    41     5  ...      2   308   247<br>     7    98     2  ...     26     3    29<br>   347    30     3  ...     42    21   279<br>        ...          ⋱          ...       <br>     5  2712     2  ...      3   373   163<br>     2    37    18  ...     74  3290    51<br>    18  2449   451  ...    109  1155   112<br> [torch.cuda.LongTensor of size 73x64 (GPU 0)], Variable containing:<br>     7<br>    98<br>     2<br>   ⋮  <br>    75<br>    48<br>     0<br> [torch.cuda.LongTensor of size 4672 (GPU 0)])</pre><p>em_sz : the embedding vector size <br>nh : number of activations in each layer<br>nl : number of layers</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2113dd3b13e9478f071a883ce27de1ab/href">https://medium.com/media/2113dd3b13e9478f071a883ce27de1ab/href</a></iframe><p>the optimizer function <br>the 0.7 momentum value works best so we’re not using the default value of 0.9 .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e29a332e39ae0b31003faeee2a403c16/href">https://medium.com/media/e29a332e39ae0b31003faeee2a403c16/href</a></iframe><p>fastai uses a variant of the state of the art AWD LSTM Language Model developed by Stephen Merity. so that’s the model we’re using too .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/868db32c438fec84abd1ec2132b69f76/href">https://medium.com/media/868db32c438fec84abd1ec2132b69f76/href</a></iframe><p>we train the model a little bit :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1a2895f45c744f26e4d048feead7af46/href">https://medium.com/media/1a2895f45c744f26e4d048feead7af46/href</a></iframe><pre>HBox(children=(IntProgress(value=0, description=&#39;Epoch&#39;, max=75), HTML(value=&#39;&#39;)))</pre><pre>epoch      trn_loss   val_loss                              <br>    0      6.263493   6.211119  <br>    1      5.307442   5.09662                               <br>    2      5.006752   4.829635                              <br>    3      4.864222   4.715991                              <br>    4      4.826177   4.692709                              <br>    5      4.64845    4.495593                              <br>    6      4.478897   4.357899                              <br>    7      4.363075   4.269142                              <br>    8      4.283838   4.214092                              <br>    9      4.214998   4.175522                              <br>    10     4.162878   4.152645                              <br>    11     4.130756   4.13245                               <br>    12     4.104955   4.122128                              <br>    13     4.100972   4.117318                              <br>    14     4.087452   4.115951                              <br>    15     4.19065    4.151838                              <br>    16     4.134897   4.129659                              <br>    17     4.121771   4.120149                              <br>    18     4.071009   4.110334                              <br>    19     4.039296   4.102594                              <br>    20     4.021478   4.095991                              <br>    21     3.978974   4.087306                              <br>    22     3.95643    4.079997                              <br>    23     3.939815   4.078523                              <br>    24     3.937055   4.074559                              <br>    25     3.900932   4.069588                              <br>    26     3.850793   4.075124                              <br>    27     3.855999   4.065688                              <br>    28     3.80646    4.072929                              <br>    29     3.809924   4.069959                              <br>    30     3.80369    4.067943                              <br>    31     3.812806   4.067925                              <br>    32     3.807241   4.067443                              <br>    33     3.768194   4.069095                              <br>    34     3.761686   4.070869                              <br>    35     3.954122   4.091923                              <br>    36     3.935312   4.091307                              <br>    37     3.938833   4.092941                              <br>    38     3.918598   4.088365                              <br>    39     3.897253   4.091039                              <br>    40     3.876273   4.094545                              <br>    41     3.86413    4.096863                              <br>    42     3.863323   4.088464                              <br>    43     3.841176   4.09819                               <br>    44     3.870419   4.084446                              <br>    45     3.831362   4.09162                               <br>    46     3.804627   4.102284                              <br>    47     3.808969   4.085278                              <br>    48     3.805639   4.087205                              <br>    49     3.777925   4.095012                              <br>    50     3.751137   4.1057                                <br>    51     3.761073   4.0914                                <br>    52     3.71945    4.108792                              <br>    53     3.705185   4.110999                              <br>    54     3.69791    4.110427                              <br>    55     3.691067   4.108355                              <br>    56     3.672802   4.113387                              <br>    57     3.68202    4.103521                              <br>    58     3.649078   4.115421                              <br>    59     3.676602   4.09705                               <br>    60     3.630621   4.114202                              <br>    61     3.613245   4.121602                              <br>    62     3.615716   4.118817                              <br>    63     3.629888   4.109546                              <br>    64     3.601175   4.118355                              <br>    65     3.578798   4.127409                              <br>    66     3.569921   4.126852                              <br>    67     3.564669   4.127878                              <br>    68     3.569504   4.126769                              <br>    69     3.558655   4.127853                              <br>    70     3.573381   4.126194                              <br>    71     3.559762   4.128838                              <br>    72     3.572878   4.128811                              <br>    73     3.578699   4.130381                              <br>    74     3.545339   4.130868</pre><pre>[4.130867616921622]</pre><p>We save the embeddings for further use.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b63b516b0d51f7ec4f71dcb30da0ccb6/href">https://medium.com/media/b63b516b0d51f7ec4f71dcb30da0ccb6/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3dd08e8451d62dd45b7273b95e8d7147/href">https://medium.com/media/3dd08e8451d62dd45b7273b95e8d7147/href</a></iframe><p>Let’s test the language model to see how it works :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e4d8215bd399951e22dd8a69c66dfe75/href">https://medium.com/media/e4d8215bd399951e22dd8a69c66dfe75/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/abcaa7c1718cb971eff1d8ac8359e579/href">https://medium.com/media/abcaa7c1718cb971eff1d8ac8359e579/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7a215b515ede63ee4b2192dcb5b9f90f/href">https://medium.com/media/7a215b515ede63ee4b2192dcb5b9f90f/href</a></iframe><pre>[&#39;&lt;unk&gt;&#39;, &#39;بود&#39;, &#39;و&#39;, &#39;عالی&#39;, &#39;خوب&#39;, &#39;با&#39;, &#39;هم&#39;, &#39;که&#39;, &#39;به&#39;, &#39;خیلی&#39;]</pre><p>It seems to be working fine !</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6dadd012942c5675cf7e7d433dcb6c7b/href">https://medium.com/media/6dadd012942c5675cf7e7d433dcb6c7b/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e0f344b0c1b4a2c3851fb7216f9aa02a/href">https://medium.com/media/e0f344b0c1b4a2c3851fb7216f9aa02a/href</a></iframe><pre>کباب &lt;unk&gt; خیلی خوشمزه و خوب بود. فقط ای کاش &lt;unk&gt; که تو منو زده بودن با چیزی ک ارسال شد ...</pre><h3>Sentiment Analysis:</h3><p>Here we define our fastai/torchtext dataset :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6504f501ffd96a780e94ee94e35ceee9/href">https://medium.com/media/6504f501ffd96a780e94ee94e35ceee9/href</a></iframe><p>sequential=False means that the text should be tokenized first.<br>splits is a torchtext method that creates train, test, and validation sets.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/055c6197fdde6faef1ccf66623d2bd7a/href">https://medium.com/media/055c6197fdde6faef1ccf66623d2bd7a/href</a></iframe><p>using fastai we can create ModelData object from torchtext splits.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4fe1e48955aebd84d89cb0d7262776c1/href">https://medium.com/media/4fe1e48955aebd84d89cb0d7262776c1/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/490091f019c8880474f600ed7cacc7dd/href">https://medium.com/media/490091f019c8880474f600ed7cacc7dd/href</a></iframe><p>fine-tuning pretrained models gives the opportunity to use differential learning rates .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4216e0a2569cf03b4c7ce9040498a8de/href">https://medium.com/media/4216e0a2569cf03b4c7ce9040498a8de/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/490091f019c8880474f600ed7cacc7dd/href">https://medium.com/media/490091f019c8880474f600ed7cacc7dd/href</a></iframe><pre>HBox(children=(IntProgress(value=0, description=&#39;Epoch&#39;, max=5), HTML(value=&#39;&#39;)))</pre><pre>epoch      trn_loss   val_loss   accuracy                       <br>    0      0.914599   1.026246   0.526914  <br>    1      0.892305   1.073445   0.526914                       <br>    2      0.890372   1.029869   0.526914                       <br>    3      0.895171   1.018384   0.526914                       <br>    4      0.907669   1.034427   0.526914</pre><pre>HBox(children=(IntProgress(value=0, description=&#39;Epoch&#39;, max=25), HTML(value=&#39;&#39;)))</pre><pre>epoch      trn_loss   val_loss   accuracy                      <br>    0      0.726858   0.807458   0.658791  <br>    1      0.691951   0.799014   0.674747                      <br>    2      0.672657   0.861043   0.665392                      <br>    3      0.666511   0.858039   0.677368                      <br>    4      0.659266   0.843153   0.678958                      <br>    5      0.687026   0.764765   0.669744                      <br>    6      0.6328     0.80844    0.670203                      <br>    7      0.643107   0.885304   0.672158                      <br>    8      0.625191   0.980532   0.674329                      <br>    9      0.627007   0.818966   0.679019                      <br>    10     0.662649   0.773294   0.658341                      <br>    11     0.651018   0.732899   0.693246                      <br>    12     0.640039   0.774128   0.681997                      <br>    13     0.646645   0.818812   0.685979                      <br>    14     0.629662   0.818131   0.687009                      <br>    15     0.627583   1.013715   0.684953                      <br>    16     0.620577   0.789786   0.691546                      <br>    17     0.621732   0.755267   0.689668                      <br>    18     0.616339   0.812259   0.69037                       <br>    19     0.628711   0.816256   0.685763                      <br>    20     0.631954   0.869854   0.663502                      <br>    21     0.648067   0.792397   0.689935                      <br>    22     0.621395   0.811201   0.687528                      <br>    23     0.627996   0.75971    0.692931                      <br>    24     0.61478    0.766722   0.694669</pre><pre>[0.7667221994412634, 0.6946690829240649]</pre><p>Accuracy :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/40136ef129179bf606f9c7267d551b99/href">https://medium.com/media/40136ef129179bf606f9c7267d551b99/href</a></iframe><pre>0.7319195114773396</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bf88ed19ff1dea8c35156dc97a893512/href">https://medium.com/media/bf88ed19ff1dea8c35156dc97a893512/href</a></iframe><pre>HBox(children=(IntProgress(value=0, description=&#39;Epoch&#39;, max=5), HTML(value=&#39;&#39;)))</pre><pre>epoch      trn_loss   val_loss   accuracy                       <br>    0      0.605273   0.795732   0.697832  <br>    1      0.605745   0.737048   0.701651                       <br>    2      0.61152    0.726605   0.705618                       <br>    3      0.618649   0.898131   0.680639                       <br>    4      0.607915   0.835565   0.684423</pre><pre>HBox(children=(IntProgress(value=0, description=&#39;Epoch&#39;, max=25), HTML(value=&#39;&#39;)))</pre><pre>epoch      trn_loss   val_loss   accuracy                      <br>    0      0.63425    0.712354   0.70744   <br>    1      0.625679   0.7335     0.696126                      <br>    2      0.598707   0.762889   0.694836                      <br>    3      0.618014   0.803139   0.694313                      <br>    4      0.592371   0.827738   0.683457                      <br>    5      0.593831   0.758763   0.684881                      <br>    6      0.612648   0.803343   0.663893                      <br>    7      0.613446   0.759131   0.694913                      <br>    8      0.590957   0.866204   0.691297                      <br>    9      0.59238    1.015298   0.688887                      <br>    10     0.631985   0.876454   0.65384                       <br>    11     0.610506   0.83315    0.689219                      <br>    12     0.609516   0.753415   0.695011                      <br>    13     0.612919   0.785118   0.691572                      <br>    14     0.59885    0.770415   0.695097                      <br>    15     0.59031    0.845301   0.677381                      <br>    16     0.616533   0.770404   0.684773                      <br>    17     0.60326    0.84417    0.691907                      <br>    18     0.587279   0.800148   0.688964                      <br>    19     0.611245   0.831045   0.689919                      <br>    20     0.604469   0.737693   0.707982                      <br>    21     0.594355   0.855759   0.676296                      <br>    22     0.600001   0.890678   0.679592                      <br>    23     0.578646   0.789512   0.691539                      <br>    24     0.606092   0.871126   0.687559</pre><pre>[0.8711255853035156, 0.6875590290895354]</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/47bf5d205c84957f8cc60d5b00fec1a6/href">https://medium.com/media/47bf5d205c84957f8cc60d5b00fec1a6/href</a></iframe><pre>HBox(children=(IntProgress(value=0, description=&#39;Epoch&#39;, max=1), HTML(value=&#39;&#39;)))</pre><pre>61%|██████    | 1335/2188 [00:40&lt;00:25, 33.37it/s, loss=2.29]</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e7e87258d1cea748db5149a2a026dc88/href">https://medium.com/media/e7e87258d1cea748db5149a2a026dc88/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*6foEt8OvgyxGBs14oAu24g.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=702f66ceea8e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ai-backyard/comment-sentiment-analysis-702f66ceea8e">Comment sentiment analysis</a> was originally published in <a href="https://medium.com/ai-backyard">AI Backyard</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>