<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Namita Atri on Medium]]></title>
        <description><![CDATA[Stories by Namita Atri on Medium]]></description>
        <link>https://medium.com/@namita.atri?source=rss-9a1ab64aa75b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*GNFHtQCJ70r-IOsS.jpg</url>
            <title>Stories by Namita Atri on Medium</title>
            <link>https://medium.com/@namita.atri?source=rss-9a1ab64aa75b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 22:04:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@namita.atri/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Abstractive-Extractive Text Summarization using Reinforcement Learning]]></title>
            <link>https://medium.com/@namita.atri/abstractive-extractive-text-summarization-using-reinforcement-learning-ca1103691533?source=rss-9a1ab64aa75b------2</link>
            <guid isPermaLink="false">https://medium.com/p/ca1103691533</guid>
            <category><![CDATA[text-summarization]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[news-summary]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Namita Atri]]></dc:creator>
            <pubDate>Tue, 30 Apr 2019 09:31:21 GMT</pubDate>
            <atom:updated>2019-04-30T09:31:21.900Z</atom:updated>
            <content:encoded><![CDATA[<p>Sequence to sequence (Seq2Seq) learning has recently been used for abstractive and extractive summarization. For our project, we have developed a model to produce news summaries using the BBC dataset. We propose a novel document-context based Seq2Seq models using RNNs for abstractive and extractive summarizations. Intuitively, this is similar to news summary mobile applications that employ human editors to translate news reports to summaries. We use this idea and propose that Seq2Seq models should employ contextual information to obtain better summaries. Our purpose for doing so is to automate the process of content-creation of the aforementioned news applications.</p><p>Document summarization has two main models, extractive, and the other abstractive. The former method directly chooses the important sentences in the original document and is chosen essentially for the speed and if the application requires this exact functionality. The other one essentially involves rewriting the text to produce a summary, and has seen a lot of rise in the recent times after the advent of sequence-to-sequence neural networks.</p><p>There are a few problems with an abstractive-only paradigm. They are slow as the attention model goes over all the encoded words in a very long paragraph sequentially. Also, they produce redundancies, especially when trying to generate multiple sentences. However, there is a clear advantage of using abstractive models that they produce concise results.</p><p>To combine the advantages of both the models, and address the issues seen in them, we decided to follow a hybrid extractive-abstractive structure. We have also implemented a policy-based reinforcement learning model to connect these two architectures. The extractive model first selects the salient sentences from the given text and the abstractive model then rewrites these to produce the final summary. Next, we need to learn the sentence saliency, for which we use policy gradient with sentence-level rewards to connect these two models. We have used a word-sentence hierarchy to deal with the structural aspects of the language, as well as parallelize the modelling.</p><p>Our abstractor is a simple encoder-aligner-decoder model which is trained on sentence pairs obtained via simple automatic matching criteria from the summary.</p><p>Essentially, our model adopts a human-inspired coarse-to-fine approach that first extracts all the salient sentences and then rewrites them in parallel.</p><p>We analyse the performance of our model using ROUGE, which is basically a software package used for evaluating automatic summarization and machine translation software in natural language processing.</p><h3>Related Work</h3><p>Reinforcement learning has been used in earlier works to optimize the non-differentiability of language generation (Ranzato et al., 2016; Bahdanau et al., 2017). Paulus et al. (2018) employ policy gradient RL methods for abstractive summarization, utilizing sequence-level metric rewards (Ranzato et al., 2016). We use sentence-level rewards to optimize the extractor while keeping our ML trained abstractor decoder fixed, so as to achieve the best of both worlds.</p><h3>Methodology and Approach</h3><p>Formally, given a training set of document-summary pairs <em>{xᵢ , yᵢ}ᵢᴺ=1</em>, our goal is to approximate the function <em>h : X → Y, X = {xᵢ}ᵢᴺ=1, Y = {yᵢ}ᵢᴺ=1</em> such that <em>h(xᵢ) = yᵢ , 1 ≤ i ≤ N</em>.</p><p>Furthermore, we define an abstracting function <em>g</em> as: <em>∀s ∈ Sᵢ , ∃d ∈ Dᵢ</em> such that <em>g(d) = s, 1 ≤ i ≤ N</em>, where <em>Sᵢ</em> is the set of summary sentences in <em>xᵢ</em> and <em>Dᵢ</em> the set of document sentences in <em>yᵢ</em>. i.e., in any given pair of document and summary, every summary sentence can be produced from some document sentence. We can further define another function <em>f : X → Dₙ</em> that satisfies <em>f(x) = {dₜ}ⁿⱼ=1 and y = h(x) =[g(d₁), g(d₂), . . . , g(dₙ)]</em>, where <em>g(d₁), g(d₂), . . . , g(dₙ) </em>are concatenated. This latent function <em>f</em> can be seen as an extractor that chooses the important sentences in a given document which the abstractor will rewrite in order to produce a corresponding summary.</p><h3>Extractor Agent</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ajpD7vh4Pid1R9wKV5pb_w.png" /><figcaption>Extractor Model</figcaption></figure><p>We use a temporal convolutional model to compute <em>rⱼ</em>, the representation of each individual sentence in the documents. To also integrate a whole sense of the document in the summary, a bidirectional LSTM-RNN is applied on the convolutional output. This enables learning a strong representation, denoted as <em>hⱼ</em> (for the <em>j</em>-th sentence in the document), that takes into account the context of all previous and future sentences in the same document.</p><p>Next, we add another LSTM-RNN to extract sentences recurrently. At each time step t, the decoder performs a 2-hop attention mechanism: It first looks at all <em>hⱼ</em>’s to get a context vector et and then attends to <em>hⱼ</em> ’s again for the extraction probabilities. This model is essentially classifying all sentences of the document at each extraction step.</p><h3>Abstractor Network</h3><p>The abstractor network approximates <em>g</em>, which as we saw earlier, squeezes and rescripts an extracted sentence to a summary sentence. We use the standard encoder-aligner-decoder model to implement the same.</p><h3>Learning</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OfstoOAhdMgeu3klCHaAUg.png" /><figcaption>Learning Process</figcaption></figure><p>We have used standard policy gradient methods to bridge the backpropagation and to allow end-to-end training. A randomly-initialized network causes the extractor to select sentences that are irrelevant, which makes it difficult for the abstractor to produce effective summaries. Also, since the abstractor is not properly trained, the extractor receives dubious rewards, which leads to a bad estimate of the policy gradient and consequently a sub-optimal policy. Therefore, we optimized each sub-module separately: train the extractor to select salient sentences (fit <em>f</em>) and the abstractor to generate shortened summary (fit <em>g</em>). Finally, RL is applied to train the full model end-to-end (fit <em>h</em>).</p><h3>Maximum-Likelihood Training for Submodules</h3><p>Most of the summarization datasets are end-to-end document-summary pairs without extraction (saliency) labels for each sentence. For each ground-truth summary sentence, we find the most similar document sentence <em>dⱼₜ</em> by:</p><p><em>jₜ = argmaxᵢ (ROUGE-Lrecall (dᵢ , sₜ))</em></p><p>Given these proxy training labels, the extractor is then trained to minimize the cross-entropy loss.</p><p>For the abstractor training, we create training pairs by taking each summary sentence and pairing it with its extracted document sentence. The network is trained to minimize the cross-entropy loss of the decoder model at each generation step.</p><p>After this, we apply policy-based reinforcement learning to our complete model. Intuitively, our RL training works as follow: If the extractor chooses a good sentence, after the abstractor rewrites it, the ROUGE match would be high and thus the action is encouraged. If a bad sentence is chosen, though the abstractor still produces a summary version of it, the compressed result would not match the ground truth and the low ROUGE score discourages this action.</p><p>We use RL as a saliency guide without altering the abstractor’s language model, while previous work applied RL on the word-level, which could affect the language fluency. In the RL training phase, we add another set of trainable parameters <em>vₑₒₑ</em> (<em>eoe</em> stands for ‘End-Of-Extraction’) with the same dimensions as the sentence representation. The decoder treats <em>vₑₒₑ</em> as one of the extraction candidates and hence naturally results in a stop action in the stochastic policy. We set the reward for the agent performing <em>eoe</em> to <em>ROUGE-F₁ ([{g(dⱼₜ)}ₜ ], [{sₜ}ₜ ])</em>; whereas the agent receives zero reward for any unwanted extraction step. The model is therefore encouraged to extract when there are still remaining ground-truth summary sentences, to accumulate intermediate reward, and to learn to stop by optimizing a global ROUGE and avoiding extra extraction.</p><h3>Results</h3><p>We trained a model on our dataset (BBC News) to be treated as the baseline. We also refer to the metrics achieved by the model on a larger dataset as the global standard, also mentioned here for comparison.</p><p>Text Summarization resulted in a ROUGE-L score of 30.72 ,after comparison with the baseline model which produced scores of 34.28 on the same dataset, and 37.87 on CNN/Daily Mail news dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KACbV4UHo46lf3Lk1YvAqQ.png" /><figcaption>Results comparison of all models</figcaption></figure><p>Training on CNN/Daily Mail, which is a much larger dataset should significantly improve these numbers.</p><p>An example of the summarization produced is mentioned in the images below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7T1NPuAWgsNyTXawtrNfJA.png" /><figcaption>Original news article</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BgKT-buv44utfxAdKK_XnQ.png" /><figcaption>Corresponding summary</figcaption></figure><p>As seen in the example summary above, the result is quite descriptive and appropriate for our purpose and intent, which is to summarize a news article so as to automate the content-writing of news summary apps. This is opposed to smaller news summaries that were produced using the baseline model.</p><h3>Future Work</h3><p>In the future, we hope to improve the performance of the ASR model, after which we would like to combine the two models of ASR and text-summarization.</p><p>We would like to explore better methods for implementing audio summarization</p><h3>References</h3><p><a href="https://arxiv.org/pdf/1807.08000.pdf">Abstractive and Extractive Text Summarization using Document Context Vector and Recurrent Neural Networks</a></p><p><a href="https://aclweb.org/anthology/D15-1044">A Neural Attention Model for Sentence Summarization</a></p><p><a href="https://arxiv.org/pdf/1807.02305.pdf">Neural Document Summarization by Jointly Learning to Score and Select Sentences</a></p><p><a href="https://arxiv.org/pdf/1807.02305.pdf">Towards Automatic Text Summarization: Extractive Methods</a></p><p><a href="https://arxiv.org/pdf/1803.10357.pdf">Deep Communicating Agents for Abstractive Summarization</a></p><p><a href="https://www.aclweb.org/anthology/K17-1021">A Supervised Approach to Extractive Summarisation of Scientific Papers</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ca1103691533" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>