<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Adria M. on Medium]]></title>
        <description><![CDATA[Stories by Adria M. on Medium]]></description>
        <link>https://medium.com/@adriamoyaortiz?source=rss-320e6bfc736------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ES5_QVfapHAMhJo4bF07Tw.jpeg</url>
            <title>Stories by Adria M. on Medium</title>
            <link>https://medium.com/@adriamoyaortiz?source=rss-320e6bfc736------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 21:01:12 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@adriamoyaortiz/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Build a financial newsletter with Deep Learning and AWS — Part 2]]></title>
            <link>https://medium.com/@adriamoyaortiz/classifying-news-to-build-a-newsletter-with-deep-learning-part-2-cd6104fa7317?source=rss-320e6bfc736------2</link>
            <guid isPermaLink="false">https://medium.com/p/cd6104fa7317</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[news]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Adria M.]]></dc:creator>
            <pubDate>Sun, 20 Jan 2019 17:36:43 GMT</pubDate>
            <atom:updated>2019-01-20T22:11:09.069Z</atom:updated>
            <content:encoded><![CDATA[<p>This is a continuation of my <a href="https://medium.com/@adriamoyaortiz/classifying-news-articles-to-build-a-customized-newsletter-automatically-ede6ad9ef725">previous post</a>, where I outlined the logic behind the news classifier. This second part covers the deployment of the application on AWS (some knowledge is useful to follow along).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*anZnZMacG-nGeoaetW8a9w.png" /><figcaption>Newsletter application structure on AWS.</figcaption></figure><p>The picture above shows the different AWS services used to deploy the application:</p><ul><li><strong>CloudWatch</strong>: for monitoring and scheduling the crawling process.</li><li><strong>Lambda</strong>: a handler function that dispatches the crawlers.</li><li><strong>S3</strong>: for storing scrapped article urls and trained tokenizers and classifiers used to predict which articles are candidate for the newsletter.</li><li><strong>EC2</strong>: for downloading the content of articles (title, summary, text, keywords, …), predicting and clustering candidate articles and building the newsletter and sending it.</li><li><strong>RDS</strong>: for storing articles included in the newsletter.</li></ul><p>Notes: the main idea was to run the whole application using a serverless framework but, by the time this was put into production, Lambda’s time, size and computational limitations made it impossible to run the classifiers there. Therefore, the second most optimal solution was to start and stop an EC2 instance to run the resource-intensive jobs. Additionally, I opted for RDS because there were no budget limitations, but given that only a bunch of articles are stored everyday, one could have decided to go for a cheaper solution, like S3.</p><p>Now that time has passed since I built this and I see it in perspective, perhaps I would have approached the application differently — the design is a bit monolithic and it could have benefited from using a micro-services structure. For example, having multiple services such as crawlers, classifiers, email builder, …</p><p>With this in mind, let’s dive into the details of the current deployment. There are two main Python projects (hosted by different AWS services):</p><ol><li>Lambda function: Crawlers</li><li>EC2: Articles Download + Classification + Text Similarity + Email</li></ol><p>These are the main components of the application in more detail:</p><h3>Cron rule in Cloudwatch</h3><p>A cron rule in Cloudwatch is responsible for awakening the crawling process every day. The cron expression (30 07 ? * MON-FRI *) generates a call to the connected Lambda function every weekday at 7:30am GMT. There is some data attached within this call and it basically is a key-value pair that specifies the Lambda handler which is the first crawler to be launched. In this case, it is set to: { &quot;newspaper&quot; : &quot;cincodias&quot; } .</p><h3>Lambda — Crawlers</h3><p>The focus was placed on refactoring the original crawlers to be as lightweight as possible so that they can run on Lambda. Therefore, although they were originally written using scrapy, I ended up using requests and beautifulsoup for the same purpose.</p><p>A Lambda handler function is in charge of dispatching the different crawlers. This function receives the payload from the cron rule and tells the first crawler to run (same Lambda execution call). Once the first crawler has scraped all article urls from its newspaper, urls are saved into a specific S3 bucket directory and the Lambda execution is over. The bucket emits an event connected to the same Lambda every time a csv is stored there. This is how recurrence is introduced into this process: the Lambda dispatches a crawler, urls are stored in S3, S3 sends a callback to the Lambda specifying which urls were stored and the dispatcher launches the next crawler.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-MsWVUm_waL-kULyKrwoeg.png" /><figcaption>S3 events property (launches Lambda when a file ending in urls.csv is being put.</figcaption></figure><p>In order to get a full understanding on how to generate and ship the function and dependencies to populate the Lambda, I did write the python project and created a Makefile that would be called within a docker container to install dependencies, test and package the application+dependencies in the ideal environment to make them suitable for Lambda. If you want to make your life easier, you can go for tools like zappa to do all this under the hood. Again, all I wanted was to “get my hands dirty” and learn more stuff on the go.<br>You can see the details of the code <a href="https://github.com/adriamoya/bcpnews_lambda">here</a>.</p><p>When the last crawler has finished its work, an S3 event will re-execute the Lambda handler function and, this time, the Lambda will start the EC2 instance where all other processes will be executed.</p><p>The structure of the Lambda handler is written below:</p><pre>from __future__ import print_function<br>import boto3<br>import datetime<br>from crawlers.cincodias import parse_cincodias<br>from crawlers.elconfidencial import parse_elconfidencial<br>from crawlers.eleconomista import parse_eleconomista<br>from crawlers.expansion import parse_expansion</pre><pre>def lambda_handler(event, context):<br>   &quot;&quot;&quot;Lambda handler for the scraping / downloading process&quot;&quot;&quot;<br>   ec2 = boto3.resource(&#39;ec2&#39;, REGION)<br>   instance = ec2.Instance(INSTANCE_ID)<br>   crawl_date = datetime.datetime.now()</pre><pre>   try:<br>      trigger_file = event[&#39;Records&#39;][0][&#39;s3&#39;][&#39;object&#39;][&#39;key&#39;]<br>   except:<br>      trigger_file = &#39;&#39;<br>   <br>   if trigger_file:<br>      if &#39;cincodias&#39; in trigger_file:<br>         parse_elconfidencial(crawl_date, BUCKET_NAME)<br>      elif &#39;elconfidencial&#39; in trigger_file:<br>         parse_eleconomista(crawl_date, BUCKET_NAME)<br>      elif &#39;eleconomista&#39; in trigger_file:<br>         parse_expansion(crawl_date, BUCKET_NAME)<br>      elif &#39;expansion&#39; in trigger_file:<br>         response = instance.start()<br>   else:<br>      parse_cincodias(crawl_date, BUCKET_NAME)</pre><h3>EC2 — Articles download, classification, text similarity and email</h3><p>When the EC2 is started, a cronjob is called (@reboot) and it executes the second main Python project (it actually executes the following shell script).</p><pre>#!/bin/bash </pre><pre># Printing date<br>now=&quot;$(date)&quot;<br>echo &quot;--&gt; Starting $now&quot; </pre><pre># Running application<br>cd /home/ubuntu/ec2_model<br>stdbuf -o0 /home/ubuntu/ec2_model/env/bin/python /home/ubuntu/ec2_model/main.py &gt; log</pre><p>The main.py script controls the execution of each step before the newsletter is sent. These are the main steps:</p><ul><li><strong>Download all articles from newspapers</strong>: the crawlers have already parsed all interesting newspapers and stored the urls from all the articles for the last three days in S3 (this is necessary to capture relevant articles that might have been sent during the weekend). Here I used a really convenient library (newspaper3k) to standardize the content downloaded from these different sources. Feel free to check this in my <a href="https://medium.com/@adriamoyaortiz/classifying-news-articles-to-build-a-customized-newsletter-automatically-ede6ad9ef725">previous post</a>.</li><li><strong>Classify articles</strong>: use pre-trained tokenizers and classifiers (stored in S3) to predict the score of each article. There are only three features that are taken into account separately: title, summary and text of each article. Four different algorithms for each feature: CNN, LSTM, Bidirectional LSTM and CNN+LSTM. Cross-validation was also implemented.</li></ul><pre>for each feature in [&#39;summary&#39;, &#39;text&#39;, &#39;title&#39;]:<br>   for each model in [&#39;CNN&#39;, &#39;LSTM&#39;, &#39;BiLSTM&#39;, &#39;CNNLSTM&#39;]:<br>      for each fold in FOLDS:<br>         model_path = &#39;models/{model}{feature}{fold}.h5&#39;<br>         # download and load model from S3<br>         clf.predict()<br>      # averaging scores by fold</pre><pre># stacking<br># using xgboost on previous predictions to get final score</pre><ul><li><strong>Select candidate articles</strong>: those articles with &gt;0.5 score are candidates for the newsletter (some additional controlling could be implemented here).</li><li><strong>Save articles to RDS</strong>: save these articles with a date stamp on RDS and evaluate if there are duplicates that were already included in previous newsletter. If so, exclude from current selection of candidate articles.</li><li><strong>Text similarity to cluster articles</strong>: there are articles that have scored high and might be related to the same piece of news but from different newspapers. It would be ideal to identify these groups of articles that have a similar content so they are displayed together in the newsletter.<br>In order to enable this functionality, a text similarity process evaluates how similar all candidate articles are between each other. I used gensim library for this purpose. Implementing a tf-idf algorithm is pretty straightforward and results are really accurate. The result of this is a similarity matrix that is eventually used to cluster similar articles together based upon a similarity measure (100% would mean both articles are exactly identical). After some investigation, a 30% threshold yielded good results for the majority of the cases (a dynamic threshold or other algorithms could be also used here to refine this process).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jErU_C0JVPHFPXOYUiwjlQ.png" /><figcaption>This article in Cincodías can also be read in Expansion and El Confidencial.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9w9ZHc6XgxKPWkJ6v6zfOw.png" /><figcaption>Similar article in Expansion.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*voUYKNEFXGpYdPsyNoD9kw.png" /><figcaption>Similar article in El Confidencial.</figcaption></figure><ul><li><strong>Sending the email</strong>: this is the last step before the email is sent. Once the candidate articles are grouped together and sorted by score (high score articles will show up at the top of the newsletter), it is time to generate the newsletter. To do so I ended up using smtpib and email libraries.<br>The first library is used to create a client session object from which the email can be sent to another SMTP listener (in this case the target is a domain from gmail).<br>From email one can import the MIMEMultipart class (and others) to build the structure of the email.<br>The email body is created using HTML. The headers and footer are fairly static and the only part that needs to be modified is the body, where articles are rolled out. This part has been coded in Python but one could use Jinja, for example.<br>The list of recipients, in this particular case, is fixed and hardcoded but it could be dynamic and imported from another AWS service (e.g. a database), connected to a subscription page, etc.</li></ul><p>When the email is sent, the EC2 is stopped until next call. You can see the details of this part of the code <a href="https://github.com/adriamoya/bcpnews_ec2">here</a>.</p><p>This is how it looks like 🗞</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*AwGd5g1ySA9OrD0JJ_fFcA.png" /></figure><h3>Next</h3><p>There are lots of things that can be built on top of this application (e.g. by using the data store in RDS). One could think of building a web app were users could rate or like articles to keep training the models and recommending news, etc.</p><p>Additionally, there is a lot of potential stuff to be done in order to improve the current application. Better testing and monitoring, packaging and publishing the application, build a subscription service, transitioning to micro-services, etc.</p><p>I hope you liked it and thanks for reading 🙏</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cd6104fa7317" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build a financial newsletter with Deep Learning and AWS — Part 1]]></title>
            <link>https://medium.com/@adriamoyaortiz/classifying-news-articles-to-build-a-customized-newsletter-automatically-ede6ad9ef725?source=rss-320e6bfc736------2</link>
            <guid isPermaLink="false">https://medium.com/p/ede6ad9ef725</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[lstm]]></category>
            <category><![CDATA[news]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Adria M.]]></dc:creator>
            <pubDate>Sun, 13 May 2018 14:21:10 GMT</pubDate>
            <atom:updated>2019-01-20T22:09:54.242Z</atom:updated>
            <content:encoded><![CDATA[<p>Real case application for a financial consulting firm.</p><p>For over more than two years, an experienced employee from this firm collected the most relevant press articles from different newspapers each day. The goal was spreading a tailored, daily newsletter to all employees with interesting news about the financial and economic landscape (with both national and international scope). This process could take up to 2 hours every day and comprise task such as reading multiple newspaper front pages, keep track of cross-cutting topics from other sectors, structuring and ranking articles and manually writing and sending the newsletter.</p><p>We are lucky to have scraping and natural language processing (NLP) to do all this work for us now. We can very accurately reproduce her criteria to classify articles thanks to recent developments in deep learning and NLP.</p><p>This story gives an overview of the project, with focus in the (1) <strong>data gathering</strong> and (2) <strong>modelling </strong>phases. All code is written in Python3 and can be found <a href="https://github.com/adriamoya/bcpnews/">here</a>.</p><h3>1. Data Gathering</h3><p>This part, in general, is the most time-consuming task in every data project. In this case, however, the problem can be largely simplified by using a few python libraries.</p><p>The idea is to build a dataset with a large collection of articles where some of them were included in past newsletters (“ground truth”). These “included” articles should be extracted from the large list of newsletters sent over these years.</p><p>Before moving forward, let’s spend a second introducing the <a href="https://newspaper.readthedocs.io/en/latest/">newspaper3k</a> library. Newspaper is an amazing python library for extracting and curating articles. In a nutshell, to parse and download an article it is easy as passing the article url to the Article class from newspaper and use some of its methods.</p><blockquote>Newspaper is an amazing python library for extracting and curating articles</blockquote><p>With this, the workload reduces significantly and guarantees that all articles downloaded will follow the same structure, regardless of their source (text, title, authors, summary, …).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QoGrqPhuaInJ3WLOor3DYA.png" /><figcaption><strong>Fig 1</strong>. Data Gathering process.</figcaption></figure><p>We should now focus on how to access all previous newsletters so that we can search the article urls that were sent (that is all we need for the “ground truth”). We will use <a href="https://docs.python.org/3.6/library/imaplib.html">imaplib</a> to connect to an IMAP server (in this case, connecting over an SSL encrypted socket) and <a href="https://docs.python.org/3.5/library/email-examples.html">email</a> to parse the content of the emails.</p><pre># Variables<br>M = imaplib.IMAP4_SSL(&#39;imap.gmail.com&#39;)</pre><pre># Retrieve credentials<br>user = input(&#39;User email: &#39;)<br>pass = getpass.getpass()</pre><pre># Login<br>M.login(user, pass)<br>M.select()</pre><pre># Search all emails<br>typ, data = M.search(None, &#39;(FROM &quot;name@domain.com&quot;)&#39;)</pre><p>Once we have processed the data (emails), we can easily extract the article urls using regex (you can see all the details in <a href="https://github.com/adriamoya/bcpnews/tree/master/1_construction/1_mail_download">here</a>).</p><p>Now let’s focus on filling the dataset with articles not included in past newsletters (“negative examples”). To do this we have used <a href="https://doc.scrapy.org/en/latest/">Scrapy</a> and have crawled 3 newspapers (the most frequent ones in the previous newsletters): Expansión, Cincodías and El Confidencial.</p><p>Every time you face a scraping problem, it is essential to spend time browsing the website in order to find the right page to start crawling. In this case, since we are interested in downloading articles (links) published over the last two years, it would be best to find some sort of newspaper library or repository where you can filter articles by their “publication date”. Once we have spotted that starting url, we just need to figure out how to loop over different dates and retrieve all the news for each date.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/83cfc8012e4243f16eeed89e2b9179c5/href">https://medium.com/media/83cfc8012e4243f16eeed89e2b9179c5/href</a></iframe><p>Considerations: before jumping to building your spiders, play around with the <a href="https://doc.scrapy.org/en/latest/topics/shell.html">Scrapy shell</a> to understand what kind of url response you get back. Some websites are async rendered and you might need JavaScript integration.</p><p>In any case, you can find all the spiders used in this project <a href="https://github.com/adriamoya/bcpnews/tree/master/1_construction/2_scrapy_projects">here</a> and you can also follow this <a href="https://realpython.com/web-scraping-with-scrapy-and-mongodb/">tutorial</a> if this is new to you.</p><p>At this point we have successfully <strong>collected news article links</strong> from:</p><ul><li><strong>Emails </strong>(“ground truth”): using imaplib + email</li><li><strong>Online newspapers</strong> (“negative examples”): using Scrapy</li></ul><p>Now we can import <a href="https://newspaper.readthedocs.io/en/latest/">newspaper</a> and <strong>download the content of each article</strong>. For this we have adapted the Article class to suit our needs.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/51ff6e951bfcea4a68241422cb3c5f4c/href">https://medium.com/media/51ff6e951bfcea4a68241422cb3c5f4c/href</a></iframe><p>We can now use this new class to parse, download and dump all the content of each article to a local file. <em>Note that we are storing everything in local JSON files because it can be comfortably handled this way.</em></p><p>These are the data fields and how an arbitrary article looks like:</p><ul><li><strong>authors</strong>: author(s) of the article (array)</li><li><strong>date</strong>: date of article publication (datetime)</li><li><strong>day_of_week</strong>: day of the week of article publication (string)</li><li><strong>domain</strong>: newspaper website domain</li><li><strong>flag</strong>: target variable (0: not included in BBB newsletter, 1: included)</li><li><strong>keywords</strong>: list of keywords suggested by nlp application (array)</li><li><strong>section</strong>: newspaper section</li><li><strong>summary</strong>: brief summary of the article</li><li><strong>text</strong>: body text of the article</li><li><strong>title</strong>: title of the article</li><li><strong>url</strong>: url link to the article</li></ul><pre>{<br>  &quot;authors&quot;:[&quot;Eduardo Segovia&quot;,&quot;José Antonio Navas&quot;,&quot;E. Sanz&quot;,&quot;M. Valero&quot;,&quot;Contacta Al Autor&quot;],<br>  &quot;date&quot;:1478304000000,<br>  &quot;day_of_week&quot;:&quot;Saturday&quot;,<br>  &quot;domain&quot;:&quot;elconfidencial.com&quot;,<br>  &quot;flag&quot;:1,<br>  &quot;keywords&quot;:[&quot;todos&quot;,&quot;inmuebles&quot;,&quot;para&quot;,&quot;retasar&quot;,&quot;pisos&quot;,&quot;sus&quot;,&quot;en&quot;,&quot;y&quot;,&quot;banco&quot;,&quot;del&quot;,&quot;el&quot;,&quot;que&quot;,&quot;los&quot;,&quot;vivienda&quot;,&quot;se&quot;,&quot;las&quot;,&quot;rebajas&quot;,&quot;la&quot;,&quot;noticias&quot;],<br>  &quot;section&quot;:&quot;vivienda&quot;,<br>  &quot;summary&quot;:&quot;Malos tiempos para encontrar un chollo entre los pisos que tienen a la venta los bancos.\nEsta norma obliga a volver a tasar (y provisionar) todos los inmuebles que tenga el banco en balance con el descuento aplicado a las ventas, por lo que cualquier alegría en los precios puede tener un enorme impacto en sus cuentas.\nLas provisiones es dinero que los bancos apartan para cubrir el posible impago de los créditos o pérdida de valor de inmuebles, acciones o bonos.\nNo se trata de una norma estándar para todo el sector, ya que en esto se aplican los llamados \&quot;modelos internos\&quot;, que son diferentes para cada entidad.\nEn todo caso, con esta norma se refuerza la idea de que cuanto mejor estén provisionados los inmuebles, más fácil será venderlos.&quot;,<br>  &quot;text&quot;:&quot;Malos tiempos para encontrar un chollo entre los pisos que tienen a la venta los bancos. El interés de algunos por librarse de las viviendas adjudicadas les ha llevado a ofrecer rebajas muy interesantes en el pasado, pero esta práctica se ha acabado con la nueva circular contable del Banco de España que acaba de entrar en vigor. Esta norma obliga a volver a tasar (y provisionar) todos los inmuebles que tenga el banco en balance con el descuento aplicado a las ventas, por lo que cualquier alegría en los precios puede tener un enorme impacto en sus cuentas.\n\nHasta ahora, cuando un banco vendía un piso por debajo de su valor de tasación, tenía la obligación de provisionar la diferencia con el precio al que lo tuviera valorado en su balance (valor en libros) y apuntarse la pérdida correspondiente. Pero solo para ese inmueble individual. Las provisiones es dinero que los bancos apartan para cubrir el posible impago de los créditos o pérdida de valor de inmuebles, acciones o bonos. Este dinero resta del beneficio de la entidad, o lo que es lo mismo, supone una pérdida.\n\nEl gobernador del Banco de España, Luis Linde (EFE)\n\nLo que cambia la nueva circular es que, a partir de ahora, esa pérdida no solo se referirá a cada piso individual, sino que esa rebaja tendrá que aplicarse a todos los inmuebles similares que tenga el banco en su balance. Y, en consecuencia, deberá provisionarse la pérdida de valor de todos ellos, y dado que la banca todavía tiene activos adjudicados por valor de 81.500 millones de euros (solo se han reducido un 4% desde 2011), estamos hablando de un impacto potencial muy importante en sus resultados.\n\nNo se trata de una norma estándar para todo el sector, ya que en esto se aplican los llamados \&quot;modelos internos\&quot;, que son diferentes para cada entidad. Y algunos toman como referencia las ventas realizadas en los últimos tres meses, otros las de los últimos seis... Pero lo que es obligatorio es que estos modelos incluyan la referencia de las operaciones realizadas en los meses anteriores, según fuentes del sector. La justificación de esta medida es que los inmuebles deben estar valorado a precios de mercado y que ese precio debe ser similar para todos los de la misma entidad; por tanto, si vende algunos con un descuento, ese descuento debe aplicarse a todos.\n\nEl BdE critica que los bancos no vendan pisos\n\nResulta un tanto contradictorio que el mismo Banco de España que ha establecido esta norma -que entró en vigor en octubre y se aplicará ya a los resultados de cierre del año- critique en su último Informe de Estabilidad Financiera que los bancos no den salida a sus adjudicados con mayor celeridad: \&quot;En el último año, este importe de activos improductivos [incluye los préstamos morosos] se ha reducido en un 12 %, si bien aún representa un porcentaje significativo del activo total de los bancos en su negocio en España y constituye un elemento de presión negativo sobre la cuenta de resultados y la rentabilidad de las entidades\&quot;.\n\nSede del Banco de España, en la Plaza de Cibeles en Madrid (EFE)\n\nComo adelantó El Confidencial, la nueva norma también obliga a las entidades a volver a tasar sus inmuebles adjudicados todos los años, en vez de cada tres ejercicios como sucedía hasta ahora. Esto también pretende que los bancos tengan su ladrillo valorado a niveles realistas y, de nuevo, que doten las provisiones necesarias para cubrir la diferencia con el valor al que se lo adjudicaron. En la presentación de los resultados del tercer trimestre, la mayoría de los grandes bancos explicaron que la nueva circular significará un trasvase de provisiones desde el crédito moroso a los adjudicados, pero que el efecto neto será mínimo.\n\nEn todo caso, con esta norma se refuerza la idea de que cuanto mejor estén provisionados los inmuebles, más fácil será venderlos. Esa es una de las razones que explican las dudas del mercado sobre la capacidad del Banco Popular de deshacerse de 15.000 millones de adjudicados en dos años, ya que su tasa de cobertura con provisiones se sitúa en el 36%, frente a una media del 50% en el sector.&quot;,<br>  &quot;title&quot;:&quot;Rebajas: Los bancos acaban con los chollos de pisos para no retasar todos sus inmuebles. Noticias de Vivienda&quot;,<br>  &quot;url&quot;:&quot;http:\/\/www.elconfidencial.com\/vivienda\/2016-11-05\/bancos-rebajar-pisos-retasar-inmuebles-adjudicados-circular-contable_1285149\/&quot;<br>}</pre><p>Next steps before jumping to modelling are:</p><ul><li><strong>EDA and data cleaning</strong>: there might be articles that can be discarded since their content has not been properly downloaded. Other articles have been published in different languages or in newspapers that will not be considered in production. What actions should be considered?</li><li><strong>Train / test split</strong>: the time component in the modelling part is important. Our goal is to efficiently classify future articles in production, and hence it makes more sense to assign to the test dataset the most recent articles, and train the model with older ones. This approach is arguable.</li><li>…</li></ul><blockquote>Thousands of online articles have been downloaded over the last two years and those included in previous newsletters have been labeled</blockquote><h3>2. Modelling</h3><p>This is where the real fun begins. The basic idea here is to build a classifier that correctly discriminates between interesting and non-relevant news.</p><p>I have hosted a <a href="https://www.kaggle.com/c/bcpnews/">Kaggle</a> competition to engage other colleagues to take part of the project in the modelling phase. This is the one reason why the train / test split was done before.</p><p>Since our interest is training a model that should generalize correctly in production and the problem consists on obtaining a single prediction for a given article (include / not include), the most straightforward and reliable architecture is a <strong>neural network-based text classifier</strong> (only use text and title here).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oDNUZLTLRtm-K5bQBA0XOQ.png" /><figcaption><strong>Fig 2</strong>. Modelling process.</figcaption></figure><p>Before defining the layout of our classifier, it is good practice to do some <strong>text pre-processing</strong>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fb9f23bbddef2ea295db264edd93e9bb/href">https://medium.com/media/fb9f23bbddef2ea295db264edd93e9bb/href</a></iframe><h4>Words to Numbers</h4><p>In this part we need to build a <strong>dictionary</strong> that will be used to convert words to numbers. Our dictionary will contain all words that have appeared at least 5 times (in all articles). Once we have the unique list of such words, we proceed to assign to each word a number. The first and most frequent word (=1) is de, the second one (=2) is la, …, the #200 is situación, etc. Now we can convert sentences with words to lists with integers.</p><pre>[&#39;los&#39;,<br> &#39;nueve&#39;,<br> &#39;consejeros&#39;,<br> &#39;del&#39;,<br> &#39;banco&#39;,<br> &#39;de&#39;,<br> &#39;inglaterra&#39;,<br> &#39;han&#39;,<br> &#39;acordado&#39;,<br> &#39;mantener&#39;,<br> &#39;los&#39;,<br> &#39;tipos&#39;,<br> &#39;de&#39;,<br> &#39;interés&#39;,<br> &#39;en&#39;,<br> &#39;el&#39;,<br> &#39;025&#39;,<br> &#39;.&#39;]</pre><pre>[   12,<br>    17,<br>  4265,<br>   202,<br>     0,<br>     1,<br> 31026,<br>   207,<br>  1195,<br>     9,<br>    70,<br>   332,<br>    11,<br>  1017,<br> 31027]</pre><p>You cannot pass differing lengths of text sequences to the algorithm. Hence we will pre-pad the sentence with &lt;PAD&gt; (new “word” in our dictionary). In other words, truncate articles’ text to a determinate number of words (integers). The threshold of our choice is 500 words for text (lower threshold if we were to consider title). Sequences less than 500 in length will be pre-padded and sequences that are longer than 500 will be truncated.</p><h3>Neural network-based text classifiers</h3><p>Inputs are ready to train the model’s parameters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/1*1fC-cY1Hyr9XcGUquyaDKQ.png" /></figure><p>The basic structure used is:</p><ul><li>Input (pre-padded text sequences)</li><li>Embeddings</li><li>Convolutional layers (CNN)</li><li>LSTM (RNN)</li><li>Fully connected part</li></ul><p>There are countless deep learning frameworks available today but for this case study we are using <a href="https://keras.io/">Keras</a> running on top of <a href="https://github.com/tensorflow/tensorflow">TensorFlow</a>.</p><h4>Embeddings</h4><p>Embedding layers take a sequence of word ids as an input and produce a sequence of corresponding vectors as an output. Their functionality is really straightforward, and since the actual semantics of those vectors are not interesting for our problem, the only remaining question is “What is the best way to initialize the weights?” Depending on the problem, the answers may be a “generate your own synthetic labels, train word2vec on them, and init the embedding layer with them.”</p><p>In <strong>Fig 3</strong> we have used Continuous Bag of Words (CBoW) with TensorFlow to build an embeddings matrix and used t-SNE technique for dimensionality reduction to plot a few words in 2D. It can be clearly seen how similar words cluster together (numbers, years, banks, etc.).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/1*04WcMpeGd_j56Ifrpb9Ebw.png" /><figcaption><strong>Fig 3.</strong> Using Continuous Bag of Words (CBoW) to build embeddings matrix in all available articles.</figcaption></figure><p>The go-to solution here, however, is to <strong>use a transitional layer for embeddings</strong>. LSTMs learn to distinguish important and unimportant parts of the sequence by themselves, but we can’t be sure that the representation from the embedding layer is the best input, especially if we don’t finetune the embeddings.</p><p>In other words, instead of building the embeddings matrix with word2vec and transferring the fixed weights to the embedding layer, we let the neural network train the embeddings weights.</p><p>“Adding a layer that is applied to each word embedding independently can improve your results, acting as a simple attention layer.”</p><p>Before feeding the sequence model with data, we need to truncate and pad the input sequences so that they are all the same length for modelling. The model will learn the zero values carry no information so indeed the sequences are not the same length in terms of content, but same length vectors are required to perform the computation in Keras.</p><h4>Convolutional</h4><p><strong>Convolutional Neural Networks</strong> excel at learning the spatial structure in input data. We can easily add a one-dimensional CNN and max pooling layers after the Embedding layer which then feed the consolidated features to the LSTM. <a href="https://github.com/Shawn1993/cnn-text-classification-pytorch">Here</a> you can find a simple implementation of Convolutional Neural Network for sentence classification.</p><h4>LSTM</h4><p><strong>Recurrent Neural Networks</strong> like <a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/">LSTM</a> generally have the problem of overfitting and this is why these layers are normally combined with dropout.</p><p>Since the main work is being done in the recurrent layer, it’s important to make sure that it captures only the relevant information. It’s a frequent challenge for natural language applications and an open scientific problem.</p><h4>Dense classifier</h4><p>A<strong> fully-connected part</strong> performs a series of transformations on the deep representation and finally outputs the scores for each class. The best practice here is to apply the transformations as follows:</p><ol><li>Fully-connected layer</li><li>Batch normalization</li><li>(Optional) Non-linear transformation (hyperbolic tangent or ELU)</li><li>Dropout</li></ol><p>To add more predictive power we have decided to <strong>stack</strong> two neural network-based text classifiers (for text and title) and use their outputs to feed a <a href="http://xgboost.readthedocs.io/en/latest/model.html">XGBoost</a> classifier (Extreme Gradient Boosting) in top of them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gNPKdggnK3WeXD4HSF3QvQ.png" /><figcaption><strong>Fig 4. </strong>Model layout.</figcaption></figure><h4>Evaluation</h4><p>Receiver Operating Characteristic (ROC) curves are typically used in binary classification to study the output of a classifier. The AUC is the Area Under the Curve and is used here to evaluate the performance of the model. The AUC has several equivalent interpretations:</p><ul><li>The expectation that a uniformly drawn random positive is ranked before a uniformly drawn random negative.</li><li>The expected proportion of positives ranked before a uniformly drawn random negative.</li><li>The expected true positive rate if the ranking is split just before a uniformly drawn random negative.</li><li>The expected proportion of negatives ranked after a uniformly drawn random positive.</li><li>The expected false positive rate if the ranking is split just after a uniformly drawn random positive.</li></ul><p>After this simple but powerful approach, we have reached an AUC larger than <strong>0.95</strong> in the test set !!!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hzr4YYj2eJD22x2CjBG80g.jpeg" /></figure><p>Some simple HTML and this is how the newsletter looks like …</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pAExi87pF6gsC2qtzMG4uA.png" /></figure><p>There will be a <a href="https://medium.com/@adriamoyaortiz/classifying-news-to-build-a-newsletter-with-deep-learning-part-2-cd6104fa7317">second part</a> were I explain how I deployed the full application on AWS.</p><p>I hope you liked it and thanks for reading 🙏</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ede6ad9ef725" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>