<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Eduardo Perez Denadai on Medium]]></title>
        <description><![CDATA[Stories by Eduardo Perez Denadai on Medium]]></description>
        <link>https://medium.com/@eduardoperezdenadai?source=rss-65aad403c35d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*V5ta9qkpDAd64gPH.</url>
            <title>Stories by Eduardo Perez Denadai on Medium</title>
            <link>https://medium.com/@eduardoperezdenadai?source=rss-65aad403c35d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 19:19:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@eduardoperezdenadai/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Model Risk Management and Risk ops]]></title>
            <link>https://medium.com/@eduardoperezdenadai/model-risk-management-and-risk-ops-cac23d4c9488?source=rss-65aad403c35d------2</link>
            <guid isPermaLink="false">https://medium.com/p/cac23d4c9488</guid>
            <category><![CDATA[financial-services]]></category>
            <category><![CDATA[risk-management]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[model-risk-management]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Eduardo Perez Denadai]]></dc:creator>
            <pubDate>Thu, 23 Mar 2023 00:49:28 GMT</pubDate>
            <atom:updated>2023-03-23T14:16:37.406Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2FAx3x5d6CsJ59aR4fUiLQ.png" /></figure><p>Machine learning models are increasingly becoming part of critical business processes and are no longer used as a novelty or used as a way to jump into the hype bandwagon of data science and artificial intelligence. Modern ML is indeed causing an impact in processes that, if done wrong, involve adverse effects on downstream processes.</p><p>This article is a short reference from knowledge acquired from the study and practice of helping clients in regulated environments (special mention to financial services) that deal with complex controls and auditing of analytical models through model risk management (MRM) framework. let’s call the operation of MRM: <strong>Risk Ops</strong>.</p><p>I hope it will be helpful if you are part of company, or using ML models in production and have recognize the need systematically tackle model risk in production but do not want to compromise value at the expense of needless bureaucracy and documentation that usually comes by implementing traditional risk management frameworks.</p><p><strong>tldr; </strong>Many guidelines in the financial industry treat model risk as any other operational risk. There are already many frameworks for operational risk management. This article focuses on the operation of model risk frameworks at scale through meta systems on top of ML Ops layer as a tool to control the adverse consequences of misuse of models in production. This can be helpful to other unregulated industries as an endeavor is too expensive to pursue.</p><h4>Why mange model risk?</h4><p>In the 2008 Great Financial Crisis, banks lost billions because of their use of simple copula models in estimating the risk of mortgage portfolios. Also, in 2012, JP Morgan’s loss of over $6 billion due to an error in a credit model resulting from a lack of governance; this incident is the so-called London Whale Trading Incident.</p><p>Another cautionary tell of using models to drive investment decisions comes from the 500MM USD + losses incurred by real state company Zillow in which overconfidence in the machine learning models used to predict property prices exposed the company to a large size of over valued assets much above real prices. Their models were not able to capture the complexities that drove the prices down. In retrospect, had Zillow had a model risk framework set in place, then such large exposure would have been much less.</p><p>Recent advances in large Pretrained language models (LLMs) or Foundational Models, e.g., GPT style was able to surpass humans in several benchmarks such as text summarization, machine translation, story-telling, general knowledge question and answers, conversational chatbots, etc. Yet, many studies have brought to light the biases and limitations present in the results, and their applications in real world environments have been limited to non-factual settings in which results must not be interpreted as a source of veracity. Early application of this technology in industry have shown that they require safety features to mitigate the risk of being misuses and even cause reputational, legal, and financial losses. That’s why commercial LLM services explicitly state not to use them for legal, factual, financial advice, and safety critical environments.</p><p>Therefore, whenever models are used in a critical environment where adverse consequences exist, e.g., financial or life loss, it is likely there are regulations and governance in place to mitigate it. In these settings, like insurance or banking, models do drive decision-making directly and are not used as aid. These models require a higher level of scrutiny because of the adverse impact of misuse.</p><h4>Traditional MRM</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fn3_JHpHgAzeAAb8RlbJ2Q.png" /><figcaption>Value should drive the purpose of a model</figcaption></figure><blockquote>“An Analyst’s goal should be the development of the most accurate and reliable predictive model or the best model on which to base estimation or hypothesis testing”</blockquote><blockquote>Franl. E. Harrel, Jr.</blockquote><p>Models have been used in business critical application in the financial investment industry for a long time, e.g., pricing assets and predicting signals using mathematical models and statistical calibration of such models. Traditionally, inferential analysis dominated the industry. This type of analysis was focused on deriving properties about a population through hypothesis testing and deriving estimates.</p><p>Nowadays, predictive analysis is the dominant type of type of statistical analysis pursued by business. In this type of analysis, the main goal is to use historical data to build models that are able to predict future behavior. The newer generation of statistical/machine learning techniques uses a large amount of diverse data to drive predictive performance. This new modeling paradigm places a lot of emphasis on the variety, relevance, and quality of data that were not present before.</p><p>There is a lot of hype about machine learning, a subset of artificial intelligence. But, there is nothing inherently intelligent in those algorithm, as they require a lot of modeler bias to adapt general purpose algorithms, e.g., GLM, Decision Trees, SVM, Deep Learning, etc., to produce reasonable results in the real world. If a modeler wants properties such as generalizability, interpretability, explainability, stability, robustness, etc., then those are built by the modeler. ML models do not elicit business requirements and distill specifications or build themselves to those specifications. At least, not yet.</p><p>So, the responsibility of building a good model for a specific application relies on the ability of the modeler to understand the problem, to talk to the right people, choose the right data, apply the right model, build the right test set, and for the right use. Placing such trust on a single individual role may be risky, there is such sting as human error and biases, and such risks permeate into the model results. So, industries like the financial industry have separate roles for model building and model validation.</p><p>There are many ways we could perform validation, but traditional industries like financial industries does it through the application of the principle of “effective challenge” of models.</p><blockquote>“A guiding principle for managing model risk is “effective challenge” of models, that is, critical analysis by objective, informed parties who can identify model limitations and assumptions and produce appropriate changes.”</blockquote><blockquote>SR Letter 11–7, Board of Governors of the Federal Reserve System Office of the Comptroller of the Currency, published in 2011</blockquote><p>Yet, it transfers a lot of the risk to another human role and makes it difficult to scale. This guidelines is already 12 years old, before the advent of big data systems and ML algorithms at scale. The “effective challenge” depends on a combination of incentives, competence, and influence. This is difficult to scale when the number of models under scrutiny increases as the cost of paired validation scales exponentially. Also, when these conditions are not met, the principle of effective challenge breaks down.</p><p>Often, in practice, model validators become gate keepers, and eventually, the validation process falls prey to Goodhart’s law.</p><blockquote>“When a measure becomes a target, it ceases to be a good measure”</blockquote><blockquote>Charles Goodhart, Economist.</blockquote><p>Or more relevant when use in the context of risk modeling:</p><blockquote><em>A risk model breaks down when used for regulatory purposes.</em></blockquote><blockquote><em>— Jon Danielsson</em></blockquote><p>Finally, traditional MRM is highly bureaucratic and focuses on thorough documentation for control and auditing layer. So, how does traditional MRM can be integrated into today’s machine learning pipelines or foster responsible agile data-driven product developments.</p><h4>Modern MRM — Risk Ops</h4><p>In this day and age, traditional MRM frameworks used in financial institutions most be updated to deal with the ever increasing amount of models being built and integrated into decision making pipelines and the vast amount of data of varied quality.</p><p>Current technology and hyper-contextualization of products and services have changed the pace in which decision-making needs to happen to stay competitive. Long gone are the days when you would sit down in an in-person committee to approve a decision. Decisions that took weeks, now needs to take minutes or even seconds.</p><p>Regulated industries struggle to balance between fast time-to-market and controlling for adverse consequences. This means using traditional MRM in an environment that demands speed. For instance, most banks were too slow to react to the pandemic, and the consequences were already felt when banks took measures with their clients.</p><p>A successful model risk framework should, at a minimum, serve two purposes:</p><ol><li>Enable modelers to produce high-quality models and</li><li>provide the means to identify, assess, control, and measure continuously the risk of a model failing in operation during its whole lifecycle.</li></ol><p>I find it useful to explain Risk Ops in the following way. A successful Risk Ops, is like the shock absorber in an SUV car. It is there to allow a good smoothing experience of an otherwise bumpy ride in a dirty road. The problem is that companies used to drive at 80 km/hr, but nowadays, they need to drive at 180 km/hr. You know the dynamics of the car are different under those conditions.</p><p>Therefore, a Risk Ops framework should dampen the impact of adverse consequences in a company when decisions need to happen fast and market changes fast. Accomplishing these goals should not slow down development and neither choke the ability to extract value from data. Risk Ops should be lean and have the same drive to be as agile as the rest of the business requires.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zpe6YP-4M-OEFFOBusFzMg.png" /><figcaption>Any resemblance is just a coincidence.</figcaption></figure><h4>So, where do you start?</h4><p>You start from the beginning, with Metadata. Metadata exchange has its place at the core of distributed computing on commodity hardware and big data systems.</p><p>Metadata, i.e., data of data, allows building systems on top of systems (meta systems) by exchanging smaller amount (smaller than the underlying data it is describing) of domain-specific information about these system and leaving the underlying system undisturbed as is.</p><p>A modern ML system operates with three operational domains:</p><ol><li>Data Ops: Deals with continuous data protection, security, ethics, quality, collection, and feature stores.</li><li>ML Ops: Deals with data exploration, model training, optimization, deployment, and performance monitoring.</li><li>Risk Ops: Deals with alerting, controls, interventions, and auditing.</li></ol><p>These operational domains are built on top of each other with different stakeholders at each layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*PenOo3h6e7t3LqJobJX2Fw.png" /><figcaption>Operating risk at all levels of the data value chain using a layered approach.</figcaption></figure><p>With metadata, we do not need to build a tightly coupled Risk Ops system. Rather, by exchanging Data-metadata, ML-metadata, and Risk-Metadata with a centralized metadata store, it is easier to build consumer systems for alerting, control, and auditing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wMFfuMmqVPkTHL5dlzk7Pg.png" /><figcaption>Risk Ops Architecture Example.</figcaption></figure><h3>Conclusion</h3><p>Integrating Risk Ops systems with ML Ops and Data Ops through the exchange of metadata makes the task of controlling risk (reputitional, financial, operational, etc) easier and interoperable with already existing ML Ops solutions. It also benefits non-regulated industries to improve their existing ML Ops systems by building a layer that allows for better control of the risk of misuse of their algorithmic assets and help the operation of responsible and ethical systems.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cac23d4c9488" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Interpretability of Deep Learning Models]]></title>
            <link>https://medium.com/data-science/interpretability-of-deep-learning-models-9f52e54d72ab?source=rss-65aad403c35d------2</link>
            <guid isPermaLink="false">https://medium.com/p/9f52e54d72ab</guid>
            <category><![CDATA[research]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Eduardo Perez Denadai]]></dc:creator>
            <pubDate>Sat, 17 Nov 2018 21:12:40 GMT</pubDate>
            <atom:updated>2018-11-18T15:41:51.995Z</atom:updated>
            <content:encoded><![CDATA[<p>Model Interpretability of Deep Neural Networks (DNN) has always been a limiting factor for use cases requiring explanations of the features involved in modelling and such is the case for many industries such as Financial Services. Financial institution whether by regulation or by choice prefer structural models that are easy to interpret by humans that’s why deep learning models within these industries have had slow adoptions. An example of a critical use case would be risk models where usually banks prefer classic statistical methods such as Generalized Linear Models, Bayesian Models and traditional machine learning models such as Tree-based models that are easily explainable and interpret in terms of human intuition.</p><p>Interpretability since the beginning has been an important area of research since Deep Learning models can achieve high accuracy but at the expense of high abstraction (i.e. <em>accuracy vs interpretability problem</em>). This is important also because of <em>Trust</em> since a model that is not trusted is a model that will not be used (i.e. try selling to upper management a black box model).</p><p>To appreciate this issue just imagine a simple DNN of many layers that is trying to predict an item price using many feature variables and ask yourself:</p><ul><li>what does the weights of each connection mean in terms of interpreting the result?</li><li>Which is the set of weights that play the most important role in the final prediction?</li><li>Does knowing the magnitude of the weights tells me anything about the importance of the input variables?</li></ul><p>First, the weights in a neural network are a measure of how strong each connection is between each neuron. So, looking at the first dense layer in your DNN you can tell how strong the connections are between the first layer’s neurons and the inputs. Second, after the first layer you lose the <em>one-to-many</em> relationship and it turns into a monstrous <em>many-to-many </em>entangled mess. This means that a neuron in one layer may be related to some other neuron far away (i.e. neurons experience non-locality effects due to back propagation). Lastly, the weights sure tell the story about the input but the information they have is compressed in the neurons after the application of the activation functions (i.e. non-linearities) making it very difficult to decode. No wonder why Deep Learning models are called black-boxes.</p><p>Methods for explaining neural networks generally falls within two broad categories (1) Saliency Methods, and (2) Feature Attribution (FA)</p><p>Saliency methods are good at visualizing what is going on inside the network and answering questions such as (1) which weights are being activated given some inputs? or (2) what regions of an image is being detected by a particular convolutional layer? . In our case, this is not what we are after, since it really does not tells us anything about which feature is “best” at describing our final predictions.</p><p>FA methods are a type of methods that attempts to fit structural models on a subset of data in such a way as to find out the explanatory power each variable has in the output variable. S. Lundberg, S. Lee, Showed in a NIPS 2017 paper (<a href="http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf">see paper</a>) that all models Such as <em>LIME, DeepLIFT, and Layer-Wise Relevance Propagation</em> are part of a larger family of methods named Additive Feature Attribution (AFA) methods with the following definition:</p><p><strong>Additive Feature Attribution methods</strong> <em>have an explanation model that is linear function of binary variables:</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/211/1*986PLqRsnh5KNj6NmOdn7A.png" /></figure><p><em>where z’ is a subset of {0,1}^M, where M is the number of simplified features, and Phi_i are weights (i.e. contributions, or the effects).</em></p><p><em>LIME, DeepLIFT, and Layer-Wise Relevance Propagation </em>all attempts to minimize an objective function to approximate g(z’). So they are all part of the AFA family of methods.</p><p>In the same paper, the authors proposed a method for estimating the contributions (<em>Phi_i</em>) of this linear model based on game theory named Shapley values. Why introducing game theory into this picture?</p><p>There is a branch of game theory that studies collaborative games in which the goal is to predict the fairest distribution of wealth (i.e. payout) among all players that work together to achieve a common outcome. This happens to be exactly what the authors propose: <strong>To use Shapley values as a measure of feature variable contribution towards the prediction of the output of the model.</strong></p><p>The authors showed that Shapley values are the optimal solution to a particular type of collaborative games for which AFA methods are also part of (<a href="http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf">refer to the paper</a>). Lastly, they proposed <strong><em>SHAP (Shapley Additive Explanation)</em></strong> values as a unified measure of feature importance and some methods to efficiently generate them such as:</p><ul><li>Kernel SHAP (Linear LIME + Shapley values)</li><li>Deep SHAP (DeepLift + Shapley values)</li><li>Tree SHAP (Tree explainer + Shapley values)</li></ul><p>In this article, I’m going to show you how to use DeepSHAP applied to neural networks but before we start you will have to install the following python library:</p><ul><li>shap: <a href="https://github.com/slundberg/shap">https://github.com/slundberg/shap</a> (&gt; pip install shap)</li><li>seaborn</li><li>pandas</li><li>keras + tensorflow</li><li>Jupyter notebook is required (since shap uses some useful javascript plots)</li></ul><h3>DeepSHAP Example:</h3><p>We are going to use the diamonds dataset that comes with seaborn. It is a regression problem for which we are trying to estimate the price of a diamond given diamond quality measures.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7059fe57ac440c548008840bd989c555/href">https://medium.com/media/7059fe57ac440c548008840bd989c555/href</a></iframe><p>Our data looks like</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/469/1*rZUcAf0gNRmxozaRf7W8dw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/715/1*4nGDUAuBUjToH9J2DKgxNg.png" /></figure><p>Let’s get our target and feature variables and encode the categorical variables:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/24c65d7f6be3b74d958328a5092e0e48/href">https://medium.com/media/24c65d7f6be3b74d958328a5092e0e48/href</a></iframe><p>Which gives us the mappings for later use</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*53jrL4Oy3viOAsFOpsGkGQ.png" /></figure><p>Now let’s get our train and test setup:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1956786e0244601a56ab9c62a3556fe3/href">https://medium.com/media/1956786e0244601a56ab9c62a3556fe3/href</a></iframe><p>Now let’s set up a neural network model using a simple feed forward fully connected network (with <a href="https://en.wikipedia.org/wiki/Early_stopping">early stopping</a>)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c19fa320222714a2566846e942039e15/href">https://medium.com/media/c19fa320222714a2566846e942039e15/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/531/1*PN6Eq4yiXlzyubt6Y6w7TA.png" /></figure><p>My setup has an NVDIA GPU P4000, so it runs pretty quick, if you don’t have a GPU just increase your batch size based on your memory limitations. So, after 15 sec we get:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*-Wc5p3nrSw5GeGUumc-lXQ.png" /><figcaption>The final mape loss is around 15% which is good enough for our purposes. The bumps are probably the large batch sizes.</figcaption></figure><h4>Now DeepSHAP:</h4><p>Using DeepSHAP is pretty easy thanks to the shap python library. Just set it up as follows :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e2e4e28198d952f2ebb234520cead8cb/href">https://medium.com/media/e2e4e28198d952f2ebb234520cead8cb/href</a></iframe><p>The method <em>shap_values(X) </em>performs a fitting of the SHAP algorithm based on DeepLIFT using shapley sampling. This takes longer the more data you choose so beware.</p><p>Now, let’s see some individual attributions inferred by the model:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ce3119b0a471e0fc42517890f0c114b9/href">https://medium.com/media/ce3119b0a471e0fc42517890f0c114b9/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KubSdxduPSWE8Z_aOsLBvg.png" /><figcaption>Figure 1. Force plot for one record (i.e. item) in the dataset</figcaption></figure><p>To understand these force plots you need to go back to the author’s paper:</p><blockquote>…SHAP (SHapley Additive exPlanation) values attribute to each feature the change in the expected model prediction when conditioning on that feature. They explain how to get from the base value E[f(z)] that would be predicted if we did not know any features to the current output f(x). This diagram shows a single ordering. When the model is non-linear or the input features are not independent, however, the order in which features are added to the expectation matters, and the SHAP values arise from averaging the φi values across all possible orderings.</blockquote><p>Notice there is this <strong><em>base value</em></strong> which is the expected value (i.e. E[f(z)]) calculated by DeepSHAP which is just the value that would be predicted if you did not know any features. There is also this <strong><em>output value (i.e. the sumation of all feature contributions and base value) </em></strong>which is equal to the prediction of the actual model. SHAP values, then, just tells you how much contribution each feature adds in order to go from the <strong><em>base value</em></strong> to the<strong><em> output value</em></strong>.</p><p>For figure 1, you can understand it as follows:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9b32d2c5c9df1f87225510f796bf8300/href">https://medium.com/media/9b32d2c5c9df1f87225510f796bf8300/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/530/1*39OBbv5-G6hb8R1RLfKDWg.png" /><figcaption>figure 2. SHAP values from shap_df</figcaption></figure><p>These are all the individual feature contributions to get from the base value to the output value from the record (i.e. individual item in the dataset) displayed as a force plot in figure 1. You can see that there are positive and negative contributions of different magnitude. In this case <strong><em>x,y,z,carat, clarity, cut </em></strong>and <strong><em>depth </em></strong>are all contributing to increase the value from the base value to the value predicted by the model output, whereas <strong>color</strong> and <strong>table</strong> contribute to decrease the magnitude from the base value to the prediction. Also, note that <strong><em>the unit of SHAP values is the same as the actual target value (Price)</em></strong>.</p><p>Here is how you get the output value:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*WmFZ97bod3zfUaCB2uYuUg.png" /><figcaption>Figure 2. How to get the base value and output value</figcaption></figure><p>Try adding them and you will see that the output value is equal to the actual model predicted value. This is a very powerful and insightful interpretation method that helps you understand the contributions in terms of target variable units.</p><p>Now, if you want to see the overall contribution of each feature variable you just do:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a873e038384cfac185f558b98b6ddc72/href">https://medium.com/media/a873e038384cfac185f558b98b6ddc72/href</a></iframe><p>Plotting the mean values of the contributions gives you Figure 3 which shows you the average contribution of each feature variable to the overall mean model output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/517/1*izkp0l8lq53LOMFU8mvNAw.png" /><figcaption>Figure 3. Average impact of each feature en overall prediction magnitude</figcaption></figure><h3>Bonus Extra: LIME and Submodular Pick (SP)-LIME</h3><p>Just for sanity check let’s try an older but also interesting approach.</p><p><a href="https://github.com/marcotcr/lime"><strong>LIME </strong></a>is an algorithm introduced by a paper <em>“Why Should I Trust You?”: Explaining The Predictions of Any Classifier </em>by Marco Tulio Ribeiro, et al. in 2016.</p><p><strong>LIME </strong>is a clever algorithm that achieves interpretability of any black box classifier or regressor by performing a local approximation around an individual prediction using an interpretable model (i.e. in this paper case it is a linear model, etc.). That is, it answers the question<em>: Do I trust an individual prediction enough to take an action based on it? </em>So, it is a good algorithm to inspect or debug individual predictions.</p><p><strong>SP-LIME</strong> is an algorithm that attempts to answer the question <em>Do I trust the model? </em>It does it by framing the problem as a sub-modular optimization problem. That is, it picks a series of instances of the model and their corresponding predictions in a way that is representative of the whole model performance. These picks are performed in such a way that input features that explain more different instances have higher importance score.</p><h4>Result from LIME:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/61c85efadd36f795ad7f9697e14b73c9/href">https://medium.com/media/61c85efadd36f795ad7f9697e14b73c9/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dBx3jyqndg0_H_MXVxajYw.png" /><figcaption>Figure 4. Result of LIME for the same individual record as before.</figcaption></figure><p>We can see that the features x,y,z,carat,clarity, cut, and depth have all positive contributions to the LIME model although in not the same order as SHAP (but they are almost pretty similar). Also, features Table and color have negative contributions to the LIME model confirming as well SHAP values results.</p><p><strong>Result from SP-LIME:</strong></p><p>To calculate the overall contribution we have to use SP-LIME. It is a little bit more tedious as we have to compute the explanation matrix and get the mean for each feature manually. I use the sample_size of 20 and number of experiments to 5 as the SP optimization algorithm takes a heavy toll in ram.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/30adc7acfe0e0b3ab23dfc0f597cad4b/href">https://medium.com/media/30adc7acfe0e0b3ab23dfc0f597cad4b/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x2BfnRx4TpL86WIr2YFUhA.png" /><figcaption>Figure 5. Result for SP-LIME</figcaption></figure><p>SP-LIME overall attribution defers a little (although really not that much) from SHAP but keep in mind that I did not optimize the parameters for SP-LIME. It will most likely be in accordance with SHAP if I use a larger sample size and increase the number of experiments run by the SP algorithm as they should since LIME was demonstrated to be an AFA-type algorithm and is an approximation to the Shapley values.</p><h3>Conclusion:</h3><p>I hope you have learned something new and encourage you to try it in your own datasets with different methods (e.g. DNN, CNN, catboost, etc.). It works also for classification and different types of data such as image data and text data. Any question let me know at eduardo.denadai[@]gmail.com</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9f52e54d72ab" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/interpretability-of-deep-learning-models-9f52e54d72ab">Interpretability of Deep Learning Models</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transfer Learning for incremental class size datasets]]></title>
            <link>https://medium.com/@eduardoperezdenadai/transfer-learning-for-incremental-class-size-datasets-1fc94810705f?source=rss-65aad403c35d------2</link>
            <guid isPermaLink="false">https://medium.com/p/1fc94810705f</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[research]]></category>
            <dc:creator><![CDATA[Eduardo Perez Denadai]]></dc:creator>
            <pubDate>Fri, 26 Oct 2018 03:43:47 GMT</pubDate>
            <atom:updated>2018-11-18T16:23:36.085Z</atom:updated>
            <content:encoded><![CDATA[<h3>Deep Transfer Learning: Practical Guide</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*Bk15WsVSHWBNyXm68OOd9w.jpeg" /><figcaption>Learning to learn</figcaption></figure><p>Let’s say you have a model that was trained on a huge dataset, lots of time and resources went into training it, and one of the variables is categorical and has 10 classes (or levels). The input to the original model was trained with this 10 classes. This is fine if you are dealing with a situation where at the time of retraining you still have the original 10 classes or less. However, after a short while in production you start having 11, 12, or more classes? Now there are more classes than before that were not taken into account when you trained the initial model.</p><p>This situation happens constantly. Here is a simple example, you have a model that detects if an image contains an animal that flies or not, and your model has feature variables named <strong><em>animal_type: cat, dog, chicken</em></strong> but after a short while your incoming data has one more type <strong><em>animal_type: cat, dog, chicken, and platypus</em></strong>.</p><p>Wow, what to do now… You are faced with either:</p><ul><li>Retrain from scratch the whole model which is expensive in resources and cost, or…</li><li>Perform transfer learning which is cheaper.</li></ul><p>To perform transfer learning successfully on problems like these you need to know some training strategies. In this article, I will show some training strategies for transfer learning (You will need Keras/tensorflow for this).</p><h3>Transfer learning strategy #1:</h3><p>The first approach assumes you have access to the initial training that that was used in the original model. Let’s say you have the following model:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b7d7eb54a63cc620445159147cfd04f3/href">https://medium.com/media/b7d7eb54a63cc620445159147cfd04f3/href</a></iframe><p>Now, what we can do is add the old model (model_1) to our new model (model_2). In order to take advantage of the last state of the trained original model, we<strong><em> load the original model and fixed the old model weights as non-trainable and attach our new model to the last dense layer of the old model.</em></strong></p><p>That is a mouthful but what it means is that we are instructing the model to not perform back propagation to the layers of the original model so that they remain the same (do not mess with was learned in the old model) and just perform training on the new model.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/92033eb2a281782d4e0f1bbd8bb2f900/href">https://medium.com/media/92033eb2a281782d4e0f1bbd8bb2f900/href</a></iframe><p>Now, you have model that looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1BisHvOeQa2Se9TeBXBWuw.png" /></figure><p>An important aspect to note on this approach is that in order to have an add layer the tensors have that are added are required to have equal dimensions.</p><h3>Transfer learning strategy #2:</h3><p>Another way we can try to perform transfer learning is to directly add the old model layers into a new sequential model. This can be accomplished as follows:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7ccef60c12d80edf1ee54d45140dda1b/href">https://medium.com/media/7ccef60c12d80edf1ee54d45140dda1b/href</a></iframe><p>You can see in the code above in <strong><em>code line 22</em></strong> that we are directly accessing the weights old model layer and fixed them(set as non-trainable), in this case I chose the widest hidden layer but you can actually get all of them, or an arbitrary subset of them. The code above will give you the following model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/364/1*HbWkPir0_PHN27_IGMV2UQ.png" /></figure><p>You can see <strong><em>layer_11</em></strong> is the layer (with non-trainable wights) of the old model inserted sequentially in the neural network.</p><h3>Transfer learning strategy #3:</h3><p>It is interesting to note that you don’t have to set the weights fixed through the whole training cycle of your model. Research shows that you can actually leverage your old model better and have better performance if you alternate training between batches[<a href="https://arxiv.org/pdf/1606.09282">see paper 1</a>]. You can accomplish this as follows:</p><h4>Approach #1: The hack</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/320e3a4a1e4a4619fee6f89fc2fbcbcd/href">https://medium.com/media/320e3a4a1e4a4619fee6f89fc2fbcbcd/href</a></iframe><h4>Approach #2: the better approach</h4><p>Develop a custom callback alternating between trainable and non-trainable between batches.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/41e498238afb876101d63193ea70134a/href">https://medium.com/media/41e498238afb876101d63193ea70134a/href</a></iframe><h3>Conclusion</h3><p>I hope you have learned at least something new and I encourage you to try these approaches. If you have any question let me know eduardo.denadai[@]gmail.com</p><h3>Reference</h3><ol><li><a href="https://arxiv.org/pdf/1606.09282">https://arxiv.org/pdf/1606.09282</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1fc94810705f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multi-task Deep Learning Architectures Worth Knowing and codes]]></title>
            <link>https://medium.com/@eduardoperezdenadai/multi-task-deep-learning-architectures-worth-knowing-and-codes-e981dec7132e?source=rss-65aad403c35d------2</link>
            <guid isPermaLink="false">https://medium.com/p/e981dec7132e</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[research]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Eduardo Perez Denadai]]></dc:creator>
            <pubDate>Thu, 04 Oct 2018 20:36:13 GMT</pubDate>
            <atom:updated>2018-10-05T03:05:25.920Z</atom:updated>
            <content:encoded><![CDATA[<h3>Multi-task Deep Learning Architectures Worth Knowing</h3><p><em>***For this article I will jump straight to code so I assume you understand most topics about ANN/DL layers (e.g. Fully-Connected Layers, Convolutional Layers, Embeddings, etc.) and know your way around Keras + tensorflow. Also, There is a pretty neat review about Multi-Task learning </em><a href="http://ruder.io/multi-task/index.html"><em>here</em></a><em> that I encourage you to read .***</em></p><p>Deep Learning (DL) is a very flexible family of algorithms that apply artificial neural networks (ANN) to many types of classification and regression problems that has had great success surpassing human level accuracy on many specific tasks. I would like to tell you about some DL architectures that are straightforward and very usuleful.</p><p>There is a sub-field of Machine Learning (ML) that attempts to develop models that can perform many different tasks at once by sharing domain information about a specific problems. It is rightfully named Multi-task Learning (MTL).</p><p>I m going to jump right in and show the architecture and code so you can start prototyping. These architectures are not fully functional but are meant to get you going right away.</p><h3>Two or more Regression Tasks</h3><p>Lets say you are tasked with predicting both users Cholesterol level and weight. You are given several variables from demographic, socioeconomic, geographical and even their textual reviews from a food review app.</p><p>If you look at this problem you can see that both objective variables might share a lot of feature variables (i.e. predictors) that may belong to different knowledge domains. So by using MTL you can transfer learning from one domain to the other.</p><p>This same technique applies to two or more classification tasks, you would only change the last activation layer for either softmax (multiclass) or sigmoid (binary).</p><h4>Example Architecture №1:</h4><p>Two different feature domains and 2 regression tasks with Fully Connected Layers Only.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/893/1*IHAQeOzWWPslyRtkCvSVtg.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a74380e8afd5dcd236716e935971203b/href">https://medium.com/media/a74380e8afd5dcd236716e935971203b/href</a></iframe><h4>Example Architecture №2:</h4><p>Two different feature domains, with domain specific embeddings. This Architecture is very powerful as it allows to apply vector embeddings to the different domains. This can be useful for further understanding of the problem such as segmentation, ranking, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/925/1*PrOgXy62QrJ67Iv6mnWwww.jpeg" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e4d1b6ba89b4a52b6834ce49cfeccb44/href">https://medium.com/media/e4d1b6ba89b4a52b6834ce49cfeccb44/href</a></iframe><h4>Example Architecture №3:</h4><p>Now let’s say you want to apply 2d-convolutional layers with max pooling to the problem. This is actually useful when you would like to study the structure and relations among features that are learned by the convolutional filters. Or, even have a domain space with images. However, if you have only tabular data and still want to use 2d-convolutions you need to structure your problem in such a way that it can fit into the 2d-convolution paradigm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PKDs-hzZh9s6Sa9oPt4kdQ.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/985987d559adb68037cd3c5dcc656cda/href">https://medium.com/media/985987d559adb68037cd3c5dcc656cda/href</a></iframe><h3>Mixed Regression and Classification Tasks</h3><p>Now Imagine you are given the task to build an algorithm that predicts both Cholesterol level and the probability that a user will fall into three categories (A) no sick no hospital, (B) sick but no hospital, or (3) sick and hospital.</p><p>As before, it may seem highly plausible that our model could be benefited by using MTL since both objective variables might share information from different domains. So, lets say we have data from medical records, demographic, socioeconomic, and geographical.</p><h4>Example Architecture №1:</h4><p>Two different feature domains, with individual embeddings, a 2d-convolution, with 2 tasks (one is a classification and one is a multilabel classification).</p><p>I will show you only this architecture since one you can iterate over it to better suite your needs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1L5PaSBOtdmWvyR_frathQ.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/86a84dafab6aac90cd5bb06d0e30b6de/href">https://medium.com/media/86a84dafab6aac90cd5bb06d0e30b6de/href</a></iframe><h3>Conclusion</h3><p>Hopefully this article will help you quick start prototyping functional multitask deep learning architectures if you are interested in the topic. If you have some questions on actual functional architectures or comments just drop me a message at eduardo.denadai[_at_]gmail.com</p><p>Cheers!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e981dec7132e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Soft Skills for Good Data Scientists]]></title>
            <link>https://medium.com/@eduardoperezdenadai/soft-skills-for-good-data-scientists-29b692b0fad2?source=rss-65aad403c35d------2</link>
            <guid isPermaLink="false">https://medium.com/p/29b692b0fad2</guid>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Eduardo Perez Denadai]]></dc:creator>
            <pubDate>Sat, 21 Jul 2018 16:27:39 GMT</pubDate>
            <atom:updated>2018-07-22T21:31:14.095Z</atom:updated>
            <content:encoded><![CDATA[<p>Research, prototype, code, test, deploy. That’s how a usual day goes in my life as a data scientist. I work in a large traditional banking organization that is attempting to become a data-driven business. It’s been doing a legitimate attempt by implementing a data science department for the past 2 years. Few people if none understand truly what we do but they sure care about the results of our work. Yet, to most we are black box.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*_T6i4Dgka5cuzHySwOBskg.png" /><figcaption>Maybe just a little food please.</figcaption></figure><p>So far, we have been able to deploy various traditional machine learning (ML) and some deep learning (DL) models to the real world causing a positive impact in areas such as commercial and risk management. After the word of success has spread to other departments the demand for models has increased and departments across many functional areas of the organization are interested to explore how data science can help.</p><p>This situation has led functional staff from other areas of the business to seek interaction with our data science department. These functional staff are by no means experts, nor they should be, but this is not an excuse to exclude them from the algorithm development life cycle. We have noticed that in traditional organizations there is a tendency to favor model interpretability of over high abstraction (i.e. ML vs DL models). Inclusion and involvement of these functional staff is of extreme importance in situations where abstract models (e.g. DL) are in need.</p><p>So, I want to share some core values we try to uphold in our attempt to facilitate the participation of functional business areas in the algorithm development life cycle:</p><ul><li><strong>Data Scientists must listen</strong> to take advantage of the huge business experience inside the organization in the form of more senior staff to build good models responsibly.</li><li><strong>Data Scientists must be inclusive</strong> and seek to include functional staff in the development lifecycle of an algorithm. It facilitates the deployment stage by smoothing any unnecessary friction that could arise because the model can be perceived as intrusive into the roles of other departments.</li><li><strong>Data Scientists must be sensible</strong>, our work usually tries to emulate (using algorithms) the intelligence of business processes that are usually performed somewhere in another department by a person. Obviously, working isolated can cause unnecessary friction at the moment of deploying a model.</li><li><strong>Data scientists must provide value </strong>on every algorithm that must be researched, tested and deployed. Always ask the requesting party to provide a return on investment (i.e. $) for the solution they need. This is the value of your algorithm.</li><li><strong>Data Scientists must build systems not algorithms </strong>just like anything that is related to computer software in general. No algorithm seats there on itself and by itself as they must be part of a systems or they will likely be shelved and forgotten. Traditional data warehousing architectures in large organization have data sources spread across different custodial domains whose patrons are always weary of anyone messing with it.</li></ul><p>These are just a core set of values that have proven to be a requirement for me as data scientist to achieve maximum impact and drive change (i.e. a data-driven change) in my day to day. Always have in mind that an algorithm can be shelved and forgotten very easily for many reasons if it is not sold appropriately inside an organization, just at least try not let it fail because of poor soft skills strategy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YCjf5aZ--L5J3pQoUc6kgQ.jpeg" /><figcaption>Try to include others into your algorithm development cycle.</figcaption></figure><p>I would love to hear what other values you see helpful in pushing a data driven change in your organization. In my next article, I want to share a design thinking approach to data science that helps include other functional areas into the algorithm development life cycle.</p><p>Cheers!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=29b692b0fad2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Web Scrapers y Fuzzy Matching: Enriquecimiento de Bases de Datos-Parte 1]]></title>
            <link>https://medium.com/@eduardoperezdenadai/web-scrapers-y-fuzzy-matching-vinculaci%C3%B3n-de-records-parte-1-b84636666b05?source=rss-65aad403c35d------2</link>
            <guid isPermaLink="false">https://medium.com/p/b84636666b05</guid>
            <dc:creator><![CDATA[Eduardo Perez Denadai]]></dc:creator>
            <pubDate>Tue, 20 Feb 2018 16:01:47 GMT</pubDate>
            <atom:updated>2018-02-20T16:27:06.652Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*If_dxG9I2c9XnhMF." /><figcaption>Photo by <a href="https://unsplash.com/@jammypodger7470?utm_source=medium&amp;utm_medium=referral">michael podger</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>T</em>u o tu empresa necesitan contactar a clientes pero una proporción importante de estos tienen su información desactualizada. Esta situación es común muchas industrias de servicios (como bancos, aseguradoras, etc.) para la contactabilidad de clientes en procesos de negocio o para temas regulatorios y de cumplimiento.</p><p>Aquí aprenderás un área importante de minería de datos llamada vinculación de records (<em>record linkage</em>). En la primera parte de este artículo primero te enseñaré a obtener datos de clientes en fuentes libres de datos en internet y en la segunda parte vincular estadísticamente a estas personas con tus clientes sin la necesidad de tener una llave (<em>key</em>) que los vincule directamente a tus bases de datos por medio de tecnicas de <em>fuzzy matching.</em> Entenderás que en este caso, toda data es útil ya que en el caso de baja probabilidad de vinculación tendrás un nuevo cliente potencial para contactar.</p><h3>Parte 1: Robot Web Scraper</h3><h4>¿Qué es un web scraper?</h4><p>Un robot web scraper (o bots como se les conoce) es un programa diseñado para obtener información de paginas web de forma autónoma. La escritura de web scrapers es un rubro comun en la minería de datos. No todas las paginas de internet son iguales por lo que la escritura de web scrapers es bien especifica a la página, varía en complejidad dependiendo de que tanto el web scraper tiene que emular un usuario real para obtener acceso a las paginas web.</p><h4>¿Que necesitarás?</h4><p>Para iniciar, mi principal instalación de python para el análisis de datos es el suite de Anaconda (<a href="https://www.anaconda.com/downloads">https://www.anaconda.com/downloads</a>) se los recomiendo de lo contrario tendrán que instalar muchas librerias manualmente. Una vez instalado Anaconda necesitarán:</p><ul><li>selenium python binding: framework para el desarrollo del web scraper</li></ul><pre>&gt; pip install selenium</pre><ul><li>chromedriver.exe: driver de Google Chrome que será el browser utilizado para acceder al sitio web y lo puedes encontrar en: <a href="https://sites.google.com/a/chromium.org/chromedriver/downloads">https://sites.google.com/a/chromium.org/chromedriver/downloads</a></li></ul><p>(asegurate de que la version del driver corresponda a la tu instalación de chrome)</p><h4>Las Paginas Blancas de Panamá</h4><p>En este articulo obtendremos datos de las paginas blancas. Lo primero que tienes que hacer es ingresar a la página en tu buscador, y viendo el url puedes observar que la estructura del url indica que los datos públicos de personas van desde <em>page-91</em> hasta page-838</p><blockquote><a href="http://images.paginasamarillas.com/flipping/panama/paginasblancas/files/assets/basic-html/page-1.html">http://images.paginasamarillas.com/flipping/panama/paginasblancas/files/assets/basic-html/<strong>page-91.html</strong></a></blockquote><p>…</p><blockquote><a href="http://images.paginasamarillas.com/flipping/panama/paginasblancas/files/assets/basic-html/page-838.html">http://images.paginasamarillas.com/flipping/panama/paginasblancas/files/assets/basic-html/<strong>page-838.html</strong></a></blockquote><h4>Entiende la Estructura de la Página Web</h4><p>Si utilizas google chrome, apreta <strong>F12<em> </em></strong>para abrir las herramientas de desarrollo del buscador y así poder explorar la estructura html de la pagina web.</p><p>Explorando la estructura html te darás cuenta que el tag:</p><pre>&lt;meta name=&#39;description&#39; content =&quot;DATA&quot;&gt;&lt;/meta&gt;</pre><p>contiene la información requerida.</p><h4>El Código — Bot</h4><p>Bueno, vamos all grano:</p><pre># /foo/bar/webscraper.py</pre><pre>from selenium import webdriver<br>import os</pre><pre>driver = webdriver.Chrome(&quot;C:\\foo\\bar\\chromedriver.exe&quot;)</pre><pre>with open(&quot;C:\\foo\\bar\\paginasblancas.txt&quot;, &quot;a&quot;) as myfile:<br>    for i in range(91,838):<br>        page = &quot;http://images.paginasamarillas.com/flipping/panama/paginasblancas/2013/files/assets/seo/page-&quot;+str(i)+&quot;.html&quot;<br>        driver.get(page)<br>        elem = driver.find_element_by_xpath(&quot;//meta[@name=&#39;description&#39;]&quot;)<br>        text = elem.get_attribute(&quot;content&quot;)<br>        print(text)<br>        myfile.write(text)<br>driver.close()</pre><p>Este sencillo programa hace lo siguiente:</p><ol><li>Activa driver de google chrome (chromedriver.exe)</li><li>Abre in archivo de texto donde se colocara toda la información extraida de cada direccion url.</li><li>ejecuta jalar toda la <em>DATA </em>dentro del atributo content desde la pagina 91 hasta la pagina 838.</li></ol><p>El bot se ve así:</p><pre>&gt; python webscraper.py</pre><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FV7XW2mWQJoU%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DV7XW2mWQJoU&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FV7XW2mWQJoU%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/8a36df31cb96cc0a90433f5112202bae/href">https://medium.com/media/8a36df31cb96cc0a90433f5112202bae/href</a></iframe><p>y el resultado del scraper es guardado en <em>paginasblancas.txt </em>se ve así:</p><pre>Andes2,267–9116,CasadeFernando,30,Exposicion,225–2378,ACOLOR,SA,UrbanizaciónLosAngeles,Calle63y1/2,Duplex,#45,263–9885,A.4SALESGROUP,2,VillaDe,Las,Fuentes2,,F,13,236–5014,CutAboveS,Albrook,398–0742,A.,2.,SoftwayPanamá,Fco,394–3545,AEATECNITECHOS,Sabanas,224–0800,Consulting,S,RdoJ,Alfaro,302–7927,AESPANAMA,EDIFICIOBUSINESSPARK,TORREV,PISO11,URB.COSTADELESTE,AVE.LA,ROTONDA,206–2600,A.F.NationalSecurity,S,35,AvePerú,390–2922,Progrramas,Interculturales,BellaVta,223–8480,FLPanama,Betania,236–7915,FuegoLento,50,269–3000,AGAUTOPARTES,SA,ALLADODELAGARANTIA,DERIO,ABAJO,VíaEspaña,222–1800,GB,Consulting,269–7662,ConsultingS,57,Obarrio,395–3680,G.,C.,dePanamá,CdadDel,Saber,317–3650,GConsultor,ErickDel,ValleEl,Cang,392–1361,G,Impresores,S,Nvo,Veranillo,267–1245,GMorrel,Acacias,266–7694,GS,Decoraciones,,75,SanFco,270–4911,H,&amp;MContratistas,,DonBosco,395–4161,Houseof,TradingPanamá,S,264–8102,A.I.C.Business,Bda,Loma,399–5494,I,AgencyFor,International,Education,Corp,390–9281,A.I.R.SA(AJUSTADORES-SURVEYORS),Israel,399–6900,T,Panamá,S,VíaRdo.J.Alfaro,236–8201,&amp;M,,SamuelLewis,394–9854,AJDAYAN,SA,AreaBancaria,264–0555,J,IngenierosdePanamá,S,CampoLindbergh,392–2893,Salón,MiguelBrostella,236–5069,236–6885,Joannesy,NiviaA,25,Calidonia,212–0165,L,G,,PlazaConcordia,223–7918,P,263–5875,Asociados,S,Jto,Aros,398–1870,398–1871,Lugaresy,Destinos,S,214–4193,M,ElementosS,55,UrbEl,265–1643,A.M.Impresores,Fernándezde,Córdoba,PuebloNuevo,396–7700,M.,ProductionsCorporation,395–3605,MT,Arquitectura,MontajesTécnicosPanamá,,393–3348,M&amp;MdePanamá,Argentina,399–6625,OrtizContadoresPúblicos,CampoLindberghJuanDíaz,391–3163,Tocumen,291–3407,Pools,Bethania,261–7807,ConsultoresGerenciales,,396–0184,PlusPerformance,Inc,ElviraMéndezy,EspañaBellaVta,263–0307,263–0308,A-PrintPanamá,S,Lefevre,235–8305,PrioriDecor,S,263–7636,PrioriDecor,SA,263–7656,R,Investment&amp;,A,,AlbertoNavarroBellaVta,209–2859,A.S.Fashion,S,CdadRadial,290–4998,ASGRAPHICS,,SA,SanGabrielObarrio,263–2474,SMotors,4,VtaHermosa,391–6038,SociedadDeEduc.A,NivelSuperior,39,Cuba,225–1791,Ingenieríay,Obras,S,D,UrbStaClaraJuanDíaz,22-A,233–0401,ATIEMPO,SA,Galería,Balboa,,Ofic16,264–5919,1,MediaInc,279–0278,VergaraAndCo,S,El,264–2881,Vergara&amp;,Co,S,223–7015,394–6403,Compañía,S,223–7147,WArquitectosConsultores,,70,392–2727,GSalón,S,Y,97,397–2990,YSRepaciones,AlcaldeDiaz,268–7340,A.Z.EXPRESS,SA,260–0607,&amp;,Cargo,S,Sta,Elenay,Andalucia#53ChanisAltosdel,Romeral,392–3171,Store,S,Panamá,302–3709,JobsSolutions,Dorado,393–1370,ServiciosGenerales,VillaLorena,397–5738,TecnoServices,S,392–0384,AlfaroBethania,392–0383,DDesignGroupPanamáCorp,47,269–9652,52,269–9643,Trading-,WitColor,Villa,las,Fuentes,Local5,396–8886,Corporation,,61,Obarrio3,396–6050,396–6051,GRents&amp;Management,Marbella,203–0079,203–0080,Accesories,LosAngeles,398–3161,Z,Golden,S,260–7508,AABACDETECTIVE,Cumbres,390–3596,AADetectivesPrivado,221–1713,AAPUBLICIDAD,Ave,8,63,,UrbLosAngeles,263–0063,AAServices,PpalSanAntonio,396–0939,AaasPanamá,S,Balboa,213–1344,AaasPanamáS,213–1343,AAGEHempelPanamá,S,RobertoChiari,838–5807,AarónStudio,PuntaPaitílla,392–6140,AarónStudio,S,ItaliaPaitilla,390–9092,AasPanamá,S,49Marbella,391–5417,ABDesign&amp;,Productions,ErnestoT,399–8500,Ab,el,Aguila,S,225–0095,AbForwardingCorp,260–6264,Impresiones,,236–7100,AB-proyectos,AltaGama,S,5,Cocodel,Mar,391–2069,AbaAbamilecy,TechosRomeros,SanMgto,267–3479,AbaLegalBureau,SamuelLewis,Obarrio,263–4455,58,263–5773,AbaServiciosFinancierosPanamá,S,Pta,Pacífica,216–8030,Abacal,S,EspañaP,221–7664,ABACUCCREATIVESTUDIO,ParqueLefevre,391–6039,AbacucCreativeStudio,AbacusOem,S,Gloria,260–4891,AbadiaOrquesta,(Sensaciones),PqueLefevre,221–2490,Abaroteriay,CarniceríaChong,CristoRey,392–4828,AbarroteríaCarniceríay,Bodega,7,StaMarta,76,231–5183,Abarrotería12deOctubre,12,Oct,262–5405,262–6390,AbarroteriaFumanchu,Rio,Abajo,222–0988,yAv12deOct,852,224–1911,Cl32SantaClara,,12791,253–2122,Cl14Oeste,StaAna,228–7659,SectSurTocumen,295–1082,VillaCáceres,261–3616,267–2171,Torrijos,Carter,234–3392,GrupoCanalero,272–8726,Santiago,998–7131,APARTADO0816–01349,FAX209–5401,www.abb.com,VerAvisoenSección:,Eléctricos,Materiales,Accesorios,FAX:206-2612,PLANTAS,LAESTRELLA,-,LOSVALLES-,ESTI,772-5150,FAX:772-5730,BAYANO,298-9172,298-9175,FAX:298-9180,CHANGUINOLA,758-3059,APARTADO:,0816-01990,www.aespanama.com,www.agautopartes.com,VerAvisoEnSecc:Automóviles,Repuestos,E-Mail:airsa@airpma.net,Fax399-6949,Colón,441-6920,998-6981,David,775-0593,Fax,263-8642,VERAVISO,ENSECCION:,ROTULOS,264-5148,Celular,655-14759,667-22553,24Horas,663-16612,665-34151,E-Mail:,gbisa2@hotmail.comAbarroteríaGallito,Cl,15,PbloNvo,55-A,261-0104,.,AbarroteríaLaGallinitaVerde,1,BdaPuntaDel,E,399-2828,AbarroteríaPacífico,3,PmáViejo,95,224-4961,AbarroteríaPopular,Av,9,S,Chorrillo,18-12,228-6572,AbarroteríaSanMartín,PqueLefevre,224-4756,AbarroteríaSanPedro,San,Pedro,390-3188,AbarroteríaSantaEduviges,PpalSta,Eduviges,291-1360,AbarroteríaSky,Bda,8,De,Dic,391-8818,AbarroteríaTakCheung,Cls,B,y,16,CdadRadial,78385,266-4746,Abarroteríay,BodegaAgacucho,Vía,Ppal,LasCumbres,396-1108,BodegaVíctorEle,5,R,Abajo,2457,221-1026,CarniceríaCecilia,10,Abajo,19-10,224-1488,CarniceríaCiudadJardín,PpalSect6,UrbMañanitasde,Tocumen,68,292-8402,</pre><p>Eso es todo. Ya aseguraste una fuente cruda de datos que procesada (luego de limpiar y colocarla en el formato deseado-<em>data wrangling</em>) cuenta con más de 290,000 Mil clientes potenciales.</p><p>En la segunda parte te enseñaresmos como vincular los records obtenidos de estas personas de forma estadistica, sin la necesidad de una llave (Key), utilizando tecnicas de <em>Fuzzy Matching.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b84636666b05" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>