<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Metika Sikka on Medium]]></title>
        <description><![CDATA[Stories by Metika Sikka on Medium]]></description>
        <link>https://medium.com/@metika?source=rss-127625b11501------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*LjMG_zAfL_YCxJpFl4sZyw.jpeg</url>
            <title>Stories by Metika Sikka on Medium</title>
            <link>https://medium.com/@metika?source=rss-127625b11501------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 13:45:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@metika/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Human Bias-Accuracy Trade-off]]></title>
            <link>https://medium.com/data-science/the-human-bias-accuracy-trade-off-ad95e3c612a9?source=rss-127625b11501------2</link>
            <guid isPermaLink="false">https://medium.com/p/ad95e3c612a9</guid>
            <category><![CDATA[salary]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[algorithmic-bias]]></category>
            <category><![CDATA[equality]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Metika Sikka]]></dc:creator>
            <pubDate>Tue, 23 Feb 2021 08:22:06 GMT</pubDate>
            <atom:updated>2021-02-23T13:07:48.641Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CIsH7ReAMUGHjLIJJd90nQ.jpeg" /><figcaption>Image Source: Photo by <a href="https://unsplash.com/@jontyson?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Jon Tyson</a> on <a href="https://unsplash.com/?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h4>Understanding how historical data can lead to algorithmic bias taking a naive example of a compensation prediction model</h4><h4>To be human is to be Biased?</h4><p><strong>“Bias”</strong> is a <em>tendency or inclination </em>to <strong>favor or disfavor</strong> a certain set over the other. All humans have a certain degree of bias because we are <a href="https://www.kornferry.com/insights/articles/understanding-bias-and-brain"><em>inherently</em> programmed </a>to discern anyone different as a threat. Due to this implicit bias, we tend to unconsciously ascribe traits and qualities to usually stigmatized groups in society. These include groups based on sex, race, nationality, or geographic heritage. According to<a href="https://plato.stanford.edu/entries/implicit-bias/#ImplMeas"> Stanford Encyclopedia of Philosophy</a>-</p><blockquote>“Research on ‘implicit bias’ suggests that people can act on the basis of prejudice and stereotypes without intending to do so”</blockquote><p>Now if we know and understand this fact, we have a way to eliminate this implicit bias i.e. to be aware and recognize how it influences our cognitive decision making. This would involve making conscious efforts to change stereotypes and adjusting our perspective.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*pRjfe1zOTaAm7S1eXHd3HA.gif" /><figcaption>Image Source:<a href="https://media.giphy.com/media/xT8qBit7YomT80d0M8/giphy.gif"> GIPHY</a></figcaption></figure><h4>The Endless Quest for Accuracy</h4><p>The goal of Machine Learning is to get the most accurate predictions. Data scientists have several metrics to quantify this set goal by minimizing known prediction errors. In the case of regression problems these metrics are Root Mean Squared Error, Mean absolute error, R-squared value, etc. For classification algorithms, the metrics include AUC, Accuracy, Precision, Sensitivity, Specificity, and so on. Several methods are used to further tune and then fine-tune the parameters and hyperparameters.</p><p>Everything boils to optimizing one particular <em>trade-off</em> — the bias-variance tradeoff; making sure that we get accurate predictions for any part of the population data.</p><h4>The Overlooked Trade-off</h4><p>While getting a higher accuracy, one fact is forgotten — Data collected by humans is riddled with human bias. <a href="https://www.theguardian.com/commentisfree/2016/oct/23/the-guardian-view-on-machine-learning-people-must-decide">The Guardian</a> articulates this fact well:</p><blockquote>Although neural networks might be said to write their own programs, they do so towards goals set by humans, using data collected for human purposes. <strong>If the data is skewed, even by accident, the computers will amplify injustice.</strong></blockquote><p>Human bias can creep in at any point in the data science pipeline and take the form of <strong><em>Algorithmic bias</em></strong>. Of course, the variety of data collected is the most susceptible point. And that is also connected to optimizing the bias-variance trade-off as mentioned before. Additionally, either this data is collected by humans, or the process of data collection is set up by humans. This implies that unconsciously the agent of data collection can also be the source of bias entering the pipeline. Moreover, the insights derived from the predictions generated can also be influenced by the subjectivity of the human interpreting the results. The immediate effect of algorithmic bias can be discriminatory actions against or for particular groups of people.</p><p>Surprisingly, this isn’t a new problem. The article, “<a href="https://hbr.org/2019/10/what-do-we-do-about-the-biases-in-ai">What do we do about the biases in AI</a>” (HBR, 2019) talks about how back in 1988, a British medical school making admission decisions based on an algorithm, was found to be guilty of bias against women and those with non-European names. Interestingly, this algorithm demonstrated a 90–95% accuracy in matching human decision-making.</p><p>A more recent example of algorithmic bias has been highlighted in <a href="https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/17/can-an-algorithm-be-racist-our-analysis-is-more-cautious-than-propublicas/">this article</a>, by Washington Post in 2016. It talks about the software, COMPASS used nation-wide in the United States to make bail &amp; sentencing decisions which the organization ProPublica found to be biased against African-American defendants.</p><h4>Let’s see an Example</h4><p>Gender disparity in the workplace has existed and is still prevalent. Thus, historical employee data is bound to have a biased trend against females. To understand algorithmic bias at play, let’s take the example of historical compensation data of senior officers of publicly listed companies<em>. </em>We make models <em>with and without</em> the feature <strong>‘<em>gender</em>’ </strong>of the senior officer to predict their direct compensation. Further, we compare results to see how including <em>bias-sensitive features</em> can lead to discriminatory results.</p><h4>About the Data</h4><p>The data has been obtained from BoardEx using Wharton Research Data Services. It covers four geographic categories: US, UK, European Union (EU), all other countries. Two data sets were used to get different features related to compensation of senior officers at publicly listed companies:</p><p>1. <strong>Compensation data:</strong> Includes individual data (taken from annual reports) on present and past stock option holdings, direct compensation, time to retirement from each position served, time served in the company, information about the board (if serving), sector category of the firm, information about past boards (if served).</p><p>2. <strong>Individual profile characteristics:</strong> Includes data on gender, educational qualification, age, network, or connections.</p><p>The detailed data dictionary can be accessed<a href="https://metalib.ie.edu/ayuda/Varios/BoardExWRDSDataDictionary.pdf"> here</a>.</p><p><strong>Data Cleaning</strong></p><p>There were duplicate values in the data due to companies having multiple tickers, these were removed. After which the number of unique BoardIDs (a unique identifier for company boards) in the data was — 17K from the US, 5K from the EU and UK each, and about 8K from other countries. The final merged data (all countries) had information for approximately 325K directors/senior officers.</p><p>However, for each director, the data had multiple entries taken from annual reports of the companies spanning over different years. For example, if a director served a company from 2000–2010 then there are 10 entries for this director-company pairing. For the sake of our analysis, we retained the director-company pairing with the highest total direct compensation of director X in company Y.</p><p>After choosing the variables most relevant to our analysis i.e. important in deciding a senior officer’s compensation the final data set had — 80879 rows and 21 variables. Now, our data is such that each row is a unique director-company pairing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IUmP8hWfsFemZHIGqcTPQw.png" /><figcaption>Description of Variables in the Data set (By Author)</figcaption></figure><p><strong><em>Note: </em></strong>We will use Total Direct Compensation as the dependant variable (Y) in our analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/743/1*f49LJkAylfKCp-6cY2BkyA.png" /><figcaption>Snapshot of the data (By Author)</figcaption></figure><h4>Exploratory Analysis- Is the data really biased?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/664/1*20klMCEhI0L--Cbk59iDfA.png" /><figcaption>By Author</figcaption></figure><p>Looking at just the number of Females and Males in our data, we see a huge discrepancy. We have 9,908 Females compared to 70,971 Male senior officers!</p><p>This is also corroborated by the distribution of the Gender Ratio or the proportion of male senior officers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y5N4I59L_Qg3lJqAI8doeg.png" /><figcaption>By Author</figcaption></figure><p>Let’s look at the <strong><em>mean total direct compensation </em></strong>by gender. As seen below, the mean total direct compensation for males is almost double that for females.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Mk8GpKX48N_CbHWy5IcDw.png" /><figcaption>By Author</figcaption></figure><p>Now, let’s look at two variables that can be considered as a proxy to access opportunity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uMnKs-8rYVyzXhToblE0rA.png" /><figcaption>By Author</figcaption></figure><p>Surprisingly, there isn’t much difference in the distributions of the time spent in companies between the two genders. Indicating that senior officers irrespective of gender are giving equal time to the companies but receiving different amounts of compensation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/972/1*3ox4Xln6LLQhhyecHtOEIw.png" /><figcaption>By Author</figcaption></figure><p>Comparing the distributions of the number of qualifications senior officers have by gender, we see overall women tend to have more qualifications. This could be indicative of a trend that to reach on par with their male counterparts, women need more qualifications.</p><h4>Modeling Approach</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/506/1*vCtBBw9QfTXGkyu49V8ekQ.png" /><figcaption>By Author</figcaption></figure><p>We first try Lasso regression to see if the variable Gender is picked up by the model. Our <strong><em>dependant variable</em></strong> or Y is the <strong><em>total direct compensation</em></strong></p><p>The model not only chooses Gender as a feature but also predicts the total direct compensation for males approximately 135 units more than that for females. The R-squared for Lasso is 0.38. Thus, the model is only able to explain 38% of the variance in total direct compensation.</p><p>Further, to get a higher accuracy Gradient boosting was tried which gave an R-squared of 0.57. And hence, was able to capture a higher variance in the total direct compensation.</p><p>Contrary to our expectation “Gender” did not come as an important variable according to GBM. However, the mean predicted total direct compensation for males was almost double that of female senior officers. To investigate this further, a GBM model was run without taking “Gender” as a variable. And still, the mean predicted total direct compensation for female senior officers was lower than their male counterparts.</p><h4>Limitations to our Example</h4><p>While we have taken the example of predicting total direct compensation, it is important to consider that no organization uses such an algorithm that uses variables like Gender, Race, Nationality, etc. for this purpose. The data set was chosen to illustrate how historical data can promote social bias in predictions! Another point to remember is that for senior officers of most companies the compensation provided is not only direct. There are indirect components of the compensation which include Stock, Options holdings &amp; Long-term incentive plans. Due to the sparsity of these features, they were not included in the analysis. Since, the data covered organizations from various countries, it is also important to know compensations, tax benefits &amp; firm structures differ from country to country. And hence this may have skewed the results.</p><p>With these Caveats in mind, it is still important to see the fact that a simple predictive compensation analysis can also produce socially biased results even as the accuracy of predictions improve.</p><h4>Conclusion</h4><p>So, what can we do about it? How can we make ‘<em>ethical</em>’ predictions using Machine Learning?</p><p>One solution is to use <em>weights</em>.</p><p>1. Adding different weights to variables with an implicit bias for each class of the population that could have been impacted by social bias. For example, in the case of COMPASS weighing or a normalizing, the number of prior arrests for African-Americans and their White counterparts by their population proportions could be used.</p><p>2. Give more weightage to other variables which are independent of social bias like the seriousness of the crime than to prior arrests.</p><p>Another solution could be to keep monitoring model results with real-time data assuming that the social bias would decrease in data over time.</p><p>Keeping this in mind, for fairness in predictions a shift to ‘Explainable ML/AI’ will propagate and produce more socially unbiased data. An important step in this direction is the EU’s General Data Protection Regulation or GDPR. The data usage is more regulated with special protection being applied to sensitive population characteristics like Gender, Race, ethnic origin, Nationality, Geographic locations, religious beliefs, genetics, trade union memberships, etc. under Article 9.</p><p>Another interesting point in this regulation is the law to treat even online location data of consumers as personal data under Article 4; the use of any personal data is contingent on the consent of the owner under Article 6. Moreover, under Article 13, the companies are mandated to inform the consumers how their data is being utilized.</p><p>We still don&#39;t know the extent to which human bias is affecting fairness in machine learning algorithms. But again as Sheryl Sandberg says, “<strong><em>We cannot change what we are not aware of, and once we are aware, we cannot help but change</em></strong>”!</p><h4>References</h4><p>[1] <em>The EU General Data Protection Regulation (Questions and Answers), </em><a href="https://www.hrw.org/news/2018/06/06/eu-general-data-protection-regulation">Human Rights Watch</a> (2018)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad95e3c612a9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/the-human-bias-accuracy-trade-off-ad95e3c612a9">The Human Bias-Accuracy Trade-off</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Increasing Customer Lifetime Value IRL!]]></title>
            <link>https://medium.com/analytics-in-action-columbia-business-school/increasing-customer-lifetime-value-irl-adacdbae5e01?source=rss-127625b11501------2</link>
            <guid isPermaLink="false">https://medium.com/p/adacdbae5e01</guid>
            <category><![CDATA[customer-lifetime-value]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[shopify]]></category>
            <category><![CDATA[google-analytics]]></category>
            <category><![CDATA[ecommerce]]></category>
            <dc:creator><![CDATA[Metika Sikka]]></dc:creator>
            <pubDate>Sun, 13 Dec 2020 19:19:51 GMT</pubDate>
            <atom:updated>2020-12-16T16:58:47.436Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b9PlXGgaCZ5jFqDCh8CW9w.jpeg" /><figcaption>Image Source: Photo by <a href="https://unsplash.com/@rupixen?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">rupixen.com</a> on <a href="https://unsplash.com/s/photos/ecommerce?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h4>A look into how we used Shopify, Google Analytics &amp; Klaviyo data to increase the Customer Lifetime Value for a Direct-to-consumer E-commerce Start-up.</h4><h4>Where is this going?</h4><p>Start-ups are engaged in a<strong><em> juggling-act</em></strong>: focusing on high-growth, which Reid Hoffman refers to as <a href="https://hbr.org/2016/04/blitzscaling"><em>blitzscaling</em></a><em>, </em>or increasing profitability. A common way to achieve the former is to engage in price-wars and gather more customers. However, the latter goal is a pithy one; centered around the <strong>customer lifetime value</strong> <strong>or <em>CLV</em></strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*n12o5f_lgLNmt5_X3cDBnA.gif" /><figcaption>Image Source: <a href="https://media.giphy.com/media/McVB3GGJuwDzemjPiy/giphy.gif">GIPHY</a></figcaption></figure><p>Simply put, the CLV is <strong>the net profits accrued from the entire relationship between the customer and the firm</strong>. Hence, if we increase the lifetime of this relationship, we can achieve higher profitability.</p><p>Our team at <strong><em>Columbia Business School</em></strong> worked on this objective for a contemporary e-commerce start-up; <strong>that sells one type of product and its paraphernalia directly to its customers</strong>. This blog-post will walk you through our data-driven approach!</p><h4>The Value of Returning Customers</h4><p>To increase its CLV the start-up wants to inculcate a ‘<em>habit-buying</em>’ behavior in its customers. However, a deep dive into the Shopify data revealed that <strong>on average 21% of the customers returned after their first order</strong>. Interestingly, our empirical analysis suggested <strong>an increase in 1% in return probability would increase the yearly revenues by 1.4% which is $110–165K in dollar terms</strong>! And so, we decided to focus on increasing the volume of returning customers thereby, increasing the customers with high CLV.</p><h4>Features of Returning Customers</h4><p>Our next step was to analyze the features of customers returning within 6 months of their first order, in broadly <em>4 categories. </em>To do this <strong>we analyzed the</strong> <strong>first orders of all customers between January 2019-December 2019</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3HNnO87KN0EEm7xG8U7pFQ.png" /><figcaption>Image Source: By Author</figcaption></figure><p><strong>Note: These features were derived from the <em>Shopify</em> order data and <em>Klaviyo</em> data (email marketing platform)</strong></p><h4>What Makes Customers Come Back?</h4><p>After feature engineering, we examined the relationship between these features and the returning probability of customers. Keeping our<strong> target as a binary variable signifying if the customer returns or not in the next 6 months after placing the first order</strong>, we built two models.</p><p>The first model was a <strong>ridge regression</strong> model to capture the <em>directional</em> impact of features on the target variable. We combined these inferences with the variable importance results from <strong>gradient boosting</strong>, an ensemble model. Finally, we were able to identify what did or didn’t make the customers return! We summarise the top 4 insights in the table below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VbWAtZpvkbQaBOAPFqGyIw.png" /><figcaption>Where a very high impact, high impact, and medium impact signify a percentage point change of ≥10%, ≥5%, and ≥3% respectively. Image Source: By Author.</figcaption></figure><p>The impact here refers to the percentage points change in the return probability of customers by a change of one-point in these features.</p><h4>But wait, Where do these Customers Come from?</h4><p>Our funnel approach to get a more granular look at acquisition channels has been summarized below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mL6HdmI3TrCcKt6yKif8FA.png" /><figcaption>Here, the topmost tier shows proportions of acquisition channels over total sessions. In the second tier, the percentages are conversion rates for each channel. The third tier shows the percentage of customers who returned for each channel. Further, 19% is the combined returned probability for Organic &amp; Paid search. Also, the “Direct” acquisition channel is over-represented in our data, as it is the “by default” channel when the channel is unknown. Image Source: By Author</figcaption></figure><p>As we can see, <em>Direct</em><strong><em> </em></strong>and<strong> </strong><em>Email </em>attract the most valuable consumers. <em>Email </em>has room to grow given the current proportion of sessions it generates. <em>Search</em><strong> </strong>attracts the highest number of consumers, but these consumers are average in terms of return probability. <em>Referrals</em><strong> </strong>lead to very attractive first orders in terms of conversion rate. However, these customers are least likely to return.</p><p><strong>Note: Sessions, Conversion &amp; Acquired users were calculated using <em>Google</em> <em>Analytics</em> categorization. For definitions of each channel, have a look at this </strong><a href="https://www.intechnic.com/blog/google-analytics-acquisition-and-channel-reports/"><strong>link</strong></a><strong>. Return Probability was calculated using <em>Shopify</em> order data.</strong></p><h4>Who are these Customers?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*OWEIr2EtEYBiKtpC8uiOKA.gif" /><figcaption>Source: <a href="https://media.giphy.com/media/XZ39zg4naZ1O8/giphy.gif">GIPHY</a></figcaption></figure><p>For the final stage of our analysis, we created <strong>5 profiles</strong> for returning customers across <strong><em>seven dimensions</em></strong><em> </em>based on the importance of features derived from the models. The seven dimensions are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/505/1*Z2MnW11uWXkVZ4C9PPejOw.png" /><figcaption>Image Source: By Author</figcaption></figure><p>Let’s look at highlights for the 5 profiles:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*npD16LKyf52W7CdW_m3ujw.png" /><figcaption>The YoY growth rates have been calculated using 2019 &amp; 2020 data. Table Source: By Author, Images by <a href="https://unsplash.com/@freestocks?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">freestocks</a>, <a href="https://unsplash.com/@hesherlobsteruk?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">He’s Her Lobster</a>, <a href="https://unsplash.com/@belart84?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Artem Beliaikin</a>, <a href="https://unsplash.com/@brett_jordan?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Brett Jordan</a>,<a href="https://unsplash.com/@shiqyun?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText"> Helen Shi</a> respectively on<a href="https://unsplash.com/s/photos/gift?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText"> Unsplash</a></figcaption></figure><p>At almost 40%, <em>Gifters </em>form the highest proportion of customers for the firm and at just 1% <em>Subscribers</em> for the lowest. Interestingly, right now <em>Gifters</em> have the lowest probability of return whereas <em>Subscribers </em>always come back! In general, the firm has a low proportion of males, and Newsletter Subscribers have the lowest proportion at just 13%. On digging deeper we found out that unintentionally, the newsletter did not have a gender-neutral format. Another point to be noted is that <em>Buyers of Paraphernalia</em> have the potential to be profitable customers as they come from high-income neighborhoods. As for Discounters, the affiliate websites seem to be a great success.</p><p><strong>And now a Drumroll for the Endgame…</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/449/1*W4LBI3QCpQL9NEPSZmaNjg.gif" /><figcaption>Image Source: <a href="https://media.giphy.com/media/GBvkxysAR8Svm/giphy.gif">GIPHY</a></figcaption></figure><h4>Our Recommendations</h4><p>Great, now that we know which type of customers return, let’s look at what can be done about it.</p><p>Based on the model results and the customer profiles, here are a few of the recommendations:</p><ul><li>Gifters are more likely to return as Gifters. As they form the largest proportion of customers, marketing campaigns can be aimed at ‘Gifting’ the firm’s products.</li><li>The newsletter can have a gender-neutral format and can be tested further using A/B testing.</li><li>From the model results, we also see that mail marketing should be increased in high-density areas. This can also be used to target Buyers of Paraphernalia by highlighting supporting products in the email campaigns.</li></ul><p><strong>That’s about it!</strong></p><p><em>This blog-post was written as part of the class- Analytics in Action at Columbia Business School by Jeremy Cohen, Ignacio Goulu, Ignacio S Lerin, Metika Sikka, Samantha Bacelic &amp; Vanessa Long.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=adacdbae5e01" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-in-action-columbia-business-school/increasing-customer-lifetime-value-irl-adacdbae5e01">Increasing Customer Lifetime Value IRL!</a> was originally published in <a href="https://medium.com/analytics-in-action-columbia-business-school">Analytics in Action @ Columbia Business School</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How can Videos be Used to Detect Your Personality?]]></title>
            <link>https://medium.com/data-science/can-your-video-be-used-to-detect-your-personality-d8423f6d3cb3?source=rss-127625b11501------2</link>
            <guid isPermaLink="false">https://medium.com/p/d8423f6d3cb3</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[personality-types]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[editors-pick]]></category>
            <dc:creator><![CDATA[Metika Sikka]]></dc:creator>
            <pubDate>Tue, 04 Aug 2020 10:53:28 GMT</pubDate>
            <atom:updated>2020-08-04T12:49:27.706Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZZ4OUd5qJeodR-Z2Dh0uAg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@tengyart?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Tengyart</a> on <a href="https://unsplash.com/s/photos/emotions?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h4>A look into a time distributed deep bimodal approach to predict scores for the Big-5 Personality traits based on videos from the First Impression Challenge on Google Colab.</h4><h4>Videos are the New First Impressions!</h4><p>Think about the approximate number of video calls you have been a part of since March, 2020. Now, compare it to the number of video calls you were a part of before that. I am sure the difference is <strong>huge</strong> for most of us. Meetings with family, friends, and colleagues have shifted to video calls.</p><p>Video calling has also made it possible for us to keep expanding our networks and meet new people while maintaining social distancing. Hence, it is not wrong to say that, we are making quite a few personal as well as professional first impressions over videos. The personality perception through first impressions can be quite subjective and even lead to <em>first impression bias</em>. Of course, there are self-reported personality assessment tests but they can often suffer from a <em>social desirability bias</em>. And this gives us an opportunity to leverage AI to find a more objective approach to apparent personality analysis.</p><p>Keeping this in mind, the aim of this blog post is to show one such deep learning approach which uses videos to predict the scores for the Big-5 personality traits.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/756/1*OFtDPDqqeFBQnyAss4Exjg.png" /><figcaption>Created by Author</figcaption></figure><h4>What are the Big-5 Personality Traits?</h4><p>Most<a href="https://www.verywellmind.com/the-big-five-personality-dimensions-2795422"> contemporary psychologists</a> believe that there are 5 core dimensions to personality: <strong>Extraversion</strong>, <strong>Agreeableness</strong>, <strong>Openness</strong>, <strong>Conscientiousness</strong>, and <strong>Neuroticism; </strong>often referred by the acronym <strong>OCEAN. </strong><a href="https://www.simplypsychology.org/big-five-personality.html#:~:text=The%20Big%20Five%20personality%20traits%20are%20extraversion%20(also%20often%20spelled,throughout%20most%20of%20one&#39;s%20lifetime.">Unlike many of its previous counterparts </a>which believe in the binary aspect of personality traits, the <strong>Big-5 Personality trait theory asserts that each personality trait is a</strong> <strong><em>spectrum</em></strong>.</p><p><strong>Let’s look at how each trait is characterized followed by a map of how some popular fictional characters would score on the Big-5 personality traits…</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IHOCYYtaLmftdfBMezwBfQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I6wv9Ci0-FccWn6_RPkCnw.png" /><figcaption>Created by Author using Character Images from their Wiki Profiles</figcaption></figure><p>An interesting aspect of the Big-5 personality trait theory is that, these<em> traits are independent but not mutually exclusive</em>. For example, we can see in the above image that Sheldon Cooper (The Big Bang Theory) would score low on Extraversion but would also score high on Neuroticism, Phoebe Buffay (Friends) would score low on conscientiousness but score high on openness and so on…</p><h4>About the Data Set</h4><p>The <a href="http://chalearnlap.cvc.uab.es/dataset/24/description/">First Impressions Challenge</a> provides a data set of 10k clips from 3k YouTube Videos. The aim of this challenge was to understand how a deep learning approach can be used to infer apparent personality traits from videos of subjects speaking in front of the camera.</p><p>The training set comprised of 6k videos. The validation and test sets had 2k videos each. The average duration of the videos was 15 seconds. The ground truth labels for each video, consisted of five scores representing performance on each of the Big-5 personality traits. These scores were between 0 and 1. The labeling was done by Amazon Mechanical Turk Workers. More information about the challenge and the data set can be found in this <a href="https://arxiv.org/pdf/1802.00745.pdf">paper</a>.</p><p>Video data is unstructured but rich with multimedia features. The approach explained in this blog post uses audio and visual features from the videos. The analysis and modeling was done on Google Colab. The code can be accessed on <a href="https://github.com/MS1997/Apparent-personality-analysis-using-videos">Github</a>.</p><h4>Distribution of the Ground Truth Labels</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/394/1*iS7IJ3Cw1-zaJPVSZwK_kQ.png" /><figcaption>Created by Author</figcaption></figure><p>The graph on the left shows the distributions of personality scores in the training data set. It’s interesting to note that the distributions of the scores are quite similar and even symmetric along the mean. The reason for this symmetry could be that the scores aren’t self reported. Self-reported personality assessment scores are usually skewed due to <em>social desirability bias</em>.</p><h4>Extracting Visual Features</h4><p>Videos consists of image frames. These frames were extracted from videos using <em>OpenCV</em>. In apparent personality analysis, visual features include facial cues, movement of hands, posture of the person, etc. Since the data set consisted of videos with an average duration of 15 seconds, from each video 15 random frames were extracted. Each extracted frame was then resized to 150 X 150 and scaled by a factor of 1/255.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*si3YSY1Q4Eaz4mdOIlZU3Q.png" /><figcaption>Created by Author using a Video from the First Impressions Challenge</figcaption></figure><h4><strong>Extracting Audio Features</strong></h4><p>The waveform audio was extracted from each video using <em>ffmpeg </em>subprocess. An open source toolkit, <em>pyAudioAnalysis </em>was used to extract audio features from 15 non overlapping frames (keeping frame step equal to the frame length in the audioAnalysis subprocess). These included 34 features along with their delta features. The output was 1 X 68 dimensional vector for each frame or a 15 X 68 dimensional tensor for 15 audio frames.</p><p>The types of features extracted through pyAudioAnalysis include Zero crossing rate, Chroma Vector,Chroma Deviation, MFCCs, Energy, Entropy of Energy, Spectral Centroid, Spectral spread, Spectral entropy, Spectral Flux and Spectral Rolloff.</p><h4><em>Deep Bimodal Regression Model</em></h4><p>The Functional API of Keras with Tensorflow as backend was used for defining the model. The model was defined in two phases. In the first phase the image and audio features were extracted and then, the sequential features of the videos were processed. To process audio and visual features a bimodal time distributed approach was taken in the first phase.</p><p>Keras has a time distributed layer which can be used to apply the same layer individually to multiple inputs, resulting in a “many to many” mapping. Simply put, the time distributed wrapper enables any layer to extract features from each frame or time step separately. <em>The result:</em> an additional temporal dimension in the input and the output, representing the index of the time step.</p><p>The audio features extracted via pyAudioAnalysis were passed through a dense layer with 32 units in a time distributed wrapper. Hence, the same dense layer was applied to 1 X 68 dimensional vectors of each audio frame. Similarly, each image frame was passed in <em>parallel </em>through a series of convolutional blocks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/1*ImZ03P9vhhFGBXQQ8R7JRA.png" /><figcaption>Created by Author</figcaption></figure><p>After this step the audio and visual models were concatenated. To process the chronological or temporal aspect of videos, the concatenated outputs were further passed to a stacked LSTM model with a dropout and recurrent dropout rate of 0.2. The output of the stacked LSTM was passed to a dense layer with ReLU activation and dropout rate of 0.5. The final dense layer had 5 output units (one for each personality trait), along with sigmoid activation to get predicted scores between 0 and 1.</p><h4>Generator Function</h4><p>The biggest challenge was managing the limited memory resources. This was accomplished using <em>mini batch gradient descent</em>. To implement it a custom generator function was defined as follows:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d6cc502ba99c92e6ec9a45dcc71703fd/href">https://medium.com/media/d6cc502ba99c92e6ec9a45dcc71703fd/href</a></iframe><p><em>Note:</em> The generator function yields the input for the audio and visual models in one list. Corresponding to this the model is defined by passing a list of two inputs to the Model class of keras:</p><pre>model = Model([input_img,input_aud],output) </pre><h4>Results</h4><p>The model was compiled using the Adam optimizer with a learning rate of 0.00001. The model was trained for 20 epochs with a mini batch size of 8. Mean squared error was taken as the loss function. A custom metric called Mean accuracy was defined to see the performance of the model. It was calculated as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/0*3FKHYuw1dB_pA_0s" /><figcaption><strong>Here N is the number of input videos.</strong></figcaption></figure><p>Overall the model performed quite well with a final test mean accuracy of 0.9047.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/398/0*KEql-HhxeQ093tL2" /><figcaption>Created by Author</figcaption></figure><p>The table below shows the test mean accuracy for each of the Big-5 personality traits. The model shows similar performance for all 5 personality traits.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bsyK0Zr2bYBd8s9njgjmxw.png" /><figcaption>Created by Author</figcaption></figure><h4>The Road ahead…</h4><p>The results of the model can be further improved by increasing the frame sizes and lengths depending upon the availability of processing power. NLP analysis of video transcriptions can also be used to get additional features.</p><p>While automated apparent personality analysis has important use cases, it should be made sure that,<em> algorithmic bias</em> does not affect results. The aim of such AI applications is to provide a more objective approach. However, such objectivity can only be achieved if bias is excluded at each stage i.e. from data collection to results interpretation.</p><h4>References:</h4><p>[1] T. Giannakopoulos, pyAudioAnalysis(2015), <a href="https://github.com/tyiannak/pyAudioAnalysis">https://github.com/tyiannak/pyAudioAnalysis</a></p><p>[2] A. Subramaniam, V. Patel, A. Mishra, P. Balasubramanian, A. Mittal, <a href="https://arxiv.org/abs/1610.10048">Bi-modal First Impressions Recognition using Temporally Ordered Deep Audio and Stochastic Visual Features</a> (2016)</p><p>[3] C. Zhang, H. Zhang, X.Wei, J. Wu, <a href="https://cs.nju.edu.cn/wujx/paper/eccvw16_APA.pdf">Deep Bimodal Regression for Apparent Personality Analysis</a> (2016)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d8423f6d3cb3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/can-your-video-be-used-to-detect-your-personality-d8423f6d3cb3">How can Videos be Used to Detect Your Personality?</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Balancing the Regularization Effect of Data Augmentation]]></title>
            <link>https://medium.com/data-science/balancing-the-regularization-effect-of-data-augmentation-eb551be48374?source=rss-127625b11501------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb551be48374</guid>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[data-augmentation]]></category>
            <category><![CDATA[image-segmentation]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[keras]]></category>
            <dc:creator><![CDATA[Metika Sikka]]></dc:creator>
            <pubDate>Tue, 21 Jul 2020 20:44:47 GMT</pubDate>
            <atom:updated>2020-07-21T23:20:20.381Z</atom:updated>
            <content:encoded><![CDATA[<h4>A look into the need to balance overfitting and underfitting with data augmentation using an application of Image segmentation on satellite images to identify water bodies.</h4><h4>The Effect of Data Augmentation</h4><p>When training neural networks, data augmentation is one of the most commonly used pre-processing techniques. The word <strong>“augmentation” </strong>which literally means <em>“the action or process of making or becoming greater in size or amount”, </em>summarizes the outcome of this technique. But another important effect is that <em>it increases or augments the diversity of the data. </em>The increased diversity means, at each training stage the model comes across a different version of the original data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*geA3ZUMBUa67mBBhGUknmw.png" /><figcaption>By Author</figcaption></figure><p><strong><em>Why do we need this ‘increased diversity’ in data? </em></strong>The answer lies in the core tenet of machine learning — <strong>The Bias-Variance tradeoff</strong>. More complex models like deep neural networks have low bias but suffer from high variance. This implies that, these models overfit the training data and would show poor perform on test data or the data, they haven’t seen before. This would lead to <em>higher</em> <em>prediction errors</em>. Thus, the increased diversity from data augmentation reduces the variance of the model by making it better at <strong><em>generalizing</em></strong>.</p><p>For images, some common methods of data augmentation are taking cropped portions, zooming in/out, rotating along the axis, vertical/horizontal flips, adjusting the brightness and sheer intensity. Data augmentation for audio data involves adding noise, changing speed and pitch.</p><p>While data augmentation prevents the model from overfitting, some <strong><em>augmentation combinations can actually lead to underfitting</em></strong>. This slows down training which leads to a huge strain on resources like available processing time, GPU quotas, etc. Moreover, the model isn’t able to learn as much information to give accurate predictions which, again leads to high prediction errors. In this blog post we take the example of semantic segmentation on satellite images, to see the impact of different combinations of data augmentations on training.</p><h4>About the Data set</h4><p>This <a href="https://www.kaggle.com/franciscoescobar/satellite-images-of-water-bodies">Kaggle data set</a> gives the<strong> satellite images from Sentinel 2 and their corresponding masks which segment the water bodies</strong>. The masks have been calculated using the Normalized Difference Water Index or NDWI. Out of a total 2841 images in the data set, 2560 were extracted for the train set, 256 for the validation set and 25 for the test set respectively. The entire analysis and modeling was done on Google Colab with GPU support.</p><h4>Structure of U-NET</h4><p>Simply put, a U-NET is an autoencoder with residual or skip connections from each convolutional block in the encoder to its counterpart in the decoder. This results in a symmetric ‘U’ like structure. <a href="https://towardsdatascience.com/unet-line-by-line-explanation-9b191c76baf5">This article</a> gives a comprehensive line by line explanation of the structure of a U-NET from the original paper.</p><p>We use a slightly modified version of the U-NET as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xatr7T6jWtm_gmsctH6liQ.png" /><figcaption>Snapshot of a block of the UNET used (By Author)</figcaption></figure><h4>A Look at Different Cases of Data Augmentation</h4><p>We explore 5 different cases of data augmentation with the help of Keras ImageDataGenerator. We want to see how augmentation can lead to <em>overfitting or underfitting</em> during training. Thus, for comparison of the 5 cases, <em>Accuracy</em> and <em>Loss </em>during training &amp; validation were used; where binary cross-entropy was taken as the loss function.</p><p><strong>When dealing with semantic segmentation, an important point to remember is to apply the <em>same </em>augmentations to the images and their corresponding masks!</strong></p><p>In all 5 cases, the pixel values of the images and masks were rescaled by a factor of 1/255. All images and their masks in the validation and test set were also rescaled.</p><p><strong>Case 1: </strong>This was the <em>base case</em>. Only the pixel values of images and their masks were rescaled. No augmentations were applied. <em>This case produced a training set with the least variance</em>.</p><p><strong>Case 2: </strong>In addition to rescaling, the images and their masks were randomly flipped vertically or horizontally .</p><p><strong>Case 3: </strong>For this case,<strong> </strong>rescaling, random vertical or horizontal flips and random rotations between [-20,20] degrees were applied to images and their corresponding masks.</p><p><strong>Case 4: </strong>The images and their corresponding masks were randomly shifted along the width and height by a factor of 0.3.</p><p><strong>Case 5: </strong>The sheer transformations were randomly applied to the images and their corresponding masks using a factor of 20. They were also randomly zoomed in between the range [0.2,0.5].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3ApZlJyoX39DIwOyUZwanA.png" /><figcaption>Different types of Augmentations on the same image and it’s mask (By Author)</figcaption></figure><h4>A Comparison of Results</h4><p>Across all 5 cases, the model was trained for 250 epochs with a batch size of 16. The Adam optimizer was used with a learning rate of 0.00001, beta 1 of 0.99 and beta 2 of 0.99. In the below graphs, we see that, each case of data augmentation gave a varying performance for the same model, trained for the same number of epochs with the same initial state of the optimizer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*unufCdiF4gwuGjDET-K_Ug.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/1*AqZAZ65eQwT2xNZqL220Kw.png" /><figcaption>Plots of Training history (By Author)</figcaption></figure><p>The base case gave the best performance in terms of both training accuracy and loss. However, this could imply overfitting. On the other hand the perfomance of the third, fourth and fifth case was worse. The training accuracy for these three cases did not go above 80% and the training loss did not fall below 0.1. This could mean underfitting. The second case in which data augmentation consisted of randomly flipping the images and their corresponding masks, seems to show a balanced performance. Even though it performed a bit worse than the base case, it had a much higher accuracy and lower loss than the last three cases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/483/1*XLhXwDGgtHCx3vGQKH8bHw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/511/1*CqCiCv5v8LjoCjaNFJ9iaw.png" /><figcaption>Plots of Validation history (By Author)</figcaption></figure><p>The validation accuracy and loss, showed a similar trend. The last case performed the worst for both these metrics. Looking closer we see that, the cases 3 and 4 showed similar performance to training in terms of loss but a slightly higher validation accuracy than training. This signals underfitting. For the base case, the validation loss and accuracy were much worse than their training counterparts which again points towards overfitting. While we see very high fluctuations in validation accuracy and loss for the second case, on an average it seems to perform the best out of the five.</p><p>From the four graphs we can also see that each case of data augmentation converged at a different point during training for the same number of epochs and same initial state of the optimizer. It is almost like <strong><em>each case followed a separate trajectory</em></strong>.</p><p>The maximum accuracy and the minimum loss for Training and Validation are shown in the table below. The table supports the conclusion from the graphs. The second case seems to balance overfitting and underfitting during training quite well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sOdl-1vUNPnq2L6yrLmkRg.png" /><figcaption>By Author</figcaption></figure><h4><strong>Predictions on Test images for the 5 Cases</strong></h4><p>We see predicted segmentation of water bodies in satellite images in each of the five cases using 3 test images. It is again worth noting that,<em> the same model trained using 5 different data augmentation combinations, predicted 5 different images for each test image.</em></p><p>It should be noted that, the test images were most similar in terms format to the base case since, <em>the base cases used no augmentations</em>. Hence, for all three images, the base case model was able to segment the overall shape of the water body. However, case 2 was able to capture the minute edges as well. This confirms our conclusion from the graphs that, the base case model slightly over fit the training data (due to least variance).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZS-K7zehKhuHF5X8ZlJqFQ.png" /><figcaption>By Author</figcaption></figure><p>The performance of Case 3 and 4 varied with each test image. Case 5 gave the worst prediction for all three images (especially second and third images). This could imply that, for training this model on such a data set, changing the sheer intensity or zooming in too much can lead to poor prediction results. And this can be attributed to the fact that the model under fit the training data as it had, a <em>very high variance</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WJO7-X5458548d7D8R0Rog.png" /><figcaption>By Author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UXsulxqu0ERE3LHK32hsoA.png" /><figcaption>By Author</figcaption></figure><p>In the end, it again <strong><em>comes back to the core tenet — balancing the bias and variance</em></strong>. While data augmentation does have an explicit regularization effect, exploiting it can actually lead to the model not learning enough resulting in poor prediction results. Thus, we can see that there is a need to try out different combinations of data augmentation to find the most appropriate one for the data set of the problem statement.</p><p><strong><em>The entire code can be found on </em></strong><a href="https://github.com/MS1997/Semantic-Segmentation-of-Water-Bodies"><strong><em>Github</em></strong></a><strong><em>!</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb551be48374" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/balancing-the-regularization-effect-of-data-augmentation-eb551be48374">Balancing the Regularization Effect of Data Augmentation</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Skip Connections To Enhance Denoising Autoencoder Algorithms]]></title>
            <link>https://medium.com/data-science/using-skip-connections-to-enhance-denoising-autoencoder-algorithms-849e049c0ac9?source=rss-127625b11501------2</link>
            <guid isPermaLink="false">https://medium.com/p/849e049c0ac9</guid>
            <category><![CDATA[image-denoising]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[autoencoder]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Metika Sikka]]></dc:creator>
            <pubDate>Mon, 01 Jun 2020 19:41:19 GMT</pubDate>
            <atom:updated>2020-06-02T07:02:21.278Z</atom:updated>
            <content:encoded><![CDATA[<h4>Comparing the denoising performance of Autoencoders with residual networks across the bottleneck to those without on a sample of RGB images from Flickr.</h4><h3>Autoencoders:</h3><p>The official <a href="https://blog.keras.io/building-autoencoders-in-keras.html">Keras</a> blog, calls autoencoders an example of ‘<strong>self-supervised</strong>’ algorithms as their<strong> targets are</strong> <strong><em>generated</em></strong> <strong>from the input data</strong>. Hence, they are used for tasks of image reconstruction.</p><p>The main parts of an autoencoder are: <strong><em>Encoder, Bottleneck </em>and<em> Decoder</em></strong>. The Encoder is extracts image features at each step and in the process compresses the input data. The bottleneck constrains the input to its lowest dimensions known as compressed representations of the input data. The Decoder comes after this bottleneck and is used to reconstruct the input data. A loss measure is used to compare how close the generated or reconstructed date is to the input data.</p><h3>Denoising of Images:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*hDDcgrg1NfL6aVxSjRIDoQ.jpeg" /><figcaption>source: <a href="https://giphy.com/gifs/color-colour-FkUyGd7FDh1gk">GIPHY</a></figcaption></figure><p>Data denoising images is a common application of autoencoders. Noise in images can be understood as a <em>random variation in color or brightness of images</em>, degrading their quality. Removing this noise is often a pre-processing step in various use-cases of image data. Convolutional autoencoders can be used for this purpose. The encoder learns to extract the features separating them from the noise in the image. Thereby compressing the image. The final compressed representations from the bottleneck are passed over to the decoder. The decoder finally decompresses the image minimizing the noise.</p><h4>Skip connections from the encoder to decoder</h4><p>We know that deep neural networks suffer from<em> </em>the<em> </em><a href="https://arxiv.org/abs/1512.03385"><em>degradation problem</em></a>. Since Autoencoders have multiple convolutional and deconvolutional layers, they also suffer in performance when reconstructing images due to this information loss. Residual networks comprising of skip connections are a known solution to this problem. Hence, to improve the performance of the autoencoders such ‘skip connections’ can be added from the encoder to the decoder, i.e. <strong><em>across the bottleneck</em></strong>. These additional connections can directly send the feature maps from the an earlier layer of the encoder to a later layer of the decoder. This helps the decoder form more clearly defined decompressions of the input image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lp_C2i-wq-wja5GMQdOBhA.png" /><figcaption>Residual networks across the bottleneck, Source: By Author</figcaption></figure><p>To see the advantage of these residual networks, let’s see the activation outputs at various stages of a convolutional autoencoder model. First we trained an autoencoder model without skip-connections across the bottleneck. Then we added them and train the model again on the same sample of images.</p><h3>The Data set</h3><p>A sample of 10K RGB images were taken from <a href="https://www.kaggle.com/hsankesara/flickr-image-dataset">this</a> data set of Flickr images on Kaggle. All the analysis was done using Google Colab’s GPU.</p><h3>The Model</h3><p>We have used the Functional API of Keras to form the autoencoder model. The input images were resized to a smaller size (128,128,3) because of computational limitations. The image pixels were also standardized by a factor of 1/255. Further, the images were corrupted on purpose by applying a Gaussian noise matrix. These corrupted images formed the input of the autoencoder whereas the original images were used as targets while training the model. When building convolutional networks it is important to remember that, as we go deeper, the number of channels or filters increases whereas the size (height and width) of the input decreases.</p><pre><strong>#Input </strong><br>input_img = Input(shape=(128, 128, 3))</pre><pre><strong>#Encoder </strong><br>y = Conv2D(32, (3, 3), padding=&#39;same&#39;,strides =(2,2))(input_img)<br>y = LeakyReLU()(y)<br>y = Conv2D(64, (3, 3), padding=&#39;same&#39;,strides =(2,2))(y)<br>y = LeakyReLU()(y)<br>y1 = Conv2D(128, (3, 3), padding=&#39;same&#39;,strides =(2,2))(y) <strong># skip-1</strong><br>y = LeakyReLU()(y1)<br>y = Conv2D(256, (3, 3), padding=&#39;same&#39;,strides =(2,2))(y)<br>y = LeakyReLU()(y)<br>y2 = Conv2D(256, (3, 3), padding=&#39;same&#39;,strides =(2,2))(y)<strong># skip-2</strong><br>y = LeakyReLU()(y2)<br>y = Conv2D(512, (3, 3), padding=&#39;same&#39;,strides =(2,2))(y)<br>y = LeakyReLU()(y)<br>y = Conv2D(1024, (3, 3), padding=&#39;same&#39;,strides =(2,2))(y)<br>y = LeakyReLU()(y)</pre><pre><strong>#Flattening for the bottleneck</strong><br>vol = y.shape<br>x = Flatten()(y)<br>latent = Dense(128, activation=&#39;relu&#39;)(x) </pre><p>Below is code for the decoder. Transposed convolution or deconvolutional layers were used to build this decoder. <a href="https://machinelearningmastery.com/upsampling-and-transpose-convolution-layers-for-generative-adversarial-networks/">Deconvolutional layers </a>work roughly like a combination of convolutional and upsampling layers. Initially, we trained the model without the first and second skip connections. Then these connections were added from an earlier layer of the encoder to the later layer of the decoder using Add() from the layers API in Keras. The lrelu_bn() helper function was used to apply activations to these additions and pass the result through a batch normalization layer.</p><pre><strong># Helper function to apply activation and batch normalization to the # output added with output of residual connection from the encoder</strong></pre><pre>def lrelu_bn(inputs):<br>   lrelu = LeakyReLU()(inputs)<br>   bn = BatchNormalization()(lrelu)<br>   return bn</pre><pre><strong>#Decoder</strong><br>y = Dense(np.prod(vol[1:]), activation=&#39;relu&#39;)(latent)<br>y = Reshape((vol[1], vol[2], vol[3]))(y)<br>y = Conv2DTranspose(1024, (3,3), padding=&#39;same&#39;)(y)<br>y = LeakyReLU()(y)<br>y = Conv2DTranspose(512, (3,3), padding=&#39;same&#39;,strides=(2,2))(y)<br>y = LeakyReLU()(y)<br>y = Conv2DTranspose(256, (3,3), padding=&#39;same&#39;,strides=(2,2))(y)<br>y= Add()([y2, y]) <strong># second skip connection added here</strong><br>y = lrelu_bn(y)<br>y = Conv2DTranspose(256, (3,3), padding=&#39;same&#39;,strides=(2,2))(y)<br>y = LeakyReLU()(y)<br>y = Conv2DTranspose(128, (3,3), padding=&#39;same&#39;,strides=(2,2))(y)<br>y= Add()([y1, y]) <strong># first skip connection added here</strong><br>y = lrelu_bn(y)<br>y = Conv2DTranspose(64, (3,3), padding=&#39;same&#39;,strides=(2,2))(y)<br>y = LeakyReLU()(y)<br>y = Conv2DTranspose(32, (3,3), padding=&#39;same&#39;,strides=(2,2))(y)<br>y = LeakyReLU()(y)<br>y = Conv2DTranspose(3, (3,3), activation=&#39;sigmoid&#39;, padding=&#39;same&#39;,strides=(2,2))(y)</pre><p>Since the input images were standardized, their resulting pixel values were between 0 and 1. To get a comparable reconstructed image, “sigmoid” activation was used in the final layer. Out of the training set, 1k images were used as the validation set. Binary cross entropy was used as the loss function. Finally, both models were trained for 200 epochs with a mini-batch size of 32. The Adam optimizer with a learning rate of 0.001 gave the least training and validation loss at convergence.</p><p><strong><em>Let’s look at the activation outputs at various layers of the model for a test image. This will help us clearly see the encoder-decoder in action!</em></strong></p><p>The activation outputs were created using the <a href="https://github.com/philipperemy/keract">keract</a> package.</p><h4>The Test Image:</h4><p>We take a test <a href="https://www.worldsciencefestival.com/venues/columbia-university/">image</a> of the Columbia University campus. The test image was resized for the model and random noise was added. The ‘noisy’ image was used as the input for the model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/257/0*fRE4U7lplBGyy82H" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/257/0*6NaPTmWi83ISEwKO" /><figcaption>Original resized image (left), Image with Random Noise added used as the input for the models.(Right)</figcaption></figure><h4>Activation outputs of the encoder</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XoMMB3hczjX_mbyr" /><figcaption>Activation outputs of the First convolutional layer</figcaption></figure><p>Since the encoder was same for both the models, the activation outputs were also the same for this part. On examining the activation outputs of the first three convolutional layers, it can be seen that most of the image information is retained. This could also imply that the noise in the image is also retained at this stage of the model. However, as we go deeper in the model (see activation outputs of layers after the third convolutional layer), the information retained is quite abstract. The model starts to extract higher level features such as borders, corners &amp; angles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e6OGwCCL4kS7Rk4jVOZNrA.png" /><figcaption>Activation outputs of the Second and Third convolutional layers</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IiB9P5xI5RIkZoxm" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j94rVNQMWySOM8q9" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*p9M7HQ0acp1okLV6" /><figcaption>‘Abstract’ Activation outputs of the Fourth, Fifth and Sixth convolutional layers</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yu5pvyArsROQZBBh3Z6qcw.png" /><figcaption>Activation outputs of the Seventh convolutional layer and the Compressed representation of the input image</figcaption></figure><h4>Activation outputs of the decoder (model with no skip connections across the bottleneck)</h4><p>It is interesting to note that the activation outputs of the decoder are in the exact opposite sequence as the encoder above.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NMf8GwF22qu72rUvUwNOBQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l5KkSGnVnW3IiU1TexGybA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vM_x1M3AHvFKOdcP8MhfMQ.png" /><figcaption>Activation outputs of the First, Second and Third deconvolutional layers</figcaption></figure><p>After the third de-convolutional layer, we still do not see any edges of the image being formed again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bJXu7IXG505xUbfp1lmLKw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pYCZt41dc67Fd1-H0YWlAA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MliILpXD38Bo_DdGRp5EQQ.png" /><figcaption>Activation outputs of the Fourth, Fifth and Sixth deconvolutional layers</figcaption></figure><p>There is no clearly defined decompression of the image in the final layer of the decoder as seen below!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bVSCHhB5O75Bb-5bf3udTQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sk3fDmUYSSZYeB0Mf4jNPQ.png" /><figcaption>Activation outputs of the Seventh deconvolutional layer and the Final layer</figcaption></figure><h4>Activation outputs of the decoder (model added with skip connections across the bottleneck)</h4><p>As seen in the activation outputs of the encoder, its earlier layers retain the noise but the later layers extract higher representations of the image. Hence, skip connections are made from the third and fifth convolutional layers of the encoder to third and fifth deconvolutional layers of the decoder. The activation outputs of the first, second and third de-convolutional layers for this decoder are the same as the previous version of the decoder. However, after the third layer we see a difference in the activation outputs from those of the previous decoder. Below we can clearly see the image being formed again in the activation output of the sixth deconvolutional layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EShc9qKbu88Kj-GlXOxGLA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5Ryq3IxGjo5Qs_bggCqdgA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mpV1eT6uAzGFblwY9I-Ovg.png" /><figcaption>Activation outputs of the Fourth, Fifth and Sixth deconvolutional layers</figcaption></figure><p>The final layer of this decoder has given a clearly defined decompressed version of the test input image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CQLW5qK9Lz0XJWbLDhmvWA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vkqmwX2ar-lDSBjLy5Jjmg.png" /><figcaption>Activation outputs of the Seventh deconvolutional layer and the Final layer</figcaption></figure><h3>A comparison of results</h3><p>Denoised image output from the first model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/257/0*rkl0l4AE5ih59gO8" /></figure><p>Denoised image output of the second model with skip connections across the bottleneck:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/257/0*Upux_1b3NPR_QKbv" /></figure><p>We can clearly see the better performance the autoencoder model with skip connections across the bottleneck!</p><h3>The Road Ahead…</h3><p>This approach can be used for any image reconstruction application of autoencoders apart from denoising images. While the skip connections improve the performance of the autoencoder, the positions and number of these connections can be experimented with. The model can also be trained with different levels of noise factor for better generalization of results. The full code can be accessed <a href="https://github.com/MS1997/Autoencoders-with-skip-connections">here</a>!</p><h4>References:</h4><p>J. Dong , XJ. Mao , C. Shen , YB. Yang, <a href="https://arxiv.org/pdf/1611.09119.pdf">Learning Deep Representations Using Convolutional Auto-encoders with Symmetric Skip Connections</a> (2017)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=849e049c0ac9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/using-skip-connections-to-enhance-denoising-autoencoder-algorithms-849e049c0ac9">Using Skip Connections To Enhance Denoising Autoencoder Algorithms</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Training Networks to Identify X-rays with Pneumonia]]></title>
            <link>https://medium.com/data-science/training-networks-to-identify-x-rays-with-pneumonia-c98932e47da3?source=rss-127625b11501------2</link>
            <guid isPermaLink="false">https://medium.com/p/c98932e47da3</guid>
            <category><![CDATA[transfer-learning]]></category>
            <category><![CDATA[covid19]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[resnet]]></category>
            <dc:creator><![CDATA[Metika Sikka]]></dc:creator>
            <pubDate>Sat, 16 May 2020 22:03:35 GMT</pubDate>
            <atom:updated>2020-05-18T18:39:05.955Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*qVyYLRARzH_AVxCuJqNCAg.png" /><figcaption>Source: Photo by <a href="https://unsplash.com/@nci?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">NCI</a> on <a href="https://unsplash.com/s/photos/chest-x-ray?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h4>How Transfer Learning gives a head start with limited data and time</h4><p><strong><em>Note from the editors:</em></strong><em> </em><a href="http://towardsdatascience.com/"><em>Towards Data Science</em></a><em> is a Medium publication primarily based on the study of data science and machine learning. We are not health professionals or epidemiologists, and the opinions of this article should not be interpreted as professional advice. To learn more about the coronavirus pandemic, you can click </em><a href="https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports"><em>here</em></a><em>.</em></p><p>The year 2020 has witnessed the outbreak of the pandemic, COVID-19 which has brought the entire world to a standstill. The scientific community has been continuously working towards getting a medical breakthrough for a potential cure. It has become a race against the quick spread of this virus which is also why we are seeing the fastest-ever progressing clinical trials. Data scientists across the world have been aiding this process by harnessing data. But data collection on COVID-19 patients is an on-going process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*JDKDNSYkNmS4yeXIDji2mw.gif" /><figcaption>Source:<a href="https://giphy.com/gifs/troy-landry-GgcusW5RLS9Nu"> GIPHY</a>.</figcaption></figure><p>Limited time and data being a challenge today, transfer learning seems like a good solution. It will enable us to use models which have been pre-trained on data with similar structures. For example, using models which have been pre-trained on patients having similar diseases. It also gives us an opportunity to take advantage of the learning power of deep neural networks which if trained from the ground-up would require large amounts of data and computational resources.</p><h3><strong>About the Data</strong></h3><p>The data was sourced from <a href="https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia">Kaggle</a>. It contains chest X-ray images (anterior-posterior) selected from retrospective cohorts of pediatric patients of one to five years old from Guangzhou Women and Children’s Medical Center, Guangzhou. The goal was to classify these X-ray images as Normal or positive for Pneumonia.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5q1K5QjZNbssVrPs" /><figcaption>The dataset directory structure. Source: By Author.</figcaption></figure><p>The entire data was directly imported into Google Colaboratory using the Kaggle API. And all analysis was done on the same using GPU. The code can be found <a href="https://github.com/MS1997/X-ray-classification.git">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sR6GYWEcURdfm7P9" /><figcaption>X-rays of randomly chosen patients from the training set. Source: <a href="https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia">Kaggle</a>.</figcaption></figure><h3><strong>Understanding Residual Networks or ResNets</strong></h3><p>Residual Networks (ResNets) are convolution networks which were introduced as a <em>solution to the degradation problem</em> generally faced while using ‘plain’ convolution networks<em>.</em> ResNets use ‘skip’ connections to jump over layers of the deep networks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/561/1*XcQ8BH9UjIlkWekgpsmM6A.png" /><figcaption>ResNets are like Flyover connections. Source: Photo by <a href="https://unsplash.com/@jaredmurray?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Jared Murray</a> on <a href="https://unsplash.com/s/photos/multilevel-interchange?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>The skip connections can also be seen as identity mappings or functions which add the outputs of the previous layers to the later layers. In forward propagation ResNets work to push the outputs from the skipped subnetworks called ‘the residual’ to <em>zero</em>. This makes the true output almost equal to the output of the subnetwork from where the skip connection began thereby reducing the appreciable information loss due to a deeper architecture.</p><p>An <em>additional </em>benefit of ResNets is that, during backpropagation the skip connections also propagate the gradient flow. Skipping the layers with non-linear activation functions makes the initial gradient (from the higher layers) reach faster to the earlier layers which solves the problem of vanishing gradients.</p><p><strong>For this project, ResNet50 was implemented.</strong></p><blockquote><a href="https://arxiv.org/abs/1512.03385"><strong>The ResNet50 Architecture:</strong></a><strong> </strong>The ResNet50 is a 50-layer ResNet which uses a ‘bottleneck design’ for computational efficiency. The bottleneck design indicates that they use a stack of 3 layers. These 3 layers are, 1x1, 3x3, 1x1 convolutions. 1x1 convolutions on either side are used to decrease and then restore the dimensions. Thus, the 3x3 layers become like a bottleneck with smaller input &amp; output dimensions. The ResNet50 has more than 23 million trainable parameters. This network had been pre-trained on the ImageNet dataset consisting of 1000 classes, on a training dataset of 1.28 million images, validated on 50k images and tested on another 100k images.</blockquote><p><strong><em>‘SKIP’ to the modeling…</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/1*0zfhKMaH2TANPFUDnamBqg.gif" /><figcaption>Source: <a href="https://giphy.com/gifs/happy-crazy-homer-simpson-kEKcOWl8RMLde">GIPHY</a></figcaption></figure><h3>Adjusting the pre-trained model for our dataset</h3><p>The data presented a binary classification problem with the following two categories: <em>Normal </em>&amp; <em>Pneumonia</em>. Since, the original ResNet50 network was used to classify an image into one of the 1000 categories, to take advantage of the pre-trained architecture and it’s weights, the top part of this network was removed. Thus, the original fully connected layer was replaced with a Global average pooling layer followed by a fully connected layer dense layer and an output layer. Other combinations were tried but this gave the best test set performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bkBsrOjEDrkh8EofAII2zg.png" /><figcaption>Adjusted model architecture. Source: By Author.</figcaption></figure><h3><strong>Preprocessing for the Model</strong></h3><p><strong><em>Adding more channels: </em></strong>The Image data generator functions in Keras were used for preprocessing the images. The Chest X-ray images of the data are grayscale images, which consist of a single channel. Whereas the ResNet50 model was trained on RGB images of the ImageNet dataset, which have 3 channels. Using the <em>color_mode</em> argument of the generator functions, the grayscale images were converted into having 3 channels.</p><p><strong><em>More Image transformations: </em></strong>Further, the images in the training set were augmented using horizontal flip, zooming, height/width shift and shearing transformations. The ResNet50 preprocessing function was also applied on the augmented training images, original validation &amp; test images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/871/1*F2-I2ae2JFyCeAdhd0768Q.png" /><figcaption>Sample of 9 augmented images of the training set. Note the color change. Source: By Author.</figcaption></figure><p>With the above model the best test accuracy achieved was ~<strong>83%</strong>!</p><h3>Further Fine-tuning</h3><ul><li><strong>The debate of batch normalization in Keras: </strong>Some literature suggests that since the Batch Normalization layer in Keras works differently in training and inference phases, it can create discrepancies in accuracy metrics. Hence, the model was <a href="https://towardsdatascience.com/how-to-utilize-transfer-learning-to-classify-images-with-state-of-the-art-deep-learning-models-d8e5d5bb35d4">trained</a> by freezing all layers except the batch normalization layers of the ResNet50 base model. All the remaining unfrozen layers (batch normalization layers &amp; additional layers) were then trained.</li></ul><pre>base_model = ResNet50(weights=&#39;imagenet&#39;,include_top=False,input_shape=(150,150,3))</pre><pre>x = base_model.output #adding the resnet model</pre><pre># freezing all layers except the batch normalization layers<br>for layer in base_model.layers:<br>    if isinstance(layer, BatchNormalization):<br>       layer.trainable = True<br>    else:<br>        layer.trainable = False </pre><ul><li><strong>Tuning of hyperparameters: </strong>To improve the slow convergence in the initial model, different values of learning rate and beta_1 for the Adam optimizer were tried. A learning rate of 0.01 and beta_1 of 0.9 were chosen. Looking at the batch size different powers of 2 were tried. Batch size of 32 gave the best test result.</li><li><strong>Custom Callback function: </strong>Further, it was observed that the model gave best test accuracy when the validation loss during the train model went <a href="https://stackoverflow.com/questions/37293642/how-to-tell-keras-stop-training-based-on-loss-value">below</a> 0.1. To achieve this a custom callback function was created to train the model till the validation loss fell below 0.1 with a patience parameter of 2.</li></ul><pre>#early stopping with custom call back<br>class EarlyStoppingByLossVal(Callback):<br>    def __init__(self, monitor=[&#39;val_loss&#39;],patience=0, value=0.00001, verbose=0):<br>        super(Callback, self).__init__()<br>        self.monitor = monitor<br>        self.value = value<br>        self.verbose = verbose<br>        self.patience = patience</pre><pre>    def on_train_begin(self, logs=None):<br>    # the number of epoch the model has waited when loss is below  the required value<br>        self.wait = 0</pre><pre>    def on_epoch_end(self, epoch, logs={}):<br>        current = logs.get(self.monitor)<br>        if current is None:<br>            warnings.warn(&quot;Early stopping requires %s available!&quot; % self.monitor, RuntimeWarning)<br>        if current &lt; self.value:<br>            self.wait +=1<br>            if self.wait &gt;= self.patience:<br>                if self.verbose &gt; 0:<br>                    print(&quot;Epoch %05d: early stopping&quot; % epoch)<br>                    self.model.stop_training = True</pre><p>The best test accuracy achieved after fine-tuning was <strong>93.75%</strong>. The AUROC curve was <strong>0.98</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0OoJwXpEbqYezpnli9AnTQ.png" /><figcaption>Confusion Matrix and ROC curve for the best test set performance. Source: By Author.</figcaption></figure><h3><strong>Future scope</strong></h3><p>While, this model performed well on the test set, adding patient data representing varied regions and demographies, and further hyperparameter tuning could improve the model results. Using model pre-trained specifically on X-ray images such as <a href="https://stanfordmlgroup.github.io/projects/chexnet/">ChestXNet</a> could also give better results. <em>The purpose of this project was not to make any official claims, rather aid in future research. Any diagnosis should only be make by a medical professional!</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/260/1*5EcfgtzCwEi0JXiVT-XJBQ.gif" /><figcaption>Source: <a href="https://giphy.com/gifs/mr-gif-9Ai5dIk8xvBm0">GIPHY</a>.</figcaption></figure><h3>TL;DR</h3><p>According to the <a href="https://www.who.int/docs/default-source/coronaviruse/clinical-management-of-novel-cov.pdf">World Health Organization</a>,“the most common diagnosis in severe COVID-19 patients is severe pneumonia”. This article presents a use-case of transfer learning relevant for COVID-19. A pre-trained ResNet model has been used to classify X-rays of patients as ‘Normal’ or infected with Pneumonia.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c98932e47da3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/training-networks-to-identify-x-rays-with-pneumonia-c98932e47da3">Training Networks to Identify X-rays with Pneumonia</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>