<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by raghunandepu on Medium]]></title>
        <description><![CDATA[Stories by raghunandepu on Medium]]></description>
        <link>https://medium.com/@raghunandepu?source=rss-cf5fd33a1c1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*7q_8_Zh6cUiedVjIr7NIoA.png</url>
            <title>Stories by raghunandepu on Medium</title>
            <link>https://medium.com/@raghunandepu?source=rss-cf5fd33a1c1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 04:43:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@raghunandepu/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Santander Customer Transaction Prediction: An End to End Machine Learning project]]></title>
            <link>https://medium.com/analytics-vidhya/santander-customer-transaction-prediction-an-end-to-end-machine-learning-project-2cb763172f8a?source=rss-cf5fd33a1c1------2</link>
            <guid isPermaLink="false">https://medium.com/p/2cb763172f8a</guid>
            <category><![CDATA[santander]]></category>
            <category><![CDATA[transaction-prediction]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[an-end-to-end-ml-project]]></category>
            <dc:creator><![CDATA[raghunandepu]]></dc:creator>
            <pubDate>Mon, 11 Nov 2019 08:22:38 GMT</pubDate>
            <atom:updated>2019-11-18T13:53:11.536Z</atom:updated>
            <content:encoded><![CDATA[<h3>Santander Customer Transaction Prediction: An End to End Machine Learning Project</h3><h3>Can you identify who will make a transaction?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4vg0lW_YVZluXxzdGMfCIQ.png" /></figure><h3>Table of Contents:</h3><ul><li>Problem statement</li><li>Data Acquisition</li><li>Evaluation Metric</li><li>Data Description</li><li>Exploratory data analysis</li><li>Feature engineering</li><li>Solution 1: Light GBM Model with simple features</li><li>Solution 2: Feature Engineering &amp; LightGBM model with additional features</li><li>Conclusion</li><li>Procedure Followed</li><li>Model performance comparison</li></ul><h3>Problem Statement:</h3><p>In this challenge, Santander invites Kagglers to help them identify which customers will make a specific transaction in the future, irrespective of the amount of money transacted. The data provided for this competition has the same structure as the real data we have available to solve this problem.</p><p>The data is anonymous, each row containing 200 numerical values identified just with a number.</p><p>We will explore the data, prepare it for a model, train a model and predict the target value for the test set, then prepare a submission.</p><h3>Data Acquisition:</h3><p>This is from Kaggle competition. Download the dataset from the below source.</p><p><strong>Source:</strong> <a href="https://www.kaggle.com/c/santander-customer-transaction-prediction/overview">https://www.kaggle.com/c/santander-customer-transaction-prediction/overview</a></p><h3>Evaluation Metric:</h3><p>Submissions are evaluated on <strong>area under the ROC curve</strong> between the predicted probability and the observed target.</p><h4>Submission File</h4><p>For each Id in the test set, you must make a binary prediction of the target variable. The file should contain a header and have the following format:</p><p>ID_code,target<br>test_0,0 <br>test_1,1 <br>test_2,0 <br>etc.</p><h3>Data Description</h3><p>You are provided with an anonymized dataset containing numeric feature variables, the binary target column, and a string ID_code column.</p><p>The task is to predict the value of target column in the test set.</p><h4>File descriptions</h4><ul><li>train.csv — the training set.</li><li>test.csv — the test set. The test set contains some rows which are not included in scoring.</li></ul><h3>Exploratory data analysis</h3><p>Let’s load the train and test files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RkhpuBl9Oekub1Q5WDd4rw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m3w_gFJJGVLGDICbaTi0qw.png" /></figure><p>Both train and test dataframes have 200000 entries. Train has 202 columns and test has 201 columns.</p><p>Train contains:</p><ul><li>ID_code (string);</li><li>target;</li><li>200 numerical variables, named from var_0 to var_199;</li></ul><p>Test contains:</p><ul><li>ID_code (string);</li><li>200 numerical variables, named from var_0 to var_199;</li></ul><h3><strong>Check for missing data:</strong></h3><p>Let’s check if there is any missing data.</p><p>We defined a helper function missing_data and applied on train and test data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yPrITEuZgTEam5cWG5WbYw.png" /></figure><p>There is no missing data in both train and test datasets.</p><p>Let’s check the numerical values in train and test sets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QrAOPi1eJmOkCntJb6Bj5A.png" /></figure><p><strong>Observations:</strong></p><ul><li>standard deviation is relatively large for both train and test variable data</li><li>min, max, mean, sdt values for train and test data looks quite close</li><li>mean values are distributed over a large range</li></ul><p>Let’s check the distribution of target value in train data set.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*FHNskRImYuTkZ8aae9ERsA.png" /></figure><p><strong>Observation:</strong></p><p>We can see that the data is unbalanced. Majority values have 0 as target value.</p><p><strong>Distribution of Mean and Standard Deviation:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3UlPLDiRMgtMmNp7oT5neA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ncm8x9zMIjHOsM31EBvHeg.png" /></figure><h4>Distribution of skew and kurtosis:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*clhDs8lwxG5wclBDYKRcoA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*huyXMQQNHwljc3BVYkUNHA.png" /></figure><h4><strong>Correlation of Features:</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uPuQdfTKA4za83cZby5pPg.png" /></figure><p>The correlation between the features is very small.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2H_OLg8812roiDvHBK2k_w.png" /></figure><h3>Duplicate values:</h3><p>Let us check if there were any duplicate values in train and test data sets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qoagtGYaIgrKTn6t8NY0qg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KuT_XW6TxuN6VjBPik15nw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lB9ryqCruxjHOe6dZjXIQg.png" /></figure><p>We can see that there are duplicates and top 15 max of duplicate values in train and test sets are displayed above.</p><p><strong>Observation:</strong> Same columns in train and test have the same or very close number of duplicates.</p><h3>Feature engineering</h3><p>Let us create simple features like sum, min, max, mean, std, skew, kurt and median as below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JwjjoE6eb2LECEHI79utpQ.png" /></figure><p>Let’s check the newly created features:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iEoI_8X5M9b8SN9qepS9CA.png" /></figure><p>Let’s check the distribution of these newly engineered features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*utjJOY4lbGRR8EYfxRCpsA.png" /><figcaption>Distribution of new features in train data set</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*34onJDFwZCXj7J7Gm1FOLw.png" /><figcaption>Distribution of new features in test data set</figcaption></figure><p>Rounded features:</p><p>Let us create some rounded features as below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ovy-3bFvIQq9uqETVMPW3Q.png" /></figure><p>Let us see how these features look like.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*law2o6en1Qz3lSljPDrJ3g.png" /></figure><p>Let’s now build the machine learning model.</p><h3>Solution 1: Light GBM Model with simple features:</h3><blockquote><strong><em>What is Light GBM?</em></strong></blockquote><p>Light GBM is a gradient boosting framework that uses tree based learning algorithm.</p><blockquote><strong><em>How it differs from other tree based algorithm?</em></strong></blockquote><p><strong>Light GBM grows tree vertically </strong>while other algorithm grows trees horizontally which means that Light GBM grows tree <strong>leaf-wise </strong>while other algorithm grows level-wise. It will choose the leaf with max delta loss to grow. When growing the same leaf, Leaf-wise algorithm can reduce more loss than a level-wise algorithm.</p><p><strong>Hyperparameters of light GBM model:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p9mF4WBLspPu4lGYKrSzuA.png" /></figure><p>Let’s train the model with the above selected hyperparameters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BCmET1_wsUirsu_Nifz09w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KeM8R6637YRex3CeItoluw.png" /></figure><p><strong>Feature Importance:</strong></p><p>Some of the most important features that observed from this model are as below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e6P7KUlPddCCm4_vn_Amdg.png" /></figure><p><strong>Prediction on test data:</strong></p><p>Let us now predict the results on test data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a_4gQRPbHeJFQ8jl_w47Iw.png" /></figure><blockquote>RESULT : Leaderboard Score : AUC — 0.90014 (Rank 335 which is top 3.8%)</blockquote><h3>Solution 2: Feature Engineering &amp; LightGBM model</h3><h3>Feature Engineering and Data Augmentation</h3><h4>1. Separating the Real/Synthetic Test Data and Magic Features</h4><p>Using unique value count in a row to identify synthetic samples. If a row has at least one unique value in a feature, then it is real, otherwise it is synthetic. This technique is shared by YaG320 in this kernel — List of Fake Samples and Public/Private LB split(Reference: <a href="https://www.kaggle.com/yag320/list-of-fake-samples-and-public-private-lb-split">https://www.kaggle.com/yag320/list-of-fake-samples-and-public-private-lb-split</a>) and it successfuly identifies synthetic samples in test set. This way the unusual bumps on the distribution peaks of test set features are captured. The magic features are extracted from the combination of training set and real samples in the test set.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*22qoTChvpX7ZW1HshVGyDg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*II3kRi6VrFG7_9bKxOti_A.png" /></figure><h4>2. Data Augmentation</h4><p>Data augmentation means increasing the number of data points. Oversampling the data increases CV and leaderboard score significantly since the data is imbalanced. This oversampling technique is shared by Jiwei Liu in this kernel (Reference: <a href="https://www.kaggle.com/jiweiliu/lgb-2-leaves-augment">https://www.kaggle.com/jiweiliu/lgb-2-leaves-augment</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DbiKl8LHNz1Za_W-mNNp4Q.png" /></figure><p>Let us now try to implement these with LighGBM model.</p><h3>Model — LightGBM Gradient Boosting Decision Tree</h3><p>Hyperparameters of the model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/1*kgOPeg5HcvmExGsn2k0DUw.png" /></figure><p>Implementation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MgerUnywZWZPSISFXaW1Pg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dBTBash9wWzyAclok4ouoQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zbSdt4CJYf5Ek7H_1GDD3g.png" /></figure><p><strong>Feature Importance:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UEoIHfENkBwrAFCFrsJwUw.png" /></figure><p><strong>Prediction on test data and submission:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zFWhSM23-JjRf204mR4IaA.png" /></figure><blockquote>RESULT : Leaderboard Score : AUC — 0.92358 (Rank 19 which is top 1%)</blockquote><h3>Conclusion:</h3><h3>Procedure Followed</h3><ol><li>Performed Exploaratory data analysis on train and test datasets.</li><li>Performed checks for missing values and duplicate data.</li><li>Observed various plot of density, distribution of mean, skewness, kurtosis and both train and test and also both target values 1 and 0.</li><li>Observed if there was any correlation between the features</li><li>Performed feature engineering and created features</li><li>Applied machine learning model light Gradient boosting machine on these features and observed ROC of 0.90014.</li><li>To futher improve the model’s score, used unique value count in a row to identify synthetic samples and differentiated real and synthetic samples. Created magic features in both train and test sets using this data.</li><li>As a data augmentation step, performed data oversampling as the data is imbalanced.</li><li>Applied machine learning model lightGBM again with newly created magical features and it helped to improve ROC score to 0.92358.</li></ol><h3>Model performance comparison</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hm4PyqcgPno8WvEMmx6cyQ.png" /></figure><p>Light GBM with additional features improved model score to 0.92358 and this is pretty much close to kaggle’s leaderboard top scores. More can be definitely done.</p><p>Thanks for reading!</p><h3>REFERENCES:</h3><p><a href="https://www.kaggle.com/c/santander-customer-transaction-prediction/notebooks">https://www.kaggle.com/c/santander-customer-transaction-prediction/notebooks</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2cb763172f8a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/santander-customer-transaction-prediction-an-end-to-end-machine-learning-project-2cb763172f8a">Santander Customer Transaction Prediction: An End to End Machine Learning project</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding and implementation of Residual Networks(ResNets)]]></title>
            <link>https://medium.com/analytics-vidhya/understanding-and-implementation-of-residual-networks-resnets-b80f9a507b9c?source=rss-cf5fd33a1c1------2</link>
            <guid isPermaLink="false">https://medium.com/p/b80f9a507b9c</guid>
            <category><![CDATA[resnet]]></category>
            <category><![CDATA[deep-neural-networks]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[raghunandepu]]></dc:creator>
            <pubDate>Wed, 30 Oct 2019 18:47:32 GMT</pubDate>
            <atom:updated>2024-03-12T15:12:54.801Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Residual learning framework to ease the training of networks that are substantially deeper than those used previously.</em></p><p>This article is primarily based on research paper “<strong>Deep </strong><a href="https://www.analyticsvidhya.com/blog/2023/02/deep-residual-learning-for-image-recognition-resnet-explained/?utm_source=Backlink&amp;utm_medium=SEO"><strong>Residual Learning</strong></a><strong> for </strong><a href="https://www.analyticsvidhya.com/blog/2021/01/image-classification-using-convolutional-neural-networks-a-step-by-step-guide/?utm_source=Backlink&amp;utm_medium=SEO"><strong>Image Recognition</strong></a>” published by Microsoft Research. [<a href="https://arxiv.org/pdf/1512.03385.pdf">Link to the research paper</a>] and Convolutional Neural Network course by Andrew Ng.</p><h3><strong>Table of Contents:</strong></h3><ul><li>Introduction — The problem of very deep neural networks</li><li>Why is this happening and how could we fix it?</li><li>Solution: Residual learning framework</li><li>Type of blocks used in ResNet</li><li>Take aways from Residual block</li><li>Results from the research paper</li><li>Was ResNet successful?</li><li>Building your first ResNet model (50 layers)</li><li>Summary</li></ul><h3><strong>Introduction — The problem of very deep neural networks:</strong></h3><p>In recent years, neural networks have become deeper, with state-of-the-art networks going from just a few layers (e.g., AlexNet) to over a hundred layers.</p><ul><li>One of the major benefits of a very deep network is that it can represent very complex functions.</li><li>However, a huge barrier to training them is vanishing gradients: very deep networks often have a gradient signal that goes to zero quickly, thus making gradient descent prohibitively slow.</li><li>More specifically, during gradient descent, as we backprop from the final layer back to the first layer, we are multiplying by the weight matrix on each step. If the gradients are small, due to large number of multiplications, the gradient can decrease exponentially quickly to zero (or, in rare cases, grow exponentially quickly and “explode” to take very large values).</li></ul><p>The regular networks like VGG-16 are called “plain” networks.</p><p>In plain networks, as the number of layers increase from 20 to 56 (as shown below), even after thousands of iterations, training error was worse for a 56 layer compared to a 20 layer network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4fv-3pCDh2ibQK33oDVFdA.png" /><figcaption>Training error (left) and test error (right) on CIFAR-10 with 20-layer and 56-layer “plain” networks. The deeper network has higher training error, and thus test error.</figcaption></figure><p>In theory, we expect having a deeper network should only help but in reality, the deeper network has higher training error, and thus test error.</p><h3><strong>Why is this happening and how could we fix it?</strong></h3><p>When deeper networks are able to start converging, a <em>degradation </em>problem has been exposed: with the network depth increasing, accuracy gets saturated (which might be unsurprising) and then degrades rapidly.</p><p>Using deeper networks is <strong>degrading </strong>the performance of the model. Microsoft Research paper tries to solve this problem using <strong>Deep Residual learning framework.</strong></p><p><strong>Solution:</strong> Residual Block / Identity block</p><p>The idea is that instead of letting layers learn the underlying mapping, let the network fit the residual mapping. So, instead of say H(x), initial mapping<em>, </em>let the network fit, F(x) := H(x)-x which gives H(x) := F(x) + x</p><p>The approach is to add <em>a </em><strong><em>shortcut</em></strong><em> or a </em><strong><em>skip connection</em></strong> that allows information to flow, well just say, more easily from one layer to the next’s next layer, i.e., you bypass data along with normal CNN flow from one layer to the next layer after the immediate next.</p><p>A Residual Block:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*N5FPYoOt6OhC-xTGO5chFQ.png" /><figcaption>Residual learning: a building block</figcaption></figure><p><strong>Two take aways from residual block:</strong></p><ol><li>Adding additional / new layers would not hurt the model’s performance as regularisation will skip over them if those layers were not useful.</li><li>If the additional / new layers were useful, even with the presence of regularisation, the weights or kernels of the layers will be non-zero and model performance could increase slightly.</li></ol><p>Therefore, by adding new layers, because of the “Skip connection” / “residual connection”, it is guaranteed that performance of the model does not decrease but it could increase slightly.</p><p>By stacking these ResNet blocks on top of each other, you can form a very deep network. Having ResNet blocks with the shortcut also makes it very easy for one of the blocks to learn an identity function. This means that you can stack on additional ResNet blocks with little risk of harming training set performance.</p><p>Two main types of blocks are used in a ResNet, depending mainly on whether the input/output dimensions are same or different.</p><p>1<strong>.The identity block</strong> — same as the one we saw above. The identity block is the standard block used in ResNets and corresponds to the case <em>where the input activation has the same dimension as the output activation</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BCbJZXwGDtEdytj9ag_YWw.png" /><figcaption><strong>Identity block.</strong> Skip connection “skips over” 2 layers</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4e386427850cef05aafef06c1d1a7223/href">https://medium.com/media/4e386427850cef05aafef06c1d1a7223/href</a></iframe><p>2. <strong>The Convolutional block</strong> — We can use this type of block <em>when the input and output dimensions don’t match up</em>. The difference with the identity block is that there is a CONV2D layer in the shortcut path.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sb_4xKI_bRoX6jmZcNTRWw.png" /><figcaption>Convolutional block</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e1403d307a4725f5171a3d4820885083/href">https://medium.com/media/e1403d307a4725f5171a3d4820885083/href</a></iframe><h3><strong>Results from the paper:</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z9sHmRVTn0O2VSpL9qZkWQ.png" /><figcaption>Architectures for ImageNet. Building blocks are shown in brackets, with the numbers of blocks stacked.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oeORk3YzCxc3p9xvhB8r-w.png" /><figcaption>Training on ImageNet. Thin curves denote training error, and bold curves denote validation error of the center crops. Left: Plain networks of 18 and 34 layers. Right: ResNets of 18 and 34 layers.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*NijIT5KTI8AHnTJb1Y2sIA.png" /><figcaption>Top-1 error (%, 10-crop testing) on ImageNet validation</figcaption></figure><p>The 34 layer ResNet performed better than the 18 layer ResNet and plain counter part. So the degradation problem was addressed on deep ResNet better than the shallower network, both plain and ResNet one.</p><p>For deeper networks (50 and above) authors introduced bottleneck architectures for economical gains.</p><h3><strong>Was ResNet Successful? — </strong>Yes.</h3><ul><li>Won 1st place in the ILSVRC 2015 classification competition with top-5 error rate of 3.57% (An ensemble model)</li><li>Won the 1st place in ILSVRC and COCO 2015 competition in ImageNet Detection, ImageNet localization, Coco detection and Coco segmentation.</li><li>Replacing VGG-16 layers in Faster R-CNN with ResNet-101. They observed a relative improvements of 28%</li><li>Efficiently trained networks with 100 layers and 1000 layers also.</li></ul><h3>Building your first ResNet model (50 layers)</h3><p>You now have the necessary blocks to build a very deep ResNet. The following figure describes in detail the architecture of this neural network. “ID BLOCK” in the diagram stands for “Identity block,” and “ID BLOCK x3” means you should stack 3 identity blocks together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_XlZtWXLShYsprXVtTyjpQ.png" /></figure><p>The details of the above ResNet-50 model are:</p><ul><li>Zero-padding: pads the input with a pad of (3,3)</li><li>Stage 1: The 2D Convolution has 64 filters of shape (7,7) and uses a stride of (2,2). Its name is “conv1”. BatchNorm is applied to the channels axis of the input. MaxPooling uses a (3,3) window and a (2,2) stride.</li><li>Stage 2: The convolutional block uses three set of filters of size 64x64x256, f=3, s=1 and the block is “a”. The 2 identity blocks use three set of filters of size 64x64x256, f=3 and the blocks are “b” and “c”.</li><li>Stage 3: The convolutional block uses three set of filters of size 128x128x512, f=3, s=2 and the block is “a”. The 3 identity blocks use three set of filters of size 128x128x512, f=3 and the blocks are “b”, “c” and “d”.</li><li>Stage 4: The convolutional block uses three set of filters of size 256x256x1024, f=3, s=2 and the block is “a”. The 5 identity blocks use three set of filters of size 256x256x1024, f=3 and the blocks are “b”, “c”, “d”, “e” and “f”.</li><li>Stage 5: The convolutional block uses three set of filters of size 512x512x2048, f=3, s=2 and the block is “a”. The 2 identity blocks use three set of filters of size 256x256x2048, f=3 and the blocks are “b” and “c”.</li><li>The 2D Average Pooling uses a window of shape (2,2) and its name is “avg_pool”.</li><li>The flatten doesn’t have any hyperparameters or name.</li><li>The Fully Connected (Dense) layer reduces its input to the number of classes using a softmax activation. Its name should be ‘fc‘ + str(classes).</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/06738c920e09e061ccd3eb49eb71d36c/href">https://medium.com/media/06738c920e09e061ccd3eb49eb71d36c/href</a></iframe><h3><strong>Summary:</strong></h3><ul><li>Very deep neural networks (plain networks) are not practical to implement as they are hard to train due to vanishing gradients.</li><li>The skip-connections help to address the Vanishing Gradient problem. They also make it easy for a ResNet block to learn an identity function.</li><li>There are two main types of ResNets blocks: The identity block and the convolutional block.</li><li>Very deep Residual Networks are built by stacking these blocks together.</li></ul><h3>References:</h3><ul><li>‘Deep Residual Learning for Image Recognition’ research paper from Microsoft Research — [<a href="https://arxiv.org/pdf/1512.03385.pdf">Link to paper</a>]</li><li><a href="https://www.coursera.org/learn/convolutional-neural-networks">https://www.coursera.org/learn/convolutional-neural-networks</a></li><li><a href="https://www.appliedaicourse.com/lecture/11/applied-machine-learning-online-course/3411/residual-network/8/module-8-neural-networks-computer-vision-and-deep-learning">https://www.appliedaicourse.com</a></li><li><a href="https://medium.com/@14prakash/understanding-and-implementing-architectures-of-resnet-and-resnext-for-state-of-the-art-image-cf51669e1624">https://medium.com/@14prakash/understanding-and-implementing-architectures-of-resnet-and-resnext-for-state-of-the-art-image-cf51669e1624</a></li><li><a href="https://engmrk.com/residual-networks-resnets/">https://engmrk.com/residual-networks-resnets/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b80f9a507b9c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/understanding-and-implementation-of-residual-networks-resnets-b80f9a507b9c">Understanding and implementation of Residual Networks(ResNets)</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>