<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Akshay Maheshwari on Medium]]></title>
        <description><![CDATA[Stories by Akshay Maheshwari on Medium]]></description>
        <link>https://medium.com/@akshay26072000?source=rss-956f6e31ca63------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*UcB-2euem_RJ8RCk.jpg</url>
            <title>Stories by Akshay Maheshwari on Medium</title>
            <link>https://medium.com/@akshay26072000?source=rss-956f6e31ca63------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 09:26:46 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@akshay26072000/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Convolutional Neural Network]]></title>
            <link>https://medium.com/@akshay26072000/convolutional-neural-network-309bf620b2af?source=rss-956f6e31ca63------2</link>
            <guid isPermaLink="false">https://medium.com/p/309bf620b2af</guid>
            <dc:creator><![CDATA[Akshay Maheshwari]]></dc:creator>
            <pubDate>Sun, 12 Jul 2020 07:59:47 GMT</pubDate>
            <atom:updated>2020-07-12T07:59:47.115Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*rZYU-cplR5TUneBe.png" /></figure><p>Hello and welcome Guys. Till now we have learned a lot of new things in Machine Learning. Now , today in this Blog i am going to explain the <strong>actual concept of Convolutional Neural Network , need and applications of Convolutional Neural Network.</strong></p><p>So, without wasting any single moment let’s start our journey of Learning.</p><p>Before going into the deep dive of Convolutional Neural Network, i would like to explain that Why we need Convolutional Neural Network ?</p><h3>What is the need of CNN :</h3><p>As we already know that shallow neural network are use to extract the important features from the given dataset. Dataset may be either in the form of images or in the form of Videos.</p><p>But, in shallow neural network, we have to find the important features from the images manually and if we extract features manually , then it will take a lot of time and after extracting features, these features can’t be used with another type of dataset. So, extracting the features manually is not a good way.</p><p><strong><em>That’s why we use Convolutional Neural Network (CNN). CNN extract the main features automatically. So, we don’t require to waste our time to extract the features manually in case of CNN.</em></strong></p><h3>Convolutional Neural Network (CNN)</h3><blockquote>“Convolutional neural network is one the famous deep learning technique that is used for automatic feature extraction. In CNN, we have a lots of hidden layer that are responsible to extract the main features. So, we don’t need to worry about the feature extraction process process while using CNN.”</blockquote><h4>Let’s understand the architecture of CNN (Step-wise).</h4><h3>Covolutional Neural Network Architecture :</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NYHEiYb6AHLz9FtV.jpeg" /></figure><p>Suppose an input image of car is given to us and we have extract the feature using CNN. So, let’s understand that how CNN extract the main- main features from the given image (or Dataset).</p><p>Convolutional neural Network is made up of two Words :</p><ol><li><strong><em>Convolve</em></strong></li><li><strong><em>Neural Network</em></strong></li></ol><p>Let’s understand the meaning of these two words.</p><h3><strong>1. Convolve :</strong></h3><p>Convolve is the process of selecting main-main features from the dataset and convert it into appropriate input form for the Neural Network. For doing this covolve process some steps we have to follow as i mentioned below :</p><p><strong><em>a) Convolution</em></strong></p><p><strong><em>b) Apply Activation function (for eg; Relu)</em></strong></p><p><strong><em>c) Pooling</em></strong></p><p><strong><em>d) repeat steps a to c for getting maximum feature with lower dimension</em></strong></p><p>Let’s understand the above four steps in detail :</p><p><strong>a) Convolution :<em> Convolution is the process of selecting main- main feature from the images. For doing this, we use a filter or kernel (matrix) and do dot product to extract the features. By applying the kernel or filter on the image, important features get selected.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GG-65WZ0_edwPja7.png" /></figure><p>As we can see in above figure , we apply different kernel to extract the important feature of given cat image and after applying the kernel, important features of the image get extracted as we can see.</p><p><strong><em>Group of multiple filters is called feature map because they are use to select the features.</em></strong></p><p><strong>b) Apply activation function (Relu) :</strong></p><p><strong><em>After selecting the important features from the image, we apply relu activation function in order to deal with non- linearity of the data.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*CQLr_--qCE1862IH.jpg" /></figure><p><em>As we can see from above graph, Relu activation function converts negative value into zero because pixels of the images can never be negative.</em></p><p><strong>c) Pooling :</strong></p><p><strong><em>Pooling is the technique that is use to extracting the features from the existing features. Pooling reduces the size of the image so that image will not take more Ram and CPU while training the neural network</em></strong></p><p><em>Pooling is of two types :</em><strong><em> </em></strong><em>First is</em><strong><em> max pooling </em></strong><em>and second one is</em><strong><em> average pooling.</em></strong></p><p><em>Max pool select the maximum value from the given pool size array and average pool select the average value from the given pool size array, as we can see from this figure :</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*zG1fk9aL2IzoKeP_.jpg" /></figure><p><strong>Now, in step-d, repeat steps a-&gt;c in order to get the maximum important feature with lower dimension.</strong></p><blockquote>After this we perform flattening to our dataset in order to convert it into one dimensional array because neural network accepts only one dimensional array as an input.</blockquote><h3>2. Neural Network :</h3><p>After performing the feature selecting and feature extraction, we give our one dimensional data as an input to the Neural Network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*i4MQ18B-TPHsEDNT.jpg" /></figure><p>Neural network is the network that is use to find the actual weight and bias to our model, so that whenever a new image comes, model predicts the category of that new image.</p><p>In neural network we have three layers :</p><p><strong>1. Input layer :</strong> Input layer is the layer that contains input data to the hidden layer. This input layer data pass to the hidden layers. This process is called feed forward because we are feeding our data in forward direction.</p><p><strong>2. Hidden layers : </strong>Hidden layer consist of a lots of neurons. These neurons helps to find the weight and bias after performing some calculations.</p><p>The output of hidden layer matches with the actual output for the given input and error get calculated. If error &gt; 0 then back propogation will be perform.</p><p>Before sending our data to the output layer , we apply activation funtion in order to deal with non linearity of the data.</p><p><strong>3) Output layer : </strong>This is the layer that contains output for the given input.</p><h3>Application of CNN :</h3><p><em>1 ) Decoding facial Recognition</em></p><p><em>2) Analyzing Document</em></p><p><em>3) Understanding Climate</em></p><p><em>4) Advertising</em></p><p><em>5) Grey Areas</em></p><blockquote><strong>Thanks for giving your valuable time.</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=309bf620b2af" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloud Native Development]]></title>
            <link>https://medium.com/@akshay26072000/cloud-native-development-b29e8c75b32b?source=rss-956f6e31ca63------2</link>
            <guid isPermaLink="false">https://medium.com/p/b29e8c75b32b</guid>
            <dc:creator><![CDATA[Akshay Maheshwari]]></dc:creator>
            <pubDate>Thu, 02 Jul 2020 09:26:34 GMT</pubDate>
            <atom:updated>2020-07-02T09:26:34.974Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eLoUt7w_b-ttHB3u.jpg" /></figure><p>Hello and welcome Guys. In previous article i have explained about one of most important service of AWS i.e; Amazon S3 . Now today i will start my journey by explaining Cloud Native Development. In this blog i am going to cover <strong>Cloud Native Development , microservices ,containers , difference between container and virtualization.</strong></p><p>So , Let’s start our journey of learning.</p><h3>Cloud Native Development</h3><blockquote><strong>“Cloud Native Development is an approach that builds software applications as microservices and runs them into containerized and dynamically orchestrated platform to get the maximum benefits which cloud computing promises.”</strong></blockquote><p><em>In simple words i can say that if we develop an application belonging to cloud , then it is called Cloud Native Development.</em></p><blockquote>Microservices is nothing but a small component. It means we develop small independent component like component for only login and signup, a component for video streaming, component for recommendation.</blockquote><p>Each and every microservices runs into container which can be scale automatically and communicate each other using message broken for faster communications. For example, each and every component communicate with each other using <strong>grpc (Google rpc).</strong></p><p>Cloud Native Development is a new approach and hence supports us in getting <strong>agility of development , scalability and hence allows us to cater huge workloads.</strong></p><h3>What are Microservices ?</h3><blockquote><strong>“Microservices are an architectural and organizational approach to software development where software is composed of small independent services that communicate over well-defined APIs. These services are owned by small, self-contained teams.”</strong></blockquote><blockquote><strong><em>Microservices architectures make applications easier to scale and faster to develop, enabling innovation and accelerating time-to-market for new features.</em></strong></blockquote><h3>Characteristics of Microservices</h3><h4><strong>1 ) Autonomous</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/100/0*8dMh3NUTX9pRSyT-.png" /></figure><p>Each component service in a microservices architecture can be developed, deployed, operated, and scaled without affecting the functioning of other services. Services do not need to share any of their code or implementation with other services. Any communication between individual components happens via well-defined APIs.</p><p><strong>2) Specialized</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/86/0*0weQfkoyGKuLgwtJ.png" /></figure><p>Each service is designed for a set of capabilities and focuses on solving a specific problem. If developers contribute more code to a service over time and the service becomes complex, it can be broken into smaller services.</p><h3>Benefits of Microservices</h3><ol><li>Highly maintainable and testable.</li><li>Loosely coupled.</li><li>Independently deployable.</li><li>Owned by a small team.</li><li>Organized around business capabilities.</li></ol><h3>Containers</h3><blockquote><strong>“A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.”</strong></blockquote><p>A container image is a lightweight, standalone, executable package of software that includes everything needed to run an application code , runtime, system tools, system libraries and settings.</p><h3>What’s the difference between containers and virtualization?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-M-INjgXtYd32PvUar_6Ng.png" /></figure><p><em>With virtualization technology, the package that can be passed around is a virtual machine, and it includes an entire operating system as well as the application. A physical server running three virtual machines would have a hypervisor and three separate operating systems running on top of it.</em></p><p><em>By contrast a server running three containerized applications with Docker runs a single operating system, and each container shares the operating system kernel with the other containers. Shared parts of the operating system are read only, while each container has its own mount (i.e., a way to access the container) for writing. That means the containers are much more lightweight and use far fewer resources than virtual machines.</em></p><blockquote><strong>Thanks for giving for your valuable time.</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b29e8c75b32b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dimensionality Reduction Algorithm]]></title>
            <link>https://medium.com/@akshay26072000/dimensionality-reduction-algorithm-93f76bc472c8?source=rss-956f6e31ca63------2</link>
            <guid isPermaLink="false">https://medium.com/p/93f76bc472c8</guid>
            <dc:creator><![CDATA[Akshay Maheshwari]]></dc:creator>
            <pubDate>Wed, 01 Jul 2020 11:10:52 GMT</pubDate>
            <atom:updated>2020-07-01T11:10:52.946Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/696/1*nOEJwoyAxYWHv18QyYKbZQ.jpeg" /></figure><p>Hello and welcome Guys. In previous article we learnt about Classification, we worked with datasets composed of only two independent variables. We did for two reasons:</p><ol><li>Because we needed two dimensions to visualize better how Machine Learning models worked (by plotting the prediction regions and the prediction boundary for each model).</li><li>Because whatever is the original number of our independent variables, we can often end up with two independent variables by applying an appropriate Dimensionality Reduction technique.</li></ol><h3><strong>Types of Dimensionality reduction technique :</strong></h3><p>There are two types of Dimensionality Reduction techniques:</p><ol><li><strong><em>Feature Selection</em></strong></li><li><strong><em>Feature Extraction</em></strong></li></ol><p>Via feature selection, we select a subset of the original features, whereas in feature extraction, we derive information from the feature set to construct a new feature subspace.</p><blockquote><strong><em>Feature Selection techniques are Backward Elimination, Forward Selection, Bidirectional Elimination, Score Comparison and more. I covered these techniques in my previous articles of Machine Learning.</em></strong></blockquote><p><strong>Different feature extraction techniques :</strong></p><ol><li><strong><em>Principal Component Analysis (PCA)</em></strong></li><li><strong><em>Linear Discriminant Analysis (LDA)</em></strong></li><li><strong><em>Kernel PCA</em></strong></li><li><strong><em>Quadratic Discriminant Analysis (QDA)</em></strong></li></ol><p>In this blog we will explore feature extraction. In practice, feature extraction is not only used to improve storage space or the computational efficiency of the learning algorithm, but can also improve the predictive performance by reducing the curse of dimensionality — especially if we are working with non-regularized models.</p><p>Specifically, we will discuss the <strong>Principal Component Analysis</strong> (<strong>PCA</strong>) and <strong>Linear Discriminant Analysis (LDA) </strong>algorithm used to compress a dataset onto a lower-dimensional feature subspace with the goal of maintaining most of the relevant information.</p><p>Let’s get started!</p><h3>Principle Component Analysis (PCA) :</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qNG6n_DeGsKiwVLMbuEaUQ.jpeg" /></figure><p>This method was introduced by Karl Pearson. It works on a condition that while the data in a higher dimensional space is mapped to data in a lower dimension space, the variance of the data in the lower dimensional space should be maximum.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/329/0*P5JaT5GyfYa1W86_.jpg" /></figure><p>PCA helps us to identify patterns in data based on the correlation between features. In a nutshell, PCA aims to find the directions of maximum variance in high-dimensional data and projects it onto a new subspace with equal or fewer dimensions than the original one.</p><p>If we use PCA for dimensionality reduction, we construct a <em>d</em> x <em>k</em>–dimensional transformation matrix <strong><em>W</em></strong> that allows us to map a sample vector <strong><em>x</em></strong> onto a new <em>k</em>–dimensional feature subspace that has fewer dimensions than the original <em>d</em>–dimensional feature space:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/233/0*9HhqGsOKntRYImhC.png" /></figure><p>As a result of transforming the original <em>d</em>-dimensional data onto this new <em>k</em>-dimensional subspace (typically <em>k</em> ≪ <em>d</em>), the first principal component will have the largest possible variance, and all consequent principal components will have the largest variance given the constraint that these components are uncorrelated (orthogonal) to the other principal components — even if the input features are correlated, the resulting principal components will be mutually orthogonal (uncorrelated).</p><blockquote><strong><em>Note that the PCA directions are highly sensitive to data scaling, and we need to standardize the features prior to PCA if the features were measured on different scales and we want to assign equal importance to all features.</em></strong></blockquote><h3><strong>Approach of PCA in some steps :</strong></h3><ol><li>Standardize the <em>d</em>-dimensional dataset.</li><li>Construct the covariance matrix.</li><li>Decompose the covariance matrix into its eigenvectors and eigenvalues.</li><li>Sort the eigenvalues by decreasing order to rank the corresponding eigenvectors.</li><li>Select <em>k</em> eigenvectors which correspond to the <em>k</em> largest eigenvalues, where <em>k</em> is the dimensionality of the new feature subspace (<em>k</em> ≤ <em>d</em>).</li><li>Construct a projection matrix <strong><em>W</em></strong> from the “top” <em>k</em> eigenvectors.</li><li>Transform the <em>d</em>-dimensional input dataset <strong><em>X</em></strong> using the projection matrix <strong><em>W</em></strong> to obtain the new <em>k</em>-dimensional feature subspace.</li></ol><h3><strong>Linear Discriminant Analysis</strong></h3><p><strong>Linear Discriminant Analysis</strong> or <strong>Normal Discriminant Analysis</strong> or <strong>Discriminant Function Analysis</strong> is a dimensionality reduction technique which is commonly used for the supervised classification problems. It is used for modeling differences in groups i.e. separating two or more classes. It is used to project the features in higher dimension space into a lower dimension space.</p><p>For example, we have two classes and we need to separate them efficiently. Classes can have multiple features. Using only a single feature to classify them may result in some overlapping as shown in the below figure. So, we will keep on increasing the number of features for proper classification.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/538/0*PcDtKTfhrufMG9_j.jpg" /></figure><p><strong>Example:</strong><br>Suppose we have two sets of data points belonging to two different classes that we want to classify. As shown in the given 2D graph, when the data points are plotted on the 2D plane, there’s no straight line that can separate the two classes of the data points completely. Hence, in this case, LDA (Linear Discriminant Analysis) is used which reduces the 2D graph into a 1D graph in order to maximize the separability between the two classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/459/0*4Jc0yiBZ_BLTK_xb.jpg" /></figure><p>Here, Linear Discriminant Analysis uses both the axes (X and Y) to create a new axis and projects data onto a new axis in a way to maximize the separation of the two categories and hence, reducing the 2D graph into a 1D graph.</p><p>Two criteria are used by LDA to create a new axis:</p><ol><li>Maximize the distance between means of the two classes.</li><li>Minimize the variation within each class.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/0*UKJ7gFDrtgfjzxfs.jpg" /></figure><p>In the above graph, it can be seen that a new axis (in red) is generated and plotted in the 2D graph such that it maximizes the distance between the means of the two classes and minimizes the variation within each class. In simple terms, this newly generated axis increases the separation between the dtla points of the two classes. After generating this new axis using the above-mentioned criteria, all the data points of the classes are plotted on this new axis and are shown in the figure given below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/608/0*WnePrXlE48706cc4.jpg" /></figure><p>But Linear Discriminant Analysis fails when the mean of the distributions are shared, as it becomes impossible for LDA to find a new axis that makes both the classes linearly separable. In such cases, we use non-linear discriminant analysis.</p><p><strong>Extensions to LDA:</strong></p><ol><li><strong>Quadratic Discriminant Analysis (QDA):</strong> Each class uses its own estimate of variance (or covariance when there are multiple input variables).</li><li><strong>Flexible Discriminant Analysis (FDA):</strong> Where non-linear combinations of inputs is used such as splines.</li><li><strong>Regularized Discriminant Analysis (RDA):</strong> Introduces regularization into the estimate of the variance (actually covariance), moderating the influence of different variables on LDA.</li></ol><p><strong>Applications:</strong></p><ol><li><strong>Face Recognition</strong></li><li><strong>Medical</strong></li><li><strong>Customer Identification.</strong></li></ol><h3>Advantages of dimensionality reduction</h3><ol><li>It reduces the time and storage space required.</li><li>Removal of multi-collinearity improves the interpretation of the parameters of the machine learning model.</li><li>It becomes easier to visualize the data when reduced to very low dimensions such as 2D or 3D.</li><li>It avoids the curse of dimensionality.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=93f76bc472c8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Task 3 MLOP]]></title>
            <link>https://medium.com/@akshay26072000/task-3-mlop-faf1f3d36900?source=rss-956f6e31ca63------2</link>
            <guid isPermaLink="false">https://medium.com/p/faf1f3d36900</guid>
            <dc:creator><![CDATA[Akshay Maheshwari]]></dc:creator>
            <pubDate>Wed, 01 Jul 2020 03:00:18 GMT</pubDate>
            <atom:updated>2020-07-01T03:00:18.939Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*9gbbQrdMD94-yZpXswUAfw.jpeg" /></figure><ul><li>Docker</li><li>Github</li><li>Git</li></ul><p>1. Create container image that’s has Python3 and Keras or numpy installed using dockerfile</p><p>2. When we launch this image, it should automatically starts train the model in the container.</p><p>3. Create a job chain of job1, job2, job3, job4 and job5 using build pipeline plugin in Jenkins</p><p>Job1 : Pull the Github repo automatically when some developers push repo to Github.</p><p>Job2 : By looking at the code or program file, Jenkins should automatically start the respective machine learning software installed interpreter install image container to deploycode and start training( eg. If code uses CNN, then Jenkins should start the container that has already installed all the softwares required for the cnn processing).</p><p>Job3 : Train your model and predict accuracy or metrics.</p><p>Job4 : if metrics accuracy is less than 80% , then tweak the machine learning model architecture.</p><p>Job5: Notify that the best model is being created.</p><h3>Build pipeline for jobs.</h3><p>About the task:</p><p>1.Using Redhat8 , created a Dockerfile for creating docker images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JYWxhS8SrbFRXa0a" /></figure><p>This Dockerfile contains keras , tensorflow ,matplotlib etc.</p><ul><li>To see the docker image using command “docker images”.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/965/0*mJxEkLM_7VHCFnyc" /></figure><p>Connect jenkins with GitHub</p><p>Create jobs in jenkins</p><p>Job1 : we have to give the link of Github repository to push the data to jenkins by POLLSCM and then copy the data to a folder “/root/mycode”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vWkGA69Op1q1Z49RcLHq4Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hAcHIM2V7NXG3f9_Dhv9Gw.png" /></figure><p>Job2 : By looking at the code or program file, Jenkins automatically copy the respective <em>python file</em> to the required Environment which is in image container to deploy code and start training Eg. If code uses CNN, then Jenkins should start the container that has already installed Environment (all the libraries required for the CNN processing).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d_RmEUPlrUyEMwnAZwTQqA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WeKYq0kr0KJ3MZtMc01q8w.png" /></figure><p>Job3 : Train model and predict accuracy or metrics</p><p>Check accuracy of model, If metrics accuracy is less than 85% , then tweak the machine learning model architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*yp9aHo95ByGwCJYvCuWbNg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XwPHmN3vVbZG_eeIcNMjnA.png" /></figure><p>Job4 : Notify that the best model is being created through an email…by using a python code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZfifKzSnnePzdWMXHnyISQ.png" /></figure><p>Python code for email :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OOWD9NLbPd7UqUXSsNz1Ig.png" /></figure><p>Pipeline that successful complete all the jobs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lIyAEiP8ufkSlR2gVd24Hw.jpeg" /></figure><p><strong><em>Thanks for giving your valuable time.</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=faf1f3d36900" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Association Mining And Various Technology Associated With It]]></title>
            <link>https://medium.com/@akshay26072000/association-mining-and-various-technology-associated-with-it-cdcb7de76b55?source=rss-956f6e31ca63------2</link>
            <guid isPermaLink="false">https://medium.com/p/cdcb7de76b55</guid>
            <dc:creator><![CDATA[Akshay Maheshwari]]></dc:creator>
            <pubDate>Tue, 30 Jun 2020 08:30:10 GMT</pubDate>
            <atom:updated>2020-07-01T03:27:56.590Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CHjO23v42iXcse5nj0L1xg.jpeg" /></figure><h3>What is Association ?</h3><p>If people want to buy some product , then they also buy some other product. This is called association. For this , seller of the product will give some other options to the customer . More option seller will give to the customer, chances to buy some other product will be more . For example; if any people want to buy bread then he should also buy butter and milk ,if seller of the bread will give more options to the customer.</p><blockquote><strong><em>“ People who bought something , also bought something else. People who watch something also watch something else. People who did something , also did something else.”</em></strong></blockquote><p><strong>Recommendation : </strong>Recommendation is the way you give more options to the customer. More options you will give, chances to buy the other products will be more.</p><p><strong>Note : If we have limited number of products then we can do association without using any machine (model), but if we have thousands of product, then in this case we require some model to predict the association of items.</strong></p><p>So , In order to made the model , we have to give some previous data (past data of transaction) to the machine, So that machine can easily predict association of itmes.</p><h3>Now question is that which combination should i create to association ?</h3><p>In order to decide the combination of items we use <strong>Association Rule.</strong></p><p><strong>In machine learning , we have two types of Association rule :</strong></p><ol><li><strong>Apriori Model</strong></li><li><strong>Elcat model</strong></li></ol><p><strong>Apriori Model :</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rh1OmEwpcifPelNOwFCriA.jpeg" /></figure><p><strong>Apriori algorithm</strong> is given by R. Agrawal and R. Srikant in 1994 for finding frequent itemsets in a dataset for boolean association rule. Name of the algorithm is Apriori because it uses prior knowledge of frequent itemset properties. We apply an iterative approach or level-wise search where k-frequent itemsets are used to find k+1 itemsets.</p><h4>Market Basket Analysis</h4><p>In today’s world, the goal of any organization is to increase revenue. Can this be done by pitching just one product at a time to the customer? The answer is a clear <strong>no</strong>. Hence, organizations began mining data related to frequently bought items.</p><p><strong>Market Basket Analysis</strong> is one of the key techniques used by large retailers to uncover associations between items. They try to find out associations between different items and products that can be sold together, which gives assisting in right product placement. Typically, it figures out what products are being bought together and organizations can place products in a similar manner.</p><p><strong>How does Apriori Algorithm Works ?</strong></p><p><strong>Apriori algorithm has three parts :</strong></p><ol><li><strong>Support</strong></li><li><strong>Confidence</strong></li><li><strong>Lift</strong></li></ol><p><strong>Support :</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WsDTEav2bo6BQZ2htaowwg.jpeg" /></figure><p>Let’s understand the concept of support by taking one example :</p><p>Here we have 100 people. So, we have 5 rows and 20 columns of human bieng.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PpiFqxjwhkmPnoJpZINY9Q.jpeg" /></figure><p>Let’s say we are talking about a movie and i am going to give you an example of my favourite movie <strong>ex machina.</strong> So let’s see how many of these 100 people have actually seen <strong>ex machina.</strong></p><p>There are ten people who actually seen x machina out of hundred people :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cDnmqSF2pNpUAyoTiIbiSw.jpeg" /></figure><p><strong>So , support = 10/ 100 = 10 %</strong></p><p><strong>Confidence</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zfqtcbAzKKH5k9bJ01NCvQ.jpeg" /></figure><p>let’s understand the concept of confidence by taking an exapmle :</p><p>Let’s say those people coloured in green are the one who have seen interstellar movie. Now we want to know , not out of a whole population but out of just those people who have seen interstellar , how many of them have seen x machina ?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pc9CPPgoEfnMihytEEWpNQ.png" /></figure><p>So , out of them we have only seven people who have also seen x machina.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P2KqQ6z6mh5sc0AUQPhJvw.png" /></figure><p><strong>So , confidence = 7 / 40 = 17.5 %</strong></p><p><strong>Lift :</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zc1zF8g6R3De9y74uZKqnQ.png" /></figure><h4>Let’s understand the concept of lift by taking one example :</h4><p>Those people who are in the green are the ones who have seen interstellar, and the people who are in red are the ones who have seen x machina.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1s4ExQ3jva507dJDWcdy9w.png" /></figure><p>If we just randomly suggest to a person to watch x machina then , what is the probability that they will like it ?</p><p>The probability is 10 % because of 100 we have 10 people who like x machina.</p><p>Now let’s only recommend x machine to only those people who have already seen interstellar. So , in this case the probability of liking x machina will be 17.5 %</p><p><strong>So , lift = 17.5 % / 10 % = 1.75 %</strong></p><h3>Now let’s understand the Apriori algorithm Step- wise</h3><p><strong>Step -1 : </strong>Set a minimum support and confidence.</p><p><strong>Step -2 : </strong>Take all the subsets in transactions having higher support than minimum support.</p><p><strong>Step -3 :</strong> Take all the rules of these subsets having higher confidence than minimum confidence</p><p><strong>Step -4 : </strong>Sort the rules by decreasing fit.</p><h3>Applications of association rule mining</h3><ul><li>Supermarket items association analysis .</li><li>Web mining</li><li>Medical Analysis</li><li>Bioinformatics</li><li>Network Analysis</li><li>Programming pattern finding , etc .</li></ul><h3>Advantage and Disadvantage</h3><p><strong>Advantage :</strong></p><ul><li>Predict future trends, customer purchase habits</li><li>Help with decision making</li><li>Improve company revenue and lower costs</li><li>Market basket analysis</li><li>Fraud detection</li></ul><p><strong>Disadvantage :</strong></p><ul><li>User privacy/security</li><li>Amount of data is overwhelming</li><li>Great cost at implementation stage</li><li>Possible misuse of information</li><li>Possible in accuracy of data</li></ul><blockquote><strong><em>Thanks for giving your valuable time.</em></strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cdcb7de76b55" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[K-means clustering algorithm]]></title>
            <link>https://medium.com/@akshay26072000/k-means-clustering-algorithm-4f4cbbc18a81?source=rss-956f6e31ca63------2</link>
            <guid isPermaLink="false">https://medium.com/p/4f4cbbc18a81</guid>
            <dc:creator><![CDATA[Akshay Maheshwari]]></dc:creator>
            <pubDate>Sat, 27 Jun 2020 16:44:26 GMT</pubDate>
            <atom:updated>2020-06-28T14:08:00.936Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3Dg1SfPJhFNAZeLJOv-eBQ.jpeg" /></figure><p>Let’s start our journey of learning.</p><p>Let’s imagine that we have got two variables in our dataset and we decide to plot these two variables on the X-axis and Y-axis . So, question here is , can we identify certain group among all variables and how can we do this ?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2XqsFVrdpKpWVJ_Qn6prTQ.jpeg" /></figure><p>How do we identify the number of groups ?</p><p>So, what the K-means does for you ? K-means takes out the complexity from this decision making process and it allows you to very easily identify the clusters in the given dataset.</p><p>So, after performing this K-means algorithm on the given scatterplot we will get the result like this :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-mPuf7rNZ2kRRjMWRotrpg.jpeg" /></figure><p>Now question is how K-means Algorithm works ?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oIqFuCEcD__Z2xZPhePSnA.jpeg" /></figure><p>Let’s understand the working of K-means algorithm step wise :</p><p><strong>Step -1 : Choose the number of clusters.</strong></p><p><strong>Step -2 : Select at random k points , the centroids (not necessarily from your dataset)</strong></p><p><strong>Step -3 : Assign each data point to the closest centroid -&gt; that forms k-clusters.</strong></p><p><strong>Step -4 : Compute and place the new centroid of each cluster.</strong></p><p><strong>Step -5: Reassign each data point to the new closest centroid . If any reassignment took place, go to step-4 , otherwise go to FIN.</strong></p><p><strong><em>********** Your Model is Ready********</em></strong></p><p>Let’s understand all the above steps by an exapmle :</p><p><strong>Step -1 : Choose the number of clusters.</strong></p><p>Here is our scatterplot and we have two variables .</p><p>Now question is can you identify the number of clusters in this scatterplot ?</p><p>Answer is no, we can’t identify the number of clusters by looking this scatterplot ,although we have only two variables here.</p><p>So , that’s where the K-means clustering algorithm comes into play.</p><p>Here we take , K = 2 (By using ELBOW method)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*jpCEMvucuU8X50nR1LVCqA.jpeg" /></figure><p><strong>Step -2 : Select at random k points , the centroids (not necessarily from your dataset)</strong></p><p>So, let’s say we select two centroid one is red and another is green.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E7xBNlYrwiITaUBsbD_XPQ.jpeg" /></figure><p><strong>Step -3 : Assign each data point to the closest centroid -&gt; that forms k-clusters.</strong></p><p>So , basically we need to identify that which of the data point is closer to these centroid but we are going to use some hack here. We connect these centroids with a line and then we put a perpendicular on the center of this line. Any point on this perpendicular line (green line) will be equidistance from all of the centroids.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F5oc6hwPAlV_R2XIqlHbHQ.jpeg" /></figure><p>Now we colored our data points on the scatterplot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wl5lsj-R-41WvmLajcHcKw.jpeg" /></figure><p><strong>Step -4 : Compute and place the new centroid of each cluster.</strong></p><p>Now, we need to find out the new centroid for the green points and new centroid for the red points.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*EaUYYS6cCNZK4wiBgER4xw.jpeg" /></figure><p><strong>Step -5: Reassign each data point to the new closest centroid . If any reassignment took place, go to step-4 , otherwise go to FIN.</strong></p><p>Now reassign each data point to the new closest centroid that we have computed in step-4 . If any reassignment took place, go to step-4 , otherwise go to finish the algorithm.</p><p>After performing this iterative process , we find this final cluster :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*9vOqicC6cvLfiyv9cn4sLw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/937/1*J_maPJAs9tQZ87oBKFUkgw.jpeg" /></figure><p>So in this way K- means algorithm works.</p><h3>Now question is that how can we find the optimal number of clusters ?</h3><p>The method consists of plotting the explained variation as a function of the number of clusters, and picking the elbow of the curve as the number of clusters to use. The same method can be used to choose the number of parameters in other data-driven models, such as the number of principal components to describe a data set.</p><h3>Intuition</h3><p>Using the “elbow” or “knee of a curve ” as a cutoff point is a common heuristic in mathematical optimization to choose a point where diminishing returns are no longer worth the additional cost. In clustering, this means one should choose a number of clusters so that adding another cluster doesn’t give much better modeling of the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/381/1*_0pna2g23jKlx2DRfZ9qkA.jpeg" /></figure><p>The intuition is that increasing the number of clusters will naturally improve the fit (explain more of the variation), since there are more parameters (more clusters) to use, but that at some point this is over-fitting, and the elbow reflects this. For example, given data that actually consist of <em>k</em> labeled groups — for example, <em>k</em> points sampled with noise — clustering with more than <em>k</em> clusters will “explain” more of the variation (since it can use smaller, tighter clusters), but this is over-fitting, since it is subdividing the labeled groups into multiple clusters. The idea is that the first clusters will add much information (explain a lot of variation), since the data actually consist of that many groups (so these clusters are necessary), but once the number of clusters exceeds the actual number of groups in the data, the added information will drop sharply, because it is just subdividing the actual groups. Assuming this happens, there will be a sharp elbow in the graph of explained variation versus clusters: increasing rapidly up to <em>k</em> (under-fitting region), and then increasing slowly after <em>k</em> (over-fitting region).</p><h3>k-Means Advantages and Disadvantages</h3><h3>Advantages of k-means</h3><p><strong>1)Relatively simple to implement.</strong></p><p><strong>2) Scales to large data sets.</strong></p><p><strong>3) Guarantees convergence.</strong></p><p><strong>4) Can warm-start the positions of centroids.</strong></p><p><strong>5) Easily adapts to new examples.</strong></p><p><strong>6) Generalizes to clusters of different shapes and sizes, such as elliptical clusters.</strong></p><h3>Disadvantages of k-means</h3><p><strong>1) Being dependent on initial values.</strong></p><p><strong>2) Clustering data of varying sizes and density.</strong></p><p><strong>3) Clustering outliers.</strong></p><p><strong>4) Scaling with number of dimensions.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4f4cbbc18a81" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mlops Task2]]></title>
            <link>https://medium.com/@akshay26072000/mlops-task2-4dd25e00252?source=rss-956f6e31ca63------2</link>
            <guid isPermaLink="false">https://medium.com/p/4dd25e00252</guid>
            <dc:creator><![CDATA[Akshay Maheshwari]]></dc:creator>
            <pubDate>Wed, 24 Jun 2020 13:28:58 GMT</pubDate>
            <atom:updated>2020-06-24T13:28:58.506Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nztcl5Ez53F9CxsTCCdYGg.png" /></figure><h3>What is the Task ?</h3><p>1. Create container image that’s has Jenkins installed using dockerfile</p><p>2. When we launch this image, it should automatically starts Jenkins service in the container.</p><p>3. Create a job chain of job1, job2, job3 and job4 using build pipeline plugin in Jenkins</p><p>4. Job1 : Pull the Github repo automatically when some developers push repo to Github.</p><p>5. Job2 : By looking at the code or program file, Jenkins should automatically start the respective language interpreter install image container to deploy code ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed ).</p><p>6. Job3 : Test your app if it is working or not.</p><p>7. Job4 : if app is not working , then send email to developer with error messages.</p><p>8. Create One extra job job5 for monitor : If container where app is running. fails due to any reason then this job should automatically start the container again.</p><h3>Pre-requisite</h3><p>1.Docker in your system.</p><p>2.Basic knowledge about docker commands.</p><p>3.Internet connectivity.</p><h3>Procedure</h3><p>So without wasting our more time let’s begin!!</p><p>Step 1: Create container image that’s has Jenkins installed using dockerfile.</p><p>For achieving this we have to create a docker file.</p><p>* Now first we create our workspace.</p><p>* Inside that workspace you have to create the docker file with name Dockerfile.</p><p>Note: Remember when you create any dockerfile it’s name must be Dockerfile.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HnGQhhVG4q0wJMTw" /></figure><p>*In this dockerfile we put our requirement and create the jenkins image.</p><p>* Then build this image “ docker build -t image_name:version &lt;workspace&gt;”</p><p>i.e docker build -t jenkins_image:v1 /my-ws</p><p>* Here we successfully build our image.</p><p>Step 2 : Launch your image.</p><p>Step 3 : Now we are going to create a job chain of job 1 (Github_pull), job (test_dockercmd) , job 3 (Testing) , job 4 (Email) ,job 5</p><p>Step 4 : Now create the job 1 in my case Github_pull.</p><p>* Use your github repo link whose code you want to download.</p><p>*In this job i have use Poll_SCM which will monitor my github repo every minute and aa the developer push the new code to the github it will download automatically.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gvUU8z0lFHOqNRRt" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iVLBnJCBoDFX63QS" /></figure><p>Step 5 : Now we are going to create the job 2 i.e test_dockercmd in my case. you can give any name to your job according to your convenience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yv6e6CuJfwQ9f_KU" /></figure><p>* In this job in build trigger we use build after when job 1 is successfully build.</p><p>* Now it’s time to make it more automated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*um_zvx2UrM9a8IJ_" /></figure><p>Step 6 : Creating job 3 i.e Testing</p><ul><li>Testing your app working or not.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DL-JGDRO-U_monb-" /></figure><p>Step 7 : Creating job 4 i.e Email</p><ul><li>If due to any reason your job 3 is failed then it will send you a mail to the developer so get they can manage the code.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Oyc2BO4csoxlkQ5JNLrVCQ.jpeg" /></figure><p>Step 8 : Creating the extra job</p><p>* In this we are monitoring our container if due to any reason our container goes down then we Jenkins will automatically restart that container.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*O-o1SR7YYbRfqmY3" /></figure><p>So finally we have create all the required job successfully. In this we have create a job chain using build pipeline.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XHXa2v3EfSwb4P-R" /></figure><p>Here you can see all the jobs are build successfully.</p><p><strong><em>Thanks for giving your valuable time.</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4dd25e00252" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Project :Docker-Jenkins-Github integration]]></title>
            <link>https://medium.com/@akshay26072000/project-docker-jenkins-github-integration-d8c3407e9001?source=rss-956f6e31ca63------2</link>
            <guid isPermaLink="false">https://medium.com/p/d8c3407e9001</guid>
            <dc:creator><![CDATA[Akshay Maheshwari]]></dc:creator>
            <pubDate>Wed, 24 Jun 2020 11:55:30 GMT</pubDate>
            <atom:updated>2020-06-24T11:55:30.386Z</atom:updated>
            <content:encoded><![CDATA[<h3>Project :Docker-Jenkins-Github integration</h3><h3>TASK</h3><p>There is a system of Developer side , Quality Assurance side , Client side. When Developer writes a code and commits it over Github ,it is forwarded to the Quality Aassurance side for testing . If tested OK , the developer code is deployed on the Client Side webpage .Rough diagram is as follows…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H0CMbz3C3jCkktd4ZWaOLQ.jpeg" /></figure><p>1 Jenkins:</p><p>When the Developer writes a code in a developer branch on his local Git repository and uploads it to the Git hub , Jenkins get that code and deploy on the Quality Assurance Team Environment on Docker server (Testing) . Setting are as follows…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P4SUiacmPCB4eyf8JCdHkA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ysmvgk1ZVUV7zxPoUPgRqA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JzM6QPjdBrylmYJcPOQGaA.jpeg" /></figure><h3>Job 2 : Jenkins</h3><p>Create the Client side Docker environment for displaying the changes in the Master branch , these changes are carried out by the Job 3. This is the website which is live to the clients.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E8Bund34ziC-htGieiRSCA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uqrrpYWxANrTSIh4TUFYfA.jpeg" /></figure><h3>Job 3 : Jenkins</h3><p>This is Triggered by the Quality assurance team if the developer’s code is correct . The Quality Test is triggered and it merges the Developer changes to the Master branch and then runs the Job 2 to display those change on the Client side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oiCDdBOGr14JSnk4neXlcQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2ckXlEp4eQptIAMc_Rl8mw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GvG4u8DerKzEVwozIixRUA.jpeg" /></figure><p>GitHub Setup :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fa3quNjl7DEQMLJ7fJAKSQ.png" /></figure><h3>Here comes an end of the Project deployment. All the tasks are completely done …. Thank you !!!</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d8c3407e9001" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS : Working of S3]]></title>
            <link>https://medium.com/@akshay26072000/aws-working-of-s3-3f82d5f49d08?source=rss-956f6e31ca63------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f82d5f49d08</guid>
            <dc:creator><![CDATA[Akshay Maheshwari]]></dc:creator>
            <pubDate>Wed, 24 Jun 2020 09:03:42 GMT</pubDate>
            <atom:updated>2020-06-24T09:03:42.823Z</atom:updated>
            <content:encoded><![CDATA[<h3>AWS : Working of S3</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KRZPNhdjq7r6DRQr3V1Q1Q.png" /></figure><p>In this blog we will know about the working of S3 Service of AWS, it’s advantage and creating S3 to store some permanent data.</p><h3>What is Amazon S3?</h3><p>Amazon S3 has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites.</p><p>Amazon S3 has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web.</p><h3>Advantages of using Amazon S3</h3><ul><li><strong>Creating buckets</strong> — Create and name a bucket that stores data. Buckets are the fundamental containers in Amazon S3 for data storage.</li><li><strong>Storing data</strong> — Store an infinite amount of data in a bucket. Upload as many objects as you like into an Amazon S3 bucket. Each object can contain up to 5 TB of data. Each object is stored and retrieved using a unique developer-assigned key.</li><li><strong>Downloading data</strong> — Download your data or enable others to do so. Download your data anytime you like, or allow others to do the same.</li><li><strong>Permissions</strong> — Grant or deny access to others who want to upload or download data into your Amazon S3 bucket. Grant upload and download permissions to three types of users. Authentication mechanisms can help keep data secure from unauthorized access.</li></ul><h3>Creating an S3 storage</h3><p>The S3 storage we create on AWS cloud is known as Bucket . The files we store in the bucket are known as objects. Every bucket name should be unique among all the accounts lying in the particular region in which the bucket is created.</p><p>So, Let us see how to create a bucket in our AWS account :</p><h3>Creating bucket by using WebUi of AWS Cloud :</h3><p>Login to your account and go to</p><p>Services &gt;&gt;Storage&gt;&gt;S3&gt;&gt;Create Bucket</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ldIUjeig_VHlkJgRIAw5FQ.jpeg" /></figure><p>Enter a unique bucket name . Choose the region for your bucket and click next :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OPRJzwYU_ksuSiIvokaYlA.jpeg" /></figure><p>Add some tag (optional) to your bucket and some other settings :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k4uZIE-4dULy81mHy3uhcw.jpeg" /></figure><p>Choose the public access settings you want to give and click on create bucket.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qhzZ8j9V5S8jUk5GTm4VQw.jpeg" /></figure><p>Bucket Created Successfully !!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BP3nTIo-kr0UgpLZwfzEJA.jpeg" /></figure><p>Now you can click the bucket and upload files from your local system onto it.</p><p>I have uploaded an image object in my bucket and have got a url for accessing the object , as you can see : -&gt;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ii51vGeK_F-bcpPxGB5yVQ.jpeg" /></figure><p>If i search this URL , i can see my uploaded photograph that is save in the S3 bucket , as you can see -&gt;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VFXl_Z_oABTw8joLwSr76g.jpeg" /></figure><h3>Benefits of Amazon S3</h3><h3>Industry-leading performance, scalability, availability, and durability</h3><p>Scale your storage resources up and down to meet fluctuating demands, without upfront investments or resource procurement cycles. Amazon S3 is designed for 99.999999999% (11 9’s) of data durability because it automatically creates and stores copies of all S3 objects across multiple systems. This means your data is available when needed and protected against failures, errors, and threats.</p><h3>Wide range of cost-effective storage classes</h3><p>Save costs without sacrificing performance by storing data across the S3 Storage Classes, which support different data access levels at corresponding rates. You can use S3 Storage Class Analysis to discover data that should move to a lower-cost storage class based on access patterns, and configure an S3 Lifecycle policy to execute the transfer. You can also store data with changing or unknown access patterns in S3 Intelligent-Tiering, which tiers objects based on changing access patterns and automatically delivers cost savings.</p><h3>Unmatched security, compliance, and audit capabilities</h3><p>Store your data in Amazon S3 and secure it from unauthorized access with encryption features and access management tools. S3 is the only object storage service that allows you to block public access to all of your objects at the bucket or the account level with S3 block public access. S3 maintains compliance programs, such as PCI-DSS, HIPAA/HITECH, FedRAMP, EU Data Protection Directive, and FISMA, to help you meet regulatory requirements. S3 integrates with Amazon Macie to discover and protect your sensitive data. AWS also supports numerous auditing capabilities to monitor access requests to your S3 resources.</p><p>This is all about this article of Amazon S3.</p><blockquote><strong>Thanks for giving your valuable time.</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f82d5f49d08" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Naive Bayes Algorithm]]></title>
            <link>https://medium.com/@akshay26072000/naive-bayes-algorithm-8c68e5432030?source=rss-956f6e31ca63------2</link>
            <guid isPermaLink="false">https://medium.com/p/8c68e5432030</guid>
            <dc:creator><![CDATA[Akshay Maheshwari]]></dc:creator>
            <pubDate>Thu, 18 Jun 2020 08:34:15 GMT</pubDate>
            <atom:updated>2020-06-18T08:55:07.823Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C_kQOF5pCTma2clLoSfkBQ.jpeg" /></figure><p>In this post, we’re going to dive deep into one of the most popular and simple machine learning classification algorithms — the<strong> Naive Bayes algorithm, </strong>which is based on the Bayes Theorem for calculating probabilities and conditional probabilities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*7CVYgLqOmoZMUW7JG4m1Iw.png" /></figure><p>Before we jump into the Naive Bayes classifier/algorithm, we need to know the fundamentals of Bayes Theorem, on which it’s based.</p><h3>Bayes Theorem</h3><p>Given a feature vector <em>X=(x1,x2,…,xn)</em> and a class variable <em>Ck</em>, Bayes Theorem states that:</p><p><em>P(Ck|X)=P(X|Ck)P(Ck)/(P(X)), for k=1,2,…,K</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AKBjJx4pQF6owrmTeWvAIQ.jpeg" /></figure><p>We call <em>P(Ck∣X)</em> the posterior probability,<em> P(X∣Ck)</em> the likelihood, <em>P(Ck) </em>the prior probability of a class, and <em>P(X) </em>the prior probability of the predictor. We’re interested in calculating the posterior probability from the likelihood and prior probabilities.</p><p>Using the chain rule, the likelihood <em>P(X∣Ck)</em> can be decomposed as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sqCkSGTCTJb9axbSrNjCGA.png" /></figure><h3>The term “Naive” and feature independence</h3><p>It’s quite tedious to calculate the set of probabilities for all values again and again. Fortunately, with the naive conditional independence assumption, the conditional probabilities are independent of each other.</p><p>In terms of machine learning, we mean to say that the features provided to us are independent and do not affect each other, and this does not happen in real life. The features depend on the occurrence or value of another, which is simply ignored by the Naive Bayes classifier and is hence given the term, <strong>“NAIVE”.</strong></p><p>Thus, by conditional independence, we have:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/972/1*fbusVKSGp7sFJO_7qQHVpw.png" /></figure><p>And the posterior probability can then be written as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*KXfHvlHKbXf1Dn71AoTRNw.png" /></figure><p>since the denominator remains constant for all values.</p><h3>What is a classifier?</h3><p>A classifier is a machine learning model that is used to discriminate different objects based on certain features.</p><h3>Principle of Naive Bayes Classifier:</h3><p>Naive Bayes classifier is a probabilistic machine learning model that’s used for classification task. The crux of the classifier is based on the Bayes theorem.</p><h3>Types of Naive Bayes Classifier:</h3><h3>Multinomial Naive Bayes:</h3><p>This is mostly used for document classification problem, i.e whether a document belongs to the category of sports, politics, technology etc. The features/predictors used by the classifier are the frequency of the words present in the document.</p><h3>Bernoulli Naive Bayes:</h3><p>This is similar to the multinomial naive bayes but the predictors are boolean variables. The parameters that we use to predict the class variable take up only values yes or no, for example if a word occurs in the text or not.</p><h3>Gaussian Naive Bayes:</h3><p>When the predictors take up a continuous value and are not discrete, we assume that these values are sampled from a gaussian distribution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/422/1*L63CtAOTI3gaWNheUbe2IQ.gif" /></figure><p>Since the way the values are present in the dataset changes, the formula for conditional probability changes to,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*4djLsO3wtzCTS_JPa8qbqA.png" /></figure><h3>How Naive Bayes algorithm works?</h3><p>Let’s understand it using an example. Below I have a training data set of weather and corresponding target variable ‘Play’ (suggesting possibilities of playing). Now, we need to classify whether players will play or not based on weather condition. Let’s follow the below steps to perform it.</p><p>Step 1: Convert the data set into a frequency table</p><p>Step 2: Create Likelihood table by finding the probabilities like Overcast probability = 0.29 and probability of playing is 0.64.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/1*qRpx_hKtvZh_ot_kvpoWlA.jpeg" /></figure><p>Step 3: Now, use Naive bayes equation to calculate the posterior probability for each class. The class with the highest posterior probability is the outcome of prediction.</p><p><strong>Problem: </strong>Players will play if weather is sunny. Is this statement is correct?</p><p>We can solve it using above discussed method of posterior probability.</p><p>P(Yes | Sunny) = P( Sunny | Yes) * P(Yes) / P (Sunny)</p><p>Here we have P (Sunny |Yes) = 3/9 = 0.33, P(Sunny) = 5/14 = 0.36, P( Yes)= 9/14 = 0.64</p><p>Now, P (Yes | Sunny) = 0.33 * 0.64 / 0.36 = 0.60, which has higher probability.</p><p>Naive Bayes uses a similar method to predict the probability of different class based on various attributes. This algorithm is mostly used in text classification and with problems having multiple classes.</p><h3>What are the Pros and Cons of Naive Bayes?</h3><p><strong><em>Pros:</em></strong></p><ul><li>It is easy and fast to predict class of test data set. It also perform well in multi class prediction</li><li>When assumption of independence holds, a Naive Bayes classifier performs better compare to other models like logistic regression and you need less training data.</li><li>It perform well in case of categorical input variables compared to numerical variable(s). For numerical variable, normal distribution is assumed (bell curve, which is a strong assumption).</li></ul><p><strong><em>Cons:</em></strong></p><ul><li>If categorical variable has a category (in test data set), which was not observed in training data set, then model will assign a 0 (zero) probability and will be unable to make a prediction. This is often known as “Zero Frequency”. To solve this, we can use the smoothing technique. One of the simplest smoothing techniques is called Laplace estimation.</li><li>On the other side naive Bayes is also known as a bad estimator, so the probability outputs from predict_proba are not to be taken too seriously.</li><li>Another limitation of Naive Bayes is the assumption of independent predictors. In real life, it is almost impossible that we get a set of predictors which are completely independent.</li></ul><h3>Applications of Naive Bayes Algorithms</h3><ul><li><strong>Real time Prediction: </strong>Naive Bayes is an eager learning classifier and it is sure fast. Thus, it could be used for making predictions in real time.</li><li><strong>Multi class Prediction: </strong>This algorithm is also well known for multi class prediction feature. Here we can predict the probability of multiple classes of target variable.</li><li><strong>Text classification/ Spam Filtering/ Sentiment Analysis:</strong> Naive Bayes classifiers mostly used in text classification (due to better result in multi class problems and independence rule) have higher success rate as compared to other algorithms. As a result, it is widely used in Spam filtering (identify spam e-mail) and Sentiment Analysis (in social media analysis, to identify positive and negative customer sentiments)</li><li><strong>Recommendation System: </strong>Naive Bayes Classifier and Collaborative filtering together builds a Recommendation System that uses machine learning and data mining techniques to filter unseen information and predict whether a user would like a given resource or not.</li></ul><h3>Conclusion:</h3><p><strong><em>Naive Bayes algorithms are mostly used in sentiment analysis, spam filtering, recommendation systems etc. They are fast and easy to implement but their biggest disadvantage is that the requirement of predictors to be independent.</em></strong></p><p><strong><em>Thanks for giving your valuable time.</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8c68e5432030" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>