<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Saif Kasmani on Medium]]></title>
        <description><![CDATA[Stories by Saif Kasmani on Medium]]></description>
        <link>https://medium.com/@saifkasmani84?source=rss-4715ebad5e06------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*jexXtKaz6wlQuSXo</url>
            <title>Stories by Saif Kasmani on Medium</title>
            <link>https://medium.com/@saifkasmani84?source=rss-4715ebad5e06------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 19:56:08 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@saifkasmani84/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Azure Data Fundamentals]]></title>
            <link>https://saifkasmani84.medium.com/azure-data-fundamentals-6d26ea5786f6?source=rss-4715ebad5e06------2</link>
            <guid isPermaLink="false">https://medium.com/p/6d26ea5786f6</guid>
            <dc:creator><![CDATA[Saif Kasmani]]></dc:creator>
            <pubDate>Sun, 20 Dec 2020 04:34:29 GMT</pubDate>
            <atom:updated>2020-12-20T04:34:29.979Z</atom:updated>
            <content:encoded><![CDATA[<p>Summary of the basics of Azure Data Fundamentals and what to expect in the certification exam.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/0*mHvIIOqzGMFQj7SM.png" /></figure><p>I have followed Adam Marczak to prepare for the certification exam. His cheat sheets and practice exams are on point.</p><p>Please go here <a href="https://marczak.io/az-900/">https://marczak.io/az-900/</a> for complete review of the AZ-900 certification exam.</p><p>For watching the live explanations please go to :<a href="https://www.youtube.com/channel/UCdmEIMC3LBil4o0tjaTbj0w">https://www.youtube.com/channel/UCdmEIMC3LBil4o0tjaTbj0w</a>.</p><p>Outline of the test weightage and evaluation criteria is provided here by microsoft:</p><iframe src="https://drive.google.com/viewerng/viewer?url=https%3A//query.prod.cms.rt.microsoft.com/cms/api/am/binary/RE3WzVA&amp;embedded=true" width="600" height="780" frameborder="0" scrolling="no"><a href="https://medium.com/media/12eba486644d70ea8357189778093423/href">https://medium.com/media/12eba486644d70ea8357189778093423/href</a></iframe><h3>Skills measured</h3><ul><li>Describe cloud concepts</li><li>Describe core Azure services</li><li>Describe core solutions and management tools on Azure</li><li>Describe general security and network security features</li><li>Describe identity, governance, privacy, and compliance features</li><li>Describe Azure cost management and Service Level Agreements</li></ul><p>Here is a link to schedule the exam and the study material is provided by Microsoft as well:</p><p><a href="https://docs.microsoft.com/en-us/learn/certifications/azure-fundamentals">Microsoft Certified: Azure Fundamentals - Certifications</a></p><p>Azure capabilities can be broken down in 8 main categories:</p><ol><li>Compute Services</li><li>Cloud Storage</li><li>Networking</li><li>App Hosting</li><li>Artificial Intelligence</li><li>Internet of Things</li><li>Integration</li><li>Security</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*effOQUA-qfPHh9Fo.png" /></figure><h3>Microsoft Certifications</h3><p>Microsoft offers certifications on their different products: Azure, Dynamics 365, Microsoft 365, Office, Power Platform, SQL Server, and Windows.</p><p>If we consider only the Azure product, we have 13 certifications grouped into 2 categories:</p><ul><li>Apps and Infrastructure</li><li>Data and IA</li></ul><p>Referring to the following technical job roles:</p><ul><li>Administrator</li><li>AI Engineer</li><li>Data Engineer</li><li>Data Scientist</li><li>Database Administrator</li><li>Developer</li><li>DevOps Engineer</li><li>Security Engineer</li><li>Solutions Architect</li></ul><p>The exam presents several <a href="https://www.microsoft.com/en-us/learning/certification-exams.aspx">question types</a>:</p><ul><li>Multiple choice in different forms: one correct response and three incorrect responses, select correct description, two or more correct responses out of five or more options</li><li>Drag and drop: move the response to the correct description</li><li>Hot area: answer questions (screenshot, picture, or diagram) in a graphical form</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6d26ea5786f6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Matplotlib vs Seaborn library]]></title>
            <link>https://saifkasmani84.medium.com/matplotlib-vs-seaborn-library-22a85799aa01?source=rss-4715ebad5e06------2</link>
            <guid isPermaLink="false">https://medium.com/p/22a85799aa01</guid>
            <dc:creator><![CDATA[Saif Kasmani]]></dc:creator>
            <pubDate>Wed, 09 Dec 2020 20:16:38 GMT</pubDate>
            <atom:updated>2020-12-09T20:16:38.169Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*X5gtiJcwrFGZx_fk" /></figure><p>Data Visualization tools are of great importance in the analytics industry as they give a clear idea of the complex data involved. Python is one of the most popular languages for visualization with its variety of tools.</p><p>Two of Python’s greatest visualization tools are Matplotlib and Seaborn. Seaborn library is basically based on Matplotlib. Here is a detailed comparison between the two:</p><h3>1.Functionality:</h3><p><strong>Matplotlib:</strong> Matplotlib is mainly deployed for basic plotting. Visualization using Matplotlib generally consists of bars, pies, lines, scatter plots and so on.</p><p><strong>Seaborn: </strong>Seaborn, on the other hand, provides a variety of visualization patterns. It uses fewer syntax and has easily interesting default themes. It specializes in statistics visualization and is used if one has to summarize data in visualizations and also show the distribution in the data.</p><h3>2.Handling Multiple Figures:</h3><p><strong>Matplotlib:</strong> Matplotlib has multiple figures can be opened, but need to be closed explicitly. plt.close() only closes the current figure. plt.close(‘all’) would close em all.</p><p><strong>Seaborn:</strong> Seaborn automates the creation of multiple figures. This sometimes leads to OOM (out of memory) issues.</p><h3>3.Visualization:</h3><p><strong>Matplotlib: </strong>Matplotlib is a graphics package for data visualization in Python. It is well integrated with NumPy and Pandas. The pyplot module mirrors the MATLAB plotting commands closely. Hence, MATLAB users can easily transit to plotting with Python.</p><p><strong>Seaborn:</strong> Seaborn is more integrated for working with Pandas data frames. It extends the Matplotlib library for creating beautiful graphics with Python using a more straightforward set of methods.</p><h3>4.Data frames and Arrays</h3><p><strong>Matplotlib: </strong>Matplotlib works with data frames and arrays. It has different stateful APIs for plotting. The figures and aces are represented by the object and therefore plot() like calls without parameters suffices, without having to manage parameters.</p><h3>5.Flexibility:</h3><p><strong>Matplotlib:</strong> Matplotlib is highly customizable and powerful.</p><p><strong>Seaborn:</strong> Seaborn avoids a ton of boilerplate by providing default themes which are commonly used.</p><h3>6.Use Cases:</h3><p><strong>Matplotlib: </strong>Pandas uses Matplotlib. It is a neat wrapper around Matplotlib.</p><p><strong>Seaborn: </strong>Seaborn is for more specific use cases. Also, it is Matplotlib under the hood. It is specially meant for statistical plotting.</p><h3>Which One Should You Use</h3><p>Both the popular visualization tools used in Python have differences in use cases, scalability and many other things. Based on these one should select the best visualization tool for a particular work. If one is doing statistics then Seaborn is a good choice because it has a lot of things suitable for statistical tasks, built-in.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22a85799aa01" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Experience with job search during COVID-19]]></title>
            <link>https://saifkasmani84.medium.com/experience-with-job-search-during-covid-19-3debce061c6f?source=rss-4715ebad5e06------2</link>
            <guid isPermaLink="false">https://medium.com/p/3debce061c6f</guid>
            <dc:creator><![CDATA[Saif Kasmani]]></dc:creator>
            <pubDate>Sat, 05 Dec 2020 01:32:06 GMT</pubDate>
            <atom:updated>2020-12-09T21:13:14.173Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*yOLNORHKKuh9f7Ae" /></figure><p>It can be very challenging to land an interview during these uncertain times. With the holidays around the corner, the responses from recruiters are getting slower. It can get really frustrating but we as fellow job seekers must push through it.</p><p>If we get through this phase which I know we will, there&#39;s a light at the end of the tunnel. I have set a routine for me which allows me to learn a new data science tool or library everyday to keep myself updated with the latest releases/updates etc. Apart from this make sure to take on new projects that you are interested in solving and start working on it. That will test your current skill level and also will help improve your googling skills for code syntax or code in general.</p><p>I found myself demotivated after applying for 20–30 jobs a week and have got no response for 3 weeks straight. To counter this I have taken on a new initiative, doing an Azure fundamentals certification which will raise my value in the current market. I plan to finish this certification within the next week and move on to the next part of the certification which is called Azure Data Science Associate. It is really interesting to keep learning something new and then applying it. Learning the theory is good but applying it sets it in stone.</p><p>After this certification I plan to take up an Azure-related data science project and put it to production as part of my next step.</p><p>I hope this blog reaches anyone who is struggling with their job search due to Covid-19 or the holidays. Don’t give up!</p><p>I bet you’ve heard the saying “You miss 100% of the shots you don’t take” more than enough times. And yes, Wayne Gretzky may have been talking about hockey when he said this, but little did he know he was giving career advice, too.</p><p>Yes, there are times a recruiter or professional contact will reach out to someone with a job opportunity and just place it on the person’s (lucky, lucky) lap. That does happen. But you certainly can’t count on it.</p><p>If you quit applying for jobs, you’re not going to get one. And then you’ll be right back where you started when you decided you wanted to look for a new job in the first place. So keep that personal mission in mind every time you think about quitting this process. You’ve got this!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3debce061c6f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Time Series Analysis: Time-Series Forecasting Machine learning Methods & Models]]></title>
            <link>https://saifkasmani84.medium.com/time-series-analysis-time-series-forecasting-machine-learning-methods-models-caed65b5f8a0?source=rss-4715ebad5e06------2</link>
            <guid isPermaLink="false">https://medium.com/p/caed65b5f8a0</guid>
            <dc:creator><![CDATA[Saif Kasmani]]></dc:creator>
            <pubDate>Thu, 26 Nov 2020 06:07:34 GMT</pubDate>
            <atom:updated>2020-11-26T06:07:34.366Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QNVC5ADm7VMNLwy0Df2OJA.jpeg" /></figure><p>While all the numerous advanced tools and techniques are employed for data analysis such as data science, business intelligence, NLP, classical and quantum machine learning, deep learning, implementation of Artificial Intelligence, and IoT, one of the techniques frequently preferred for analyzing financial data is statistical Time Series Analysis.</p><p>We all must have heard that people are saying that the price of different objects has decreased or increased with time, these different objects could be anything like petrol, diesel, gold, silver, eatable things, etc.</p><p>Also, the rate of interest fluctuates in banks and different for different kinds of loans. What are all this data, how it is useful? These types of data are time-series data that go through analysis for forecasts.</p><h3>Introduction to Time-Series Analysis</h3><p>In this blog, we are going to examine <strong><em>what is time series analysis, its scope in the future, how this can be used in several repetitions of financial data and services, and time series analysis using machine learning</em></strong>.</p><p>In the broad form, it is analyzed to obtain inference what has occurred in the past with the data point series and endeavor to predict what is going to appear in the coming time.</p><p>Now the questions arise “<em>how do people get to know that the price of an object as increased or decreased over time</em>”, they do so <strong><em>by comparing the price of an object over a set of the time period.</em></strong></p><blockquote><em>An ordered set of observations with respect to time periods is a time series. In simple words, a sequential organization of data accordingly to their </em><strong><em>time of occurrence is termed as time series.</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*H-Odk8hh-zpCRnzc" /></figure><p><em>Time-Series Analysis through various mode of data visualization</em></p><p>Time acts as a reference point in relation to the entire procedure. It can be noticed that time-series always depicts a relationship between two variables in which one is time and the other one is any quantitative variable.</p><p>Not necessarily there is an increment in the change of variable with respect to time in the observations, it also exhibited decrement in variable-time observational data. For example, the temperature of a particular area at a particular time increases or decreases accordingly.</p><h3>An overview of Statistical Time Series Analysis</h3><p>A time-series contains sequential data points mapped at a certain successive time duration, it incorporates the methods that attempt to surmise a time series in terms of understanding either the underlying concept of the data points in the time series or suggesting or making predictions. (As talking about making predictions using data and information, read the importance of the process EDA exclusively using pandas profiling).</p><p>Forecasting data using time-series analysis comprises the use of some significant model to forecast future conclusions on the basis of known past outcomes. An example of a restaurant in which prediction is made on the number of customers as when will more customers appear in the restaurant at a specified time duration based on the previous appearance of customers with time.</p><p>Broadly specified time-series models are <strong>Autoregressive</strong> <strong>(AR)</strong>, <strong>Integrated (I)</strong>, <strong>Moving Average(MA)</strong>, and some other models are the combination of these models such as <strong>Autoregressive Moving Average (ARMA)</strong>, and <strong>Autoregressive Integrated Moving Average (ARIMA)</strong> models.</p><p>These models reflect measurements near concurrently in time will be more closely relevant as compared to measurements distant apart.</p><h3>Examples of Time-Series Analysis</h3><p><strong>Consider an example In the financial domain,</strong> the main objective is to recognize trends, seasonal behavior, and correlation through the usage of time series analysis technique and producing filters based on the forecasts, this includes;</p><ol><li><strong>To predict expected utilities- </strong>For the perfect and successively trading, it is necessary to have accurate and reliable future predictions such as asset prices, variation in usage, products in demand in statistical form through market research, and time-series dataset.</li><li><strong>Simulate series- </strong>After getting statistical output data of financial time series, that can be used for creating simulations of future events. It helps us to determine the count of trades, expected trading costs and returns, required financial and technical investment, several risks in trading, etc.</li><li><strong>Presume relationship- </strong>Recognition of the relationship between the time series and other quantities gives us trading signs to improve the existing fashion of trading. For example, to know the spreading of foreign exchange pair and its variation with a proposal, estimated trades can be inferred for a certain period for forecasting a widespread to reduce transaction costs.</li></ol><p>Moreover, consider the simple example of a train steadiness, it includes the Time Series Data model that can be implemented to represent information such as stopping trains. It can be used</p><ol><li>To aggregate over time which train is late at which location, and</li><li>How many passengers were there on the train?</li></ol><p>Furthermore, in order to examine social media hashtags and how frequently a hashtag is utilized in a specified interval of time. More and more, how many responses this particular hashtag used during such an interval.</p><h3>Implementing Time Series Analysis in Machine Learning</h3><p>It is a well-known fact that Machine Learning is a powerful technique in imagining, speech and natural language processing for a huge explicated dataset available, on the other hand, problems based on time series do not have usually interpreted datasets, even as data is collected from various sources so exhibit substantial variations in terms of features, properties, attributes, temporal scales, and dimensionality.</p><p>Time series analysis requires such sorting algorithms that can allow it to learn time-dependent patterns across multiples models different from images and speech. Various machine learning tools such as classification, clustering, forecasting, and anomaly detection depend upon real-world business applications.</p><p>Among various defined applications, <strong><em>discussing here Time series forecasting</em></strong>, it is an important area of machine learning because there are multiple problems involving time components for making predictions. There are <strong><em>multiple models and methods</em></strong> used as approaches for time series forecasting, let’s understand them more clearly;</p><h3>What are the ML Methods in Time-Series Analysis?</h3><ol><li>In <strong><em>the Univariate Time-series Forecasting method</em></strong>, forecasting problems contain only two variables in which one is time and the other is the field we are looking to forecast. For example, if you want to predict the mean temperature of a city for the coming week, now one parameter is time( week) and the other is a city.</li><li>On the other hand, in <strong><em>the Multivariate Time-series Forecasting method</em></strong>, forecasting problems contain multiple variables keeping one variable as time fixed and others will be multiple in parameters.</li></ol><p>Consider the same example, predicting the temperature of a city for the coming week, the only difference would come here now temperature will consider impacting factors such as<em> rainfall and time duration of raining, humidity, wind speed, precipitation, atmospheric pressure, etc, and then the temperature of the city will be predicted accordingly</em>. All these factors are related to temperature and impact it vigorously. (Related blog:<strong><em> Weather Forecasting: How Does Big Data Analytics Magnify it?)</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*s2deAbqfSR0iNB-C" /></figure><p><em>Time-Series Forecasting: Methods and Models in Machine Learning</em></p><h3>What are ML Models in Time-Series Analysis?</h3><ul><li><strong>ARIMA Model: </strong>As mentioned in the above section, it is a combination of three different models itself, AR, MA and I, where</li></ul><ol><li>“AR” reflects the evolving variable of interest is regressed on its own prior values,</li><li>“MA” infers that the regression error is the linear combination of error terms values happened at various stages of time priory, and</li><li>“I” shows the data values are replaced by the difference between their values and the previous values.</li></ol><p>Combinedly “ARIMA” tries to fit the data into the model, and also ARIMA depends on the accuracy over a broad width of time series.</p><ul><li><strong>ARCH/GARCH Model: </strong>Being the extended model of its common version GARCH, Autoregressive Conditional Heteroscedasticity (ARCH) is the most volatile model for time series forecasting, and are well trained for catching dynamic variations of volatility from time series.</li><li><strong>Vector Autoregressive Model or VAR model: </strong>It gives the independencies between various time-series data which as a generalization of the Univariate Autoregression Model.</li><li><strong>LSTM: </strong>Long-short term memory(LSTM) is a deep learning model, it is a kind of Recurrent Neural Network(RNN) to read the sequence dependencies. It enables us to handle long structures during training the dataset and creates predictions according to previous data.</li></ul><h3>Conclusion</h3><p>We can use Time Series for multiple investigations to predict future as circadian rhythms, seasonal behaviors, trends, changes, etc. to interrogate the questions like predicted values, what is leading and lagging behind, connections and association, control, repetitions, and hidden pattern, etc.</p><p>Time series analysis is basically the recording of data at a regular interval of time, which could lead to taking a versed decision, crucial for trade and so have multiple applications such as <strong>Stock Market and Trends analysis, Financial forecasting, Inventory analysis, Census Analysis, Yield prediction, Sales forecasting, etc.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*1mlIcwIKbsdgmw1F" /></figure><p><em>Multiple applications of the Time-Series Analysis</em></p><p>We have discussed so far about time series analysis and its model, its role in finance, and the impact of <strong>machine learning</strong> on time series along with applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=caed65b5f8a0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beginning of my Data Science journey.]]></title>
            <link>https://saifkasmani84.medium.com/beginning-of-my-data-science-journey-b31bad633c62?source=rss-4715ebad5e06------2</link>
            <guid isPermaLink="false">https://medium.com/p/b31bad633c62</guid>
            <dc:creator><![CDATA[Saif Kasmani]]></dc:creator>
            <pubDate>Mon, 23 Nov 2020 03:04:39 GMT</pubDate>
            <atom:updated>2020-11-23T03:04:39.019Z</atom:updated>
            <content:encoded><![CDATA[<p>The first 4 days of my Data Science boot camp has been an extremely immersive experience. My instructor, Rafael Carrasco has been amazing and makes the sessions very interactive and fun.</p><p>As per him no one way is right, there are multiple ways of doing the same thing and you decide what you want to do, i strongly agree. He is really enthusiastic for everyone to ask as many questions as possible and has shown us many ways to ask questions. On the third day it felt like I was taking in too much information to keep up, but now after getting to practice that day’s lesson I feel much more comfortable about it.</p><p>I am really excited to where this journey takes me. But for now i want to enjoy learning everyday much more than the previous day. A challenge to my self in this journey would be to practice more than i did the previous day.</p><p>Why I chose data science? — There are multiple reasons, but let me talk about two. First one was that i was really looking forward to creating something of my own which will be used by a vast majority of the coding community. I want to contribute towards the betterment and more efficient ways to do one thing(Not that it is not efficient enough already). Also i was looking toward learning to develop a prediction algorithm for stocks that aim to predict the daily adjusted closing prices using data from the previous N days. By using the root mean square error (RMSE) and mean absolute percentage error (MAPE) metrics, the lower the value, the better the prediction. I am ‘sciched’ for using the scikit-learn library in python for regression and clustering.</p><p>Given my prior experience with SQL server, i am quite excited to see and test installing the pyodbc package. I love working with data and looking forward to using MongoDB as well if it is included in the syllabus this time. Overall there are nothing but positive vibes i feel from my fellow Cohorts and i’m sure this journey will be a fun one. If nosuccess() == true: print(‘Try Again’) else: print(‘Try Again Harder’)</p><p>Overall the experience with Flatiron school has been extremely positive and academically rewarding. I would definitely recommend others for this boot camp!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b31bad633c62" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What does a Data Analyst do?]]></title>
            <link>https://saifkasmani84.medium.com/what-does-a-data-analyst-do-b5a406430950?source=rss-4715ebad5e06------2</link>
            <guid isPermaLink="false">https://medium.com/p/b5a406430950</guid>
            <dc:creator><![CDATA[Saif Kasmani]]></dc:creator>
            <pubDate>Thu, 12 Nov 2020 20:46:04 GMT</pubDate>
            <atom:updated>2020-11-12T20:46:04.499Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*WU4tgAY5OwFqAfG9eua5Tw.jpeg" /></figure><p>Organizations in all industries increasingly rely on data to make critical business decisions — which new products to develop, new markets to enter, new investments to make, and new (or existing) customers to target. They also use data to identify inefficiencies and other business problems that need to be addressed.</p><p>In these organizations, the job of the data analyst is to assign a numerical value to these important business functions so performance can be assessed and compared over time. But the job involves more than just looking at numbers: An analyst also needs to know <em>how </em>to use data to enable an organization to make more informed decisions.</p><p>These roles are in high demand. IBM has estimated that there will be more than 2.7 million job openings for professionals with data skills by 2020, with nearly 40 percent of jobs in advanced data analytics requiring a master’s degree or higher. Entry-level data analysts receive an average starting annual salary in the $60,000 range; however, success in the role can lead to senior positions with salaries exceeding $135,000.</p><h3>What is Analytics?</h3><p>Analytics brings together theory and practice to identify and communicate data-driven insights that allow managers, stakeholders, and other executives in an organization to make more informed decisions. Experienced data analysts consider their work in a larger context, within their organization and in consideration of various external factors. Analysts are also able to account for the competitive environment, internal and external business interests, and the absence of certain data sets in the data-based recommendations that they make to stakeholders.</p><h3>Types of Data Analytics</h3><p>Four types of data analytics build on each other to bring increasing value to an organization.</p><ul><li><strong>Descriptive analytics </strong>examines what happened in the past: Monthly revenue, quarterly sales, yearly website traffic, and so on. These types of findings allow an organization to spot trends.</li><li><strong>Diagnostic analytics</strong> considers why something happened by comparing descriptive data sets to identify dependencies and patterns. This helps an organization determine the cause of a positive or negative outcome.</li><li><strong>Predictive analytics</strong> seeks to determine likely outcomes by detecting tendencies in descriptive and diagnostic analyses. This allows an organization to take proactive action — like reaching out to a customer who is unlikely to renew a contract, for example.</li><li><strong>Prescriptive analytics</strong> attempts to identify what business action to take. While this type of analysis brings significant value in the ability to address potential problems or stay ahead of industry trends, it often requires the use of complex algorithms and advanced technology such as machine learning.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/734/1*phPXu460Rp8oueSg8aCWTg.png" /></figure><p>In a 2016 survey of more than 2,000 business executives, the consultancy PwC found that organizations find descriptive analytics to be insufficient for informed, data-driven decision making. As such, diagnostic and predictive analytics are increasingly important to organizations.</p><h3>Key Responsibilities of a Data Analyst</h3><p>The answer to the question “What does a data analyst do?” will vary depending on the type of organization and the extent to which a business has adopted data-driven decision-making practices. Generally speaking, though, the responsibilities of a data analyst typically include the following:</p><ul><li>Designing and maintaining data systems and databases; this includes fixing coding errors and other data-related problems.</li><li>Mining data from primary and secondary sources, then reorganizing said data in a format that can be easily read by either human or machine.</li><li>Using statistical tools to interpret data sets, paying particular attention to trends and patterns that could be valuable for diagnostic and predictive analytics efforts.</li><li>Demonstrating the significance of their work in the context of local, national, and global trends that impact both their organization and industry.</li><li>Preparing reports for executive leadership that effectively communicate trends, patterns, and predictions using relevant data.</li><li>Collaborating with programmers, engineers, and organizational leaders to identify opportunities for process improvements, recommend system modifications, and develop policies for data governance.</li><li>Creating appropriate documentation that allows stakeholders to understand the steps of the data analysis process and duplicate or replicate the analysis if necessary.</li></ul><h3>Most Valuable Skills for Data Analysts</h3><p>Effective data analysts possess a combination of technical skills and leadership skills.</p><p>Technical skills include knowledge of database languages such as SQL, R, or Python; spreadsheet tools such as Microsoft Excel or Google Sheets; and data visualization software such as Tableau or Qlik. Mathematical and statistical skills are also valuable to help gather, measure, organize, and analyze data.</p><p>Leadership skills prepare a data analyst to complete decision-making and problem-solving tasks. These abilities allow analysts to think strategically about the information that will help stakeholders make data-driven business decisions and to communicate the value of this information effectively. For example, project managers rely on data analysts to track the most important metrics for their projects, to diagnose problems that may be occurring, and to predict how different courses of action could address a problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_SEz74My_geDB3r1iSGeOQ.png" /></figure><h3>Data Analysis vs. Data Science vs. Business Analysis</h3><p>The difference in what a data analyst does as compared to a business analyst or a data scientist comes down to how the three roles use data.</p><ul><li>The <strong>data analyst</strong> serves as a gatekeeper for an organization’s data so stakeholders can understand data and use it to make strategic business decisions. It is a technical role that requires an undergraduate degree or master’s degree in analytics, computer modeling, science, or math.</li><li>The <strong>business analyst </strong>serves in a strategic role focused on using the information that a data analyst uncovers to identify problems and propose solutions. These analysts typically earn a degree in a major such as business administration, economics, or finance.</li><li>The <strong>data scientist</strong> takes the data visualizations created by data analysts a step further, sifting through the data to identify weaknesses, trends, or opportunities for an organization. This role also requires a background in math or computer science, along with some study or insight into human behavior to help make informed predictions.</li></ul><p>At startups and other small organizations, it is not uncommon for a data analyst to take on some of the predictive modeling or decision-making responsibilities that may otherwise be assigned to a data scientist.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b5a406430950" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Science Careers]]></title>
            <link>https://saifkasmani84.medium.com/data-science-careers-e1f97226d263?source=rss-4715ebad5e06------2</link>
            <guid isPermaLink="false">https://medium.com/p/e1f97226d263</guid>
            <dc:creator><![CDATA[Saif Kasmani]]></dc:creator>
            <pubDate>Mon, 07 Sep 2020 21:29:04 GMT</pubDate>
            <atom:updated>2020-09-07T21:29:04.572Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AbjtiBJ7_DCEr8VPJ7KIcw.jpeg" /></figure><p>Today’s economy is leaning more toward analytics — companies have been collecting data for many years. According to LinkedIn, there is a massive demand for people who can mine and interpret data. These are the data scientists.</p><h3>Who is a Data Scientist?</h3><p>Data scientists are a mix of mathematicians, trend-spotters, and computer scientists. The data scientist’s role is to decipher large volumes of data and carry out further analysis to find trends in the data and gain a deeper insight into what it all means. Data scientists operate between the business and IT worlds and drive industries by analyzing complex datasets to tease out insights that companies can leverage into actions.</p><h3>Skills Required For Data Scientist</h3><p>If you’ve been wondering how to start a <a href="https://www.simplilearn.com/tutorials/data-science-tutorial/data-science-career">c</a>areer in data science, you’ll need hard skills like analysis, machine learning, statistics, Hadoop, etc. Still, you’ll also excel in this type of role if you excel at critical thinking, persuasive communications, and are a great listener and problem solver.</p><h3>Data Science Roles</h3><p>To name a few, some of the most common job titles for data scientists include:</p><ol><li>Business Intelligence Analyst</li></ol><ul><li>A BI analyst uses data to help figure out market and business trends by analyzing data to develop a clearer picture of where the company stands.</li></ul><p>2. Data Mining Engineer</p><ul><li>The data mining engineer examines not only the data for their own business but also that of third parties. In addition to analyzing data, a data mining engineer will create sophisticated algorithms to help analyze the data further.</li></ul><p>3. Data Architect</p><ul><li>Data architects work closely with users, system designers, and developers to create blueprints that data management systems use to centralize, integrate, maintain, and protect data sources.</li></ul><p>4. Data Scientist</p><ul><li>Data scientists begin by translating a business case into an analytics agenda, developing hypotheses, and understanding data — as well as exploring patterns to measure what impact they will have on businesses. They also find and choose algorithms to help further analyze data. They use business analytics not only to explain what effect the data is going to have on a company in the future but can also help devise solutions that will help the company to move forward.</li></ul><p>5. Senior Data Scientist</p><ul><li>A senior data scientist can anticipate what a business’s future needs will be. Apart from gathering data, they also analyze it thoroughly to resolve highly complex business problems efficiently. Through their experience, they can not only design but drive forward the creation of new standards, as well as create ways to use statistical data and also develop tools to help further analyze the data.</li></ul><h3>Data Scientist Job Market</h3><p>With millions of worldwide job openings in Big Data, the role of a data scientist has become the hottest job of the decade. In today’s data-based world, companies are using the insights that data scientists provide to stay one step ahead of their competition while keeping overhead costs low. Big names like Oracle, Apple, Microsoft, Booz Allen Hamilton, State Farm, Walmart, and more all regularly have job postings for data scientists.</p><h3>My Data Science Journey with Flatiron school</h3><p>Being a student of the Full time online data science course with Flatiron school has given me practical knowledge of how to apply myself using the tools that they’ve taught the students during the boot camp.</p><p>It has been an incredible journey and truly changed my perspective of how i look at data. The instructors were incredibly knowledgeable and helpful throughout the course. Really excited to see what challenges lie in the career path and looking forward to accomplish my goals.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e1f97226d263" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Covid-19 Detection Using Deep Learning]]></title>
            <link>https://saifkasmani84.medium.com/covid-19-detection-using-deep-learning-8f5354e5620a?source=rss-4715ebad5e06------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f5354e5620a</guid>
            <dc:creator><![CDATA[Saif Kasmani]]></dc:creator>
            <pubDate>Mon, 07 Sep 2020 21:16:29 GMT</pubDate>
            <atom:updated>2020-09-07T21:16:29.972Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>Implementing AI based models and Flask app to detect COVID-19 in Chest X-rays and CT Scans using four Deep Learning Algorithms: VGG16, ResNet50, InceptionV3 and Xception</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ccBYODOKp8s-lfyjDOFdRg.jpeg" /><figcaption>Fig 1: Novel Coronavirus disease 2019</figcaption></figure><p><strong>COVID-19</strong>, or more commonly known as the <strong>Novel Coronavirus disease</strong> is a highly infectious disease that appeared in China towards the end of 2019. This disease is caused by <a href="https://en.wikipedia.org/wiki/Severe_acute_respiratory_syndrome_coronavirus_2"><strong>SARS-CoV-2</strong></a>, a virus that belongs to the large family of coronaviruses. The disease first originated in <strong>Wuhan, China</strong> in <strong>December 2019</strong> and soon became a global pandemic, spreading to more than <strong>213</strong> countries.</p><p>The most common <strong>symptoms</strong> of COVID-19 are fever, dry cough, and tiredness. Other symptoms that people may experience include aches, pains, or difficulty in breathing. Most of these symptoms show signs of respiratory infections and lung abnormalities which can be detected by <strong>radiologists</strong>.</p><p>Thus, it is possible to use <strong>Machine Learning</strong> algorithms to detect the disease from images of <strong>Chest X-rays</strong> and <strong>CT scans</strong>. Automated applications can be created to help support radiologists. This article is an attempt to use four Deep Learning algorithms, namely: <strong>VGG16, ResNet50, InceptionV3</strong> and <strong>Xception</strong>.</p><h3>The Dataset</h3><p>The dataset for the project was gathered from two open source Github repositories:</p><ol><li><strong>Chest X-ray</strong> images (1000 images) were obtained from: <a href="https://github.com/ieee8023/covid-chestxray-dataset">https://github.com/ieee8023/covid-chestxray-dataset</a></li><li><strong>CT Scan</strong> images (750 images) were obtained from: <a href="https://github.com/UCSD-AI4H/COVID-CT/tree/master/Data-split">https://github.com/UCSD-AI4H/COVID-CT/tree/master/Data-split</a></li></ol><p>Four algorithms: VGG16, ResNet50, InceptionV3 and Xception were trained separately on Chest X-rays and CT Scans, giving us a total of 8 deep learning models. <strong>80%</strong> of the images were used for <strong>training</strong> the models and the remaining <strong>20%</strong> for <strong>testing </strong>the accuracy of the models.</p><h3>Building the Model</h3><p>I first added <strong>3 custom layers</strong> to the pretrained models so that they can be trained on our dataset. The code for adding custom layers, for example, to the <strong>ResNet50</strong> model is shown below. The code for the rest of the models remains the same. One just needs to change ResNet50 in the first line to the name of the desired model.</p><pre>res = ResNet50(weights=&quot;imagenet&quot;, include_top=False, input_tensor=Input(shape=(224, 224, 3)))<br>outputs = res.output<br>outputs = Flatten(name=&quot;flatten&quot;)(outputs)<br>outputs = Dropout(0.5)(outputs)<br>outputs = Dense(2, activation=&quot;softmax&quot;)(outputs)<br>model = Model(inputs=res.input, outputs=outputs)<br>for layer in res.layers:<br>  layer.trainable = False<br>model.compile(loss=&#39;categorical_crossentropy&#39;, optimizer=&#39;adam&#39;, metrics=[&#39;accuracy&#39;])</pre><p>The images in the dataset were of different sizes. Thus, I needed to resize them to a fixed size before they can be fed to the deep learning models for training. I resized the images to a size of <strong>224 x 224 px </strong>which is considered to be the ideal size for the ResNet50 model. Therefore, I added the input tensor of shape <strong>(224, 224 , 3) </strong>to the pretrained ResNet50 model, 3 being the number of channels.</p><p>Next, I added a <strong>Flatten</strong> layer to flatten all our features and a <strong>Dropout</strong> layer to overcome <strong>overfitting</strong>. Finally, I added the <strong>Dense</strong> output layer using <strong>softmax function</strong> as the activation function. Since the first half of the model is already pretrained, the trainable attribute of the previous layers was set to <strong>False</strong>. Finally, I compiled the model with the <strong>adam optimizer</strong> and using <strong>categorical crossentropy</strong> as the <strong>loss</strong> <strong>function</strong>.</p><h3>Training the Model</h3><p>I first defined an <strong>Image Data Generator</strong> to train the models at modified versions of the images, such as at different angles, flips, rotations or shifts.</p><pre>train_aug = ImageDataGenerator(rotation_range=20, width_shift_range=0.2, height_shift_range=0.2, horizontal_flip=True)</pre><p>Next, training of the model was performed, with all the required parameters specified as follows:</p><pre>history = model.fit(train_aug.flow(X_train, y_train, batch_size=32),validation_data=(X_test,y_test),validation_steps=len(X_test) / 32, steps_per_epoch=len(X_train) / 32, epochs=500)</pre><p>As you can see, I have trained the model for <strong>500 epochs</strong> with a batch size of <strong>32 images</strong>.</p><h3>Making Predictions</h3><p>Predictions were generated by running the trained models on images of the test set. The predictions for the first 10 images of the dataset were plotted as shown below:</p><pre>y_pred = model.predict(X_test, batch_size=batch_size)<br>prediction=y_pred[0:10]<br>for index, probability in enumerate(prediction):<br>  if probability[1] &gt; 0.5:<br>    plt.title(&#39;%.2f&#39; % (probability[1]*100) + &#39;% COVID&#39;)<br>  else:<br>    plt.title(&#39;%.2f&#39; % ((1-probability[1])*100) + &#39;% NonCOVID&#39;)<br>  plt.imshow(X_test[index])<br>  plt.show()</pre><p>The following snippet shows the plots of first 10 predictions of Chest X-rays:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/248/1*ELE8oOPwDPvHWG1foMnOsQ.png" /><figcaption>Fig 2: Visualizing first 10 predictions of Chest X-rays</figcaption></figure><h3>Evaluation &amp; Results of Machine Learning</h3><p>Following are a few important results and plots that help estimate the accuracy of the models and get insights their performance.</p><h3>Sample output of test set images</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/544/1*FgSbo0S7JoXyfQ0qqISCNQ.png" /><figcaption>Fig 3: Sample output of test set images</figcaption></figure><h3>Classification Report</h3><p>Classification Reports of VGG16 model for Chest X-rays and CT scans</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/379/1*BgnBLxmUDNATQwx2AagggA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/382/1*b7muWfgpkEcisqg3eoTd0A.png" /><figcaption>Fig 4: Classification Reports of VGG16 model for Chest X-rays (Left) and CT scans (Right)</figcaption></figure><p>Classification Reports of ResNet50 model for Chest X-rays and CT scans</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/380/1*5Boknz7vBHM0KWS-0aZJjw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/380/1*tMVCLWpONTJ7kJUCcwuOSQ.png" /><figcaption>Fig 5:Classification Reports of ResNet50 model for Chest X-rays (Left) and CT scans (Right)</figcaption></figure><p>Classification Reports of InceptionV3 model for Chest X-rays and CT scans</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/381/1*DqyM1EOU8gt_-nUxhIkzkA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/381/1*5_d_8j3dKPr-ZcwS3Pqlng.png" /><figcaption>Fig 6: Classification Reports of InceptionV3 model for Chest X-rays (Left) and CT scans (Right)</figcaption></figure><p>Classification Reports of Xception model for Chest X-rays and CT scans</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/381/1*bel69pm9RgnyVs_ZT0v_QQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/380/1*T6VkJMWnFxoHJgubs4cOxA.png" /><figcaption>Fig 7: Classification Reports of Xception model for Chest X-rays (Left) and CT scans (Right)</figcaption></figure><h3>Confusion Matrix</h3><p>Confusion Matrix of VGG16 model for Chest X-rays and CT scans</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/373/1*0Sq8VU47Ta1gVsYA3uSxDg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/373/1*sxxvWwXAKrPK2IcmzpTfZg.png" /><figcaption>Fig 8: Confusion Matrix of VGG16 model for Chest X-rays (Left) and CT scans (Right)</figcaption></figure><p>Confusion Matrix of ResNet50 model for Chest X-rays and CT scans</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/380/1*j3LCp9pRUd-fTJ6C1cI-sA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/387/1*1CLB_Xel5-lHKULrIZXbhw.png" /><figcaption>Fig 9: Confusion Matrix of ResNet50 model for Chest X-rays (Left) and CT scans (Right)</figcaption></figure><p>Confusion Matrix of InceptionV3 model for Chest X-rays and CT scans</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/372/1*H0shL5yf9x9h7_tEBZZNsg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/373/1*mnfhxhfgM-3F0EGrVVhUxg.png" /><figcaption>Fig 10: Confusion Matrix of InceptionV3 model for Chest X-rays (Left) and CT scans (Right)</figcaption></figure><p>Confusion Matrix of Xception model for Chest X-rays and CT scans</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/374/1*pzZfrJ4FRHonEBk3s49QIg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/374/1*Y5J5_uGc4LGILsT_LXyzLA.png" /><figcaption>Fig 11: Confusion Matrix of Xception model for Chest X-rays (Left) and CT scans (Right)</figcaption></figure><h3>Building the Flask App</h3><p>Now, I used the code created by me so far to build a Flask app by pasting particular segments of my code in certain flask functions. For example, the code for using the four models to generate predictions of Chest X-rays looks like:</p><pre>def uploaded_chest():<br>   resnet_chest = load_model(&#39;models/resnet_chest.h5&#39;)<br>   vgg_chest = load_model(&#39;models/vgg_chest.h5&#39;)<br>   inception_chest = load_model(&#39;models/inceptionv3_chest.h5&#39;)<br>   xception_chest = load_model(&#39;models/xception_chest.h5&#39;)   image = cv2.imread(&#39;./flask app/assets/images/upload_chest.jpg&#39;) <br>   # read file <br>   image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) <br>   # arrange format as per keras<br>   image = cv2.resize(image,(224,224))<br>   image = np.array(image) / 255<br>   image = np.expand_dims(image, axis=0)<br>   <br>   resnet_pred = resnet_chest.predict(image)<br>   probability = resnet_pred[0]<br>   print(&quot;Resnet Predictions:&quot;)<br>   if probability[0] &gt; 0.5:<br>      resnet_chest_pred = str(&#39;%.2f&#39; % (probability[0]*100) + &#39;% COVID&#39;) <br>   else:<br>      resnet_chest_pred = str(&#39;%.2f&#39; % ((1-probability[0])*100) + &#39;% NonCOVID&#39;)<br>   print(resnet_chest_pred)   vgg_pred = vgg_chest.predict(image)<br>   probability = vgg_pred[0]<br>   print(&quot;VGG Predictions:&quot;)<br>   if probability[0] &gt; 0.5:<br>      vgg_chest_pred = str(&#39;%.2f&#39; % (probability[0]*100) + &#39;% COVID&#39;) <br>   else:<br>      vgg_chest_pred = str(&#39;%.2f&#39; % ((1-probability[0])*100) + &#39;% NonCOVID&#39;)<br>   print(vgg_chest_pred)   inception_pred = inception_chest.predict(image)<br>   probability = inception_pred[0]<br>   print(&quot;Inception Predictions:&quot;)<br>   if probability[0] &gt; 0.5:<br>      inception_chest_pred = str(&#39;%.2f&#39; % (probability[0]*100) + &#39;% COVID&#39;) <br>   else:<br>      inception_chest_pred = str(&#39;%.2f&#39; % ((1-probability[0])*100) + &#39;% NonCOVID&#39;)<br>   print(inception_chest_pred)xception_pred = xception_chest.predict(image)<br>   probability = xception_pred[0]<br>   print(&quot;Xception Predictions:&quot;)<br>   if probability[0] &gt; 0.5:<br>      xception_chest_pred = str(&#39;%.2f&#39; % (probability[0]*100) + &#39;% COVID&#39;) <br>   else:<br>      xception_chest_pred = str(&#39;%.2f&#39; % ((1-probability[0])*100) + &#39;% NonCOVID&#39;)<br>   print(xception_chest_pred)<br>   return render_template(&#39;results_chest.html&#39;,resnet_chest_pred=resnet_chest_pred,vgg_chest_pred=vgg_chest_pred,inception_chest_pred=inception_chest_pred,xception_chest_pred=xception_chest_pred)</pre><h3>Screenshots of Flask App</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bq1_RQNVoPE_7vzRBMmP9g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/1*Hyu_-gvtNRttveebEA9iIQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/1*Hyu_-gvtNRttveebEA9iIQ.png" /></figure><h3>Conclusion</h3><p>In conclusion, I would like to throw light on the fact that the analysis has been done on a limited data set and that the results are preliminary. Medical validations have not been done on the approach and hence the results might differ from those observed in practical use cases.</p><p>In future, I plan to improve the performance of the models by training them on more images and possibly including other factors like age, nationality, gender, etc. Furthermore, I encourage the readers of this article to experiment with the code on their own to increase the precision of the model.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f5354e5620a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pneumonia Detection using Deep Learning]]></title>
            <link>https://saifkasmani84.medium.com/pneumonia-detection-using-deep-learning-c60c0b264270?source=rss-4715ebad5e06------2</link>
            <guid isPermaLink="false">https://medium.com/p/c60c0b264270</guid>
            <dc:creator><![CDATA[Saif Kasmani]]></dc:creator>
            <pubDate>Wed, 19 Aug 2020 16:53:05 GMT</pubDate>
            <atom:updated>2020-08-19T16:53:05.844Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1IfeWJbl6GvFYqYpEPzzjQ.png" /><figcaption>Functioning of CNN</figcaption></figure><p><strong>Goal: </strong>Build a deep learning model that can classify whether a given patient has pneumonia, given a chest x-ray image.</p><p><strong>What is Pneumonia?</strong></p><p>Pneumonia is an inflammatory condition of the lung affecting primarily the small air sacs known as alveoli.</p><p>Symptoms typically include some combination of productive or dry cough, chest pain, fever and difficulty breathing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/0*vTbdPJe0MNdf4cbC" /></figure><p><strong>Business Case: </strong>Improve healthcare delivery and increase access to medical imaging expertise in parts of the world where access to skilled radiologists is limited.</p><p><strong>Why is this model important?</strong></p><ul><li>Over 150 million people get infected with pneumonia on an annual basis especially children under 5 years old .</li><li>In such regions, the problem can be further aggravated due to the lack of medical resources and personnel.</li><li>For example, in Africa’s 57 nations, a gap of 2.3 million doctors and nurses exist. For these populations, accurate and fast diagnosis means everything. It can guarantee timely access to treatment and save much needed time and money for those already experiencing poverty.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*h2qJYFdj5oLTfXRT" /></figure><p><strong>Data:</strong></p><p>In the data i was given, there are 5857 total images out of which Pneumonia cases account for 73%.</p><p>The data set can be found here: <a href="https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia">https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cXVHUvNANxsvoR5r" /></figure><p><strong>Preprocessing images:</strong></p><ul><li>Some images were in grayscale and majority of them were in 3-channels, hence converted all images to 3-channels.</li><li>Normalize pixel values, resize all images to 224x224 size.</li><li>Data Augmentation(imgaug): Generate samples of undersampled class to try and balance overall distribution.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/469/0*3Ef4uLAgeramtxpV" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/469/0*Jg-qhugl5ctKh6Ie" /></figure><p><strong>Models used:</strong></p><ul><li>Convolutional Neural Network</li><li>Transfer learning (VGG16 architechture)</li></ul><p><strong>Results of CNN Model:</strong></p><ul><li>Accuracy: 87.5 %</li><li>Precision: 75%</li><li>Recall: 100%</li><li>F1 score: 85%</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/0*naukP_PqWHkPyAR7" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*SzjvKMgpceWv0oM1" /></figure><p><strong>Results of Transfer Learning — VGG16 Model:</strong></p><ul><li>Accuracy: 0.740385</li><li>Precision: 1.000000</li><li>Recall: 0.706522</li><li>F1 score: 0.828025</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/0*KZQH-_B9_Nr-i9ED" /></figure><p><strong>Which model do i prefer and why?</strong></p><ul><li>The CNN model outperforms the transfer learning model here, but in this scenario my goal was to minimize the False Negative rate.</li><li>In the transfer learning VGG16 model there are 0 False Negatives, hence i prefer this model.</li><li>I prefer having 0 False Negatives over accuracy as in this scenario or any other medical scenario it is crucial to have the least or no FN rate as the patient might have a disease and the model predicted it to be normal which will put lives at risk and that is unacceptable.</li></ul><p><strong>Future Work:</strong></p><ul><li>If pneumonia detected, further classify if viral or bacterial pneumonia.</li><li>Complete the flask production application for the model.</li><li>Improve current model.</li><li>Explore capsule network classification.</li><li>Use deep learning for COVID-19 detection using Darknet-19 model.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c60c0b264270" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Telecom Churn Analysis]]></title>
            <link>https://saifkasmani84.medium.com/telecom-churn-analysis-8f982807574f?source=rss-4715ebad5e06------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f982807574f</guid>
            <category><![CDATA[xgboost]]></category>
            <category><![CDATA[gradient-boosting]]></category>
            <category><![CDATA[predictive-modeling]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[eda]]></category>
            <dc:creator><![CDATA[Saif Kasmani]]></dc:creator>
            <pubDate>Wed, 15 Jul 2020 04:32:31 GMT</pubDate>
            <atom:updated>2020-07-15T04:32:31.684Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*6GaZtcKYBESxJDpCikaYug.jpeg" /></figure><p><strong>Project: </strong>Predicting churn for a telecom company so it can can effectively focus a customer retention marketing program (e.g. a special offer) or improve certain aspects based on the model to the subset of clients which are most likely to change their carrier. Therefore, the “churn” column is chosen as target and the following predictive analysis is a supervised classification problem.</p><p><strong>What is Churn?</strong></p><p>The churn rate is the percentage of subscribers to a service who discontinue their subscriptions to the service within a given time period.</p><p>For a company to expand its clientele, its growth rate, as measured by the number of new customers, must exceed its churn rate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*lqDCZ5VcejFDI1o4" /></figure><p><strong>Why is churn so important?</strong></p><p>Churn is important because it directly affects your service’s profitability. It is common to assume that the profitability of a service is directly related to the growth of its customer base. That might lead business owners to imply that in order to grow their customer base, the rate of acquiring new customers must exceed the churn rate.</p><p><strong>Data: </strong>Churn in telecom’s dataset which can be found here — <a href="https://www.kaggle.com/becksddf/churn-in-telecoms-dataset">https://www.kaggle.com/becksddf/churn-in-telecoms-dataset</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6qWgBWTovHDIzrT9wC1FeA.png" /><figcaption>Overview of the data set</figcaption></figure><p><strong>Columns and datatypes:</strong></p><pre>state                      object<br>account length              int64<br>area code                   int64<br>phone number               object<br>international plan         object<br>voice mail plan            object<br>number vmail messages       int64<br>total day minutes         float64<br>total day calls             int64<br>total day charge          float64<br>total eve minutes         float64<br>total eve calls             int64<br>total eve charge          float64<br>total night minutes       float64<br>total night calls           int64<br>total night charge        float64<br>total intl minutes        float64<br>total intl calls            int64<br>total intl charge         float64<br>customer service calls      int64<br>churn                        bool</pre><p><strong>Data Cleaning : </strong>We can see that the columns “state”, “international plan”, “voice mail plan” and “churn” have String values. The latter three seem to have just the values “yes” or “no” and are therefore converted to 1 and 0 respectively.</p><p>The “state” column is converted using the LabelEncoder, which replaces each unique label with a unique integer. In this case, a label encode is used instead of dummy variables because of the many distinct values,.</p><p>The “phone number” column is removed, because every customer has its own phone number.</p><pre>df = df.drop([“phone_number”], axis=1)</pre><p>preprocess_data function:</p><pre>def preprocess_data(df):<br> pre_df = df.copy()<br> <br> # Replace the spaces in the column names with underscores<br> pre_df.columns = [s.replace(“ “, “_”) for s in pre_df.columns]<br> <br> # convert string columns to integers<br> pre_df[“international_plan”] = pre_df[“international_plan”].apply(lambda x: 0 if x==”no” else 1)<br> pre_df[“voice_mail_plan”] = pre_df[“voice_mail_plan”].apply(lambda x: 0 if x==”no” else 1)<br> le = LabelEncoder()<br> le.fit(pre_df[‘state’])<br> pre_df[‘state’] = le.transform(pre_df[‘state’])<br> <br> return pre_df, le</pre><p>Checking value counts for churn:</p><pre>pre_df[‘churn’].value_counts()</pre><pre>False    2850<br>True      483</pre><p>We can see that we have clearly more samples for customers without churn than for customers with churn. So we have a class imbalance for the target variable which could lead to predictive models which are biased towards the majority (i.e. no churn). In order to deal with this issue we will investigate into the use of oversampling when building the models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/371/0*ZY2F9FqYQuZQ1jW9" /></figure><p><strong>EDA analysis with questions:</strong></p><p><strong>Q1) Do we see an increase in churn rate with more customer service calls?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/772/0*V89AF2iTAji1eT4t" /></figure><p>We see an increase in customer churn rate in increasing number of customer service calls made by the customer.<br>Some might be repeatedly calling for a fix and switched carriers when their issue was not fixed.<br>Was there an issue with service quality, coverage etc?</p><p>It will be interesting to investigate the data on what type of issues the customers called in for in the customer service calls.</p><p><strong>Q2) How does account length matter?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/793/0*3mE3eKUPtAH4vfRE" /></figure><p>Significant considerable maximum churn is observed in account length bracket of 75 to about 100 weeks.</p><p><strong>Q3) Monthly/Yearly Charge vs Churn Rate</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/0*EvAP83Ow4AeRmx_n" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/0*r2w6BS9dMoxHhDQY" /></figure><p>In this we see that the customers who churned were paying more per month then the customers who did not churn. They were paying about 10–15 dollars extra a month than the customers who did not churn. Churn rate increased with increase in Total Charge in a day. Approximately they paid around 100–150 more than the customers who did not churn.</p><p><strong>Q4) How does international plan affect churn rate?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/772/0*X_7Dg13Hq0fr9KdF" /></figure><p>We can see only a few customers with international plan. But in those few, there is a significant churn rate — slightly less than 50%.</p><p>In the data we were given, that may imply that the customers who left were not happy with the international plan charges.</p><p><strong>Model:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/0*O3yRAgn2M5CRX8CH" /></figure><p>The following models were tried with this dataset:</p><ul><li>LogisticRegression</li><li>XGBClassifier</li><li>MultinomialNB</li><li>AdaBoostClassifier</li><li>KNeighborsClassifier</li><li>GradientBoostingClassifier</li><li>ExtraTreesClassifier</li><li>DecisionTreeClassifier</li></ul><p>Out of all these the best performers were GradientBoostingClassifier and XGBClassifier.</p><p>Let us compare the models before and after we use Synthetic Minority Over-sampling Technique (SMOTE).</p><p>Note — We do smote only on our training data</p><p><strong>Gradient Boosting Classifier:</strong></p><pre># Gradient Boosting Classifier without smote</pre><pre>clf = GradientBoostingClassifier()<br>clf.fit(xtrain, ytrain)<br>y_test_preds = clf.predict(xtest)</pre><pre>report = classification_report(ytest, y_test_preds)<br>print(report)</pre><pre>               precision    recall  f1-score   support<br><br>       False       0.96      0.98      0.97       713<br>        True       0.89      0.74      0.81       121<br><br>    accuracy                           0.95       834<br>   macro avg       0.92      0.86      0.89       834<br>weighted avg       0.95      0.95      0.95       834</pre><pre># Gradient Boosting Classifier with smote</pre><pre>clf = GradientBoostingClassifier()<br>clf.fit(X_train_smote, y_train_smote)<br>y_test_pred_smote = clf.predict(xtest)<br>print(“accuracy_score”,accuracy_score(ytest, y_test_pred_smote))<br>print(“auc”,roc_auc_score(ytest, y_test_pred_smote))</pre><pre>accuracy_score 0.8920863309352518<br>auc 0.8099405375957714</pre><pre>report = classification_report(ytest, y_test_pred_smote)<br>print(report)</pre><pre>               precision    recall  f1-score   support<br><br>       False       0.95      0.93      0.94       713<br>        True       0.61      0.69      0.65       121<br><br>    accuracy                           0.89       834<br>   macro avg       0.78      0.81      0.79       834<br>weighted avg       0.90      0.89      0.89       834</pre><p><strong>XGBoost Classifier:</strong></p><pre># XGBoost without smote</pre><pre>clf = xgb.XGBClassifier(max_depth=7, n_estimators=200, colsample_bytree=0.8, <br>subsample=0.8, nthread=10, learning_rate=0.1)<br>clf.fit(xtrain, ytrain)<br>pred = clf.predict(xtest)<br>print(“accuracy_score”,accuracy_score(ytest, pred))<br>print(“auc”,roc_auc_score(ytest, pred))<br>xg = [clf.__class__,accuracy_score(ytest, pred),roc_auc_score(ytest, pred)]</pre><pre>accuracy_score 0.947242206235012<br>auc 0.8593534477762451</pre><pre># XGBoost with smote</pre><pre>clf_xg = xgb.XGBClassifier(max_depth=7, n_estimators=200, colsample_bytree=0.8, <br>                        subsample=0.8, nthread=10, learning_rate=0.1)<br>clf_xg.fit(xtrain, ytrain)<br>y_test_pred_smote_xg = clf_xg.predict(xtest)<br>print(&quot;accuracy_score&quot;,accuracy_score(ytest, y_test_pred_smote_xg))<br>print(&quot;auc&quot;,roc_auc_score(ytest, y_test_pred_smote_xg))</pre><pre>accuracy_score 0.947242206235012<br>auc 0.8593534477762451</pre><p>I chose XGBoost Classifier as my final model and here are the feature importances of the model.</p><pre>feat_importances = pd.Series(clf_xg.feature_importances_, index=x.columns)<br>feat_importances.nlargest(20).plot(kind=’barh’, figsize=(13,8))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/873/1*nOUgOmTsP7_gtSXFgeYQ1A.png" /></figure><p>From these feature importances i took the 4 most important features which are:</p><ul><li>International Plan</li><li>Voice Mail Plan</li><li>Customer Service Calls</li><li>Total Day Charge</li></ul><p>It would be great to know every single customer who will churn, but how much insight would that information really bring? And how would you know what to focus on if you wanted to keep them and how much you could spend to keep them before having them as a customer turned into a loss?</p><p><strong>Recommendations based on the model:</strong></p><p>1) One of the most important predictors for the Model is the number of customer service calls. This might imply that the company should improve its customer service and solve the issues of customers calling in repeatedly.</p><p>2) Another important feature is the total day minutes and the total day charge. The company could try to either lower its charge per minute for clients, which have many day minutes or it could offer flat rates for calls.</p><p>3) I would suggest the Telecom company to present special offers according to the customers life-time value(Account-Length).</p><p>4) Follow-up with customers using international plan since almost 50% of the total customers with the plan left. Investigate the reason.</p><p><strong>Future Work:</strong></p><ul><li>Get data on payment methods.</li><li>See if churn rate is affected by special promotions of other carriers.</li><li>Internet plans.</li><li>Contract based plans ( Month-to-Month, One Year, Two Year)</li><li>Type of issue information in customer service calls.</li><li>Survival Analysis ( The Cox Proportional Hazards Model ) — The Cox PH Model is a regression based model that analyses the features of a dataset with regards to how long a or customer survives. It is called the Proportional Hazards model as it describes how each feature proportionally increases risk over the baseline survival rate for a cohort.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*NMpB-gUQAn37PzMY" /><figcaption>Thank You!</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f982807574f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>