<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kang Choon Kiat on Medium]]></title>
        <description><![CDATA[Stories by Kang Choon Kiat on Medium]]></description>
        <link>https://medium.com/@choonkiat.dynp?source=rss-16a8c60aa853------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*q5R69mxgLGKl_zoGLX1Jqg.jpeg</url>
            <title>Stories by Kang Choon Kiat on Medium</title>
            <link>https://medium.com/@choonkiat.dynp?source=rss-16a8c60aa853------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 02:58:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@choonkiat.dynp/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Piping data from Salesforce to MySQL without external connectors]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/piping-data-from-salesforce-to-mysql-without-external-connectors-57a822b1d75b?source=rss-16a8c60aa853------2"><img src="https://cdn-images-1.medium.com/max/1920/0*5RKdAbXFzVQAq0wr.png" width="1920"></a></p><p class="medium-feed-snippet">I hope anyone out there that needs to pipe data from salesforce to MySQL without external connectors because your superior asks you to do&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/data-science/piping-data-from-salesforce-to-mysql-without-external-connectors-57a822b1d75b?source=rss-16a8c60aa853------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/piping-data-from-salesforce-to-mysql-without-external-connectors-57a822b1d75b?source=rss-16a8c60aa853------2</link>
            <guid isPermaLink="false">https://medium.com/p/57a822b1d75b</guid>
            <category><![CDATA[mysql]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Kang Choon Kiat]]></dc:creator>
            <pubDate>Sun, 27 Oct 2019 14:51:49 GMT</pubDate>
            <atom:updated>2019-10-29T16:29:05.846Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Framework for HR Analytics]]></title>
            <link>https://medium.com/@choonkiat.dynp/framework-for-hr-analytics-5b02d6cc379?source=rss-16a8c60aa853------2</link>
            <guid isPermaLink="false">https://medium.com/p/5b02d6cc379</guid>
            <category><![CDATA[hrtech]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[hr]]></category>
            <category><![CDATA[human-resources]]></category>
            <dc:creator><![CDATA[Kang Choon Kiat]]></dc:creator>
            <pubDate>Fri, 26 Jul 2019 14:25:57 GMT</pubDate>
            <atom:updated>2019-07-28T03:29:25.087Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*f-qRo8dE0eDb4dTp" /><figcaption>Photo by <a href="https://unsplash.com/@lukechesser?utm_source=medium&amp;utm_medium=referral">Luke Chesser</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Recently, I was tasked to research on the area of Human Resource(HR) Analytics as my Head of Department wanted to explore how this can help reduce cost for the company.</p><p>I spent about 2 weeks long understanding firstly what HR is all about as well as the implications on the company before trying to see what kind of analysis can be done.</p><p>Today , I am going to share a brief overview of what I had presented to my Head of Department that may be useful to some of you here as well.</p><p>I also want to highlight that a lot of work is needed for actual data analysis and never just slapping a logistic regression to excel data and calling that hard work. <strong>Often, there are lots of business considerations and constraints that need to be thought of.</strong></p><p><strong>Content</strong></p><ol><li>How to estimate workforce</li><li>Spans and Layers</li><li>Ratio Analysis</li><li>Forecasting Methods</li></ol><h3>1. How to estimate workforce</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/0*u7-9GZOt1dLmG58Y" /><figcaption>source: <a href="https://www.cipd.co.uk/Images/workforce-planning-guide_tcm18-42735.pdf">https://www.cipd.co.uk/Images/workforce-planning-guide_tcm18-42735.pdf</a></figcaption></figure><p>This is taken by CIPD, the professional body for human resource in the UK. These are the aforementioned important aspects when it comes to estimating the workforce requirements for a company. We can then simplify this into 3 simple steps we need</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/997/1*x_Deer-oyjDxwIZJv7vFRw.png" /></figure><p>Spans and Layers help us to get the organisational structure, ratio analysis present us with the numbers for each department and forecasting is where Machine learning comes in!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/972/1*Y94MaZY9hO4U9xBchSmIUA.png" /></figure><p>Now, lets get to Spans and layers!</p><h3>2. Spans and Layers</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*jkIFXH4goOu_OD5KsWkVwA.png" /></figure><p>Spans and layers is a popular concept used in organisational structure by a French Management Consultant, V.A.Gracinus. Span refers to the number of direct reports while layers refers to the management layers</p><p>Bain and Co , in a report has written that , “an average company, a manager has a <strong>span of six to seven direct reports </strong>and the organization has <strong>eight to nine layers between the top leadership and the frontline employee.”</strong></p><p>This is because good organisations<strong> try to reduce communication barriers by removing unnecessary layers</strong>. Having more direct reports to 1 superior also <strong>increases productivity and efficiency</strong>.</p><p>However, we do know that every department works differently and may not have the same number of direct reports and this is where Ratio Analysis comes in.</p><h3>3. Ratio Analysis</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/933/1*uPjCMZO1HLEF3CshJv5Kqw.png" /></figure><p>This is useful to determine the exact numbers you would need in your department right now as to determine the number of staff you need for the department.</p><h3>4. Forecasting Methods</h3><p>Now, with all the information you have about the labour you require. We can get down to trying to find out about attrition , future needs and also performance.</p><p>I will go through 2 models used for forecasting and prediction briefly.</p><ol><li><strong>Dynamic Human Resource Model</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/849/0*B78qb4P5QNJftoqk" /></figure><p>These 4 inputs are commonly used to ascertain and to gauge the interest and then to analyse the likelihood of an employee that will turnover. It is also commonly known as <strong>flight risk. HP </strong>is a company that uses a flight risk model to determine attrition rate.</p><p>For modelling, the common ones used are Logistic Regression such as the Dynamic Human Resource Model used by the<a href="https://trace.tennessee.edu/cgi/viewcontent.cgi?referer=https://www.google.com/&amp;httpsredir=1&amp;article=2065&amp;context=utk_gradthes"> University of Tennessee</a>.</p><p>The model uses inputs on the s<strong>taff , economic conditions as well as movement of stocks</strong> to determine the attrition rate. Since all these are factors that affects confidence and also pains a good picture of the economy, they are good indicators.</p><p>The model uses feature engineering to get the best predictors and then uses <strong>logistic regression</strong> to predict the turnover rate</p><p><strong>2. Flight Risk Model by MIT</strong></p><p><a href="http://web.mit.edu/jaillet/www/general/SSRN-id3168168.pdf">MIT developed a mathematical way to assess manpower planning from a perspective of risk on the Singapore Civil Service</a></p><p>The model has reasonable constraints such as <strong>budget , headcount, span control and productivity</strong></p><p>Inputs for this model are <strong>attrition rate, performance and wage patterns.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/1*MnsNKT7qg0oVqbcn2UyB1g.png" /></figure><p>What is interesting is how performance tends to dip after may years in a grade. The intuition is that performance first increase as you get interested in a subject but later dips when you stay too long in a grade and gets bored.</p><p>The model used is a <strong>cutting-plane algorithm</strong> , commonly used to solve convex optimisation problems. The math is pretty intense and would require knowledge of statistics and math to fully aprreciate it</p><h3>Conclusion</h3><p>I hope that I have provided a good framework that can be applied for HR analytics for a start. I also do hope that this would be useful for any HR executives who want to start on HR analytics research.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5b02d6cc379" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Finding Waldo — Feature Matching for OpenCV in Python]]></title>
            <link>https://medium.com/analytics-vidhya/finding-waldo-feature-matching-for-opencv-9bded7f5ab10?source=rss-16a8c60aa853------2</link>
            <guid isPermaLink="false">https://medium.com/p/9bded7f5ab10</guid>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[opencv]]></category>
            <dc:creator><![CDATA[Kang Choon Kiat]]></dc:creator>
            <pubDate>Sun, 14 Jul 2019 14:55:23 GMT</pubDate>
            <atom:updated>2019-07-14T18:55:52.646Z</atom:updated>
            <content:encoded><![CDATA[<h3>Finding Waldo — Feature Matching for OpenCV in Python</h3><h3>1. Introduction</h3><p>In this article, we will do simple Feature Matching, to warm up before we start to do object detection via video analysis. We shall first do some detection with static images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wCKISiDxA7bHzsR8xmfSwg.jpeg" /></figure><p>‘Where’s Wally’ is a popular British series of puzzle books that has garnered interest in both children and adults. Finding Waldo is never easy and OpenCV has a way that can allow us to find Waldo quickly :)</p><h3>2. Concepts used for Template Matching</h3><p>OpenCV has a function, <strong>cv2.MatchTemplate() </strong>that supports template matching to identify the target image.</p><p>Template Matching is the idea of sliding a target image(template) over a source image (input). The template is compared to the input. A match is determined by the how much the neighbourhood pixels in the input matches with the template.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/1*kVybdvLGy84-e32T6e8sLQ.png" /><figcaption>source: <a href="https://docs.opencv.org/2.4/doc/tutorials/imgproc/histograms/template_matching/template_matching.html">https://docs.opencv.org/2.4/doc/tutorials/imgproc/histograms/template_matching/template_matching.html</a></figcaption></figure><p>There are various methods as to how the calculation of similarity is determined. For this example, we will be using <strong>TM.CCOEFF_NORMED</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MouMPPYnxm6sLIS971lMqw.png" /></figure><p>The template patch is slid over the input with this matrix and it determines a score that will indicate whether there is a match. <strong>TM_CCOEFF_NORMED finds the average value of the template and matches it to the average of the input</strong>. A score of 1 is a perfect match, -1 is a bad match and 0 is neutral.</p><h3>3. Finding Waldo</h3><p>Below is the image which we would be using, which will form our input!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wCKISiDxA7bHzsR8xmfSwg.jpeg" /><figcaption>It is sure hell messy</figcaption></figure><p>Now, we will need a template image which of course is going to be a picture of waldo himself.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/24/1*ZfDl6V4zC1_LaIwVDXeryA.png" /><figcaption>The man himself</figcaption></figure><pre>import cv2<br>import numpy as np</pre><p>The 2 holy libraries needed for OpenCV</p><pre>img_rgb = cv2.imread(&#39;find_waldo.jpg&#39;)<br>img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)<br>template = cv2.imread(&#39;waldo.png&#39;,0)<br>#saves the width and height of the template into &#39;w&#39; and &#39;h&#39;<br>w, h = template.shape[::-1]</pre><p><em>cv2.imread</em> reads the image while <em>cv2.cvtColor</em> converts the colour image to grayscale. This is how we <strong>normally perform any type of operation in OpenCV since we reduce the dimensionality and the complexity of the image.</strong></p><pre>res = cv2.matchTemplate(img_gray,template,cv2.TM_CCOEFF_NORMED)<br>threshold = 0.6<br># finding the values where it exceeds the threshold</pre><p><em>cv2.matchTemplate() </em>is the function where we put in the input, the template and the method we use(explained above). <strong>The threshold is a value we use to determine the match, normally a value of 0.8 is chosen</strong>. I chose 0.6 because of how clustered and similar most of the people in the find_waldo image look.</p><pre>loc = np.where( res &gt;= threshold)<br>for pt in zip(*loc[::-1]):<br>    #draw rectangle on places where it exceeds threshold<br>    cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,255,0), 2)</pre><pre>cv2.imwrite(&#39;found_waldo.png&#39;,img_rgb)</pre><p>For those who are not familiar, <em>zip()</em> is a function in python that merges 2 variables together into 1 tuple.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EN9BZD5QM1Pi8xnBkzdFkQ.png" /><figcaption>source: <a href="https://www.w3schools.com/python/showpython.asp?filename=demo_ref_zip">https://www.w3schools.com/python/showpython.asp?filename=demo_ref_zip</a></figcaption></figure><p>As such, what we are trying to do is to merge the width and height into one tuple known as ‘pt’. Hence, we add ‘w’ to pt[0] which is the width and ‘h’ to pt[1] which is the height.</p><p><em>cv2.Rectangle</em> allows us to draw the rectangle by supplying the lower left and upper right coordinate.</p><p>We write this result into a PNG file known as found_waldo.png (:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yzwl5p3OinA3DHlYxMe2tQ.png" /><figcaption>I hope you can see the green rectangle at the top right of this photo!</figcaption></figure><p>And with that, we have found Waldo/Wally and have identified him by having a green rectangle drawn around him!</p><p>The full code is as below, really short and sweet.</p><pre>import cv2<br>import numpy as np<br>from matplotlib import pyplot as plt</pre><pre>img_rgb = cv2.imread(&#39;find_waldo.jpg&#39;)<br>img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)<br>template = cv2.imread(&#39;waldo.png&#39;,0)<br># saves the width and height of the template into &#39;w&#39; and &#39;h&#39;<br>w, h = template.shape[::-1]</pre><pre>res = cv2.matchTemplate(img_gray,template,cv2.TM_CCOEFF_NORMED)<br>threshold = 0.6<br># finding the values where it exceeds the threshold<br>loc = np.where( res &gt;= threshold)<br>for pt in zip(*loc[::-1]):<br>    #draw rectangle on places where it exceeds threshold<br>    cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,255,0), 2)</pre><pre>cv2.imwrite(&#39;found_waldo.png&#39;,img_rgb)</pre><h3>4. Additional Comments</h3><p>Template matching is fun to execute but does have its downsides as well.</p><p>Simple template matching is unable to operate on rotated images, flipped images or images in any other orientation.</p><p>We have to do <strong>multiscaling</strong> to consider such effects.</p><p>And while this has been fun, detecting objects in dynamic situations(such as video) would prove to be more useful. I am looking to share about complex template matching and even object detection in video very soon.</p><p>In the meantime, you can use some of your own images and toy with this simple matching first.</p><p>You can get the full code with photos in my github link:</p><p><a href="https://github.com/k-choonkiat/TemplateMatching/tree/master">https://github.com/k-choonkiat/TemplateMatching/tree/master</a></p><h3>References</h3><ol><li><a href="https://docs.opencv.org/trunk/dc/dc3/tutorial_py_matcher.html">https://docs.opencv.org/trunk/dc/dc3/tutorial_py_matcher.html</a></li></ol><p>2. <a href="https://pdfs.semanticscholar.org/8ffd/b8a1f629a3bbbd1426206e09c7d9115b8f4b.pdf">https://pdfs.semanticscholar.org/8ffd/b8a1f629a3bbbd1426206e09c7d9115b8f4b.pdf</a></p><p>3. <a href="https://docs.opencv.org/2.4/doc/tutorials/imgproc/histograms/template_matching/template_matching.html">https://docs.opencv.org/2.4/doc/tutorials/imgproc/histograms/template_matching/template_matching.html</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9bded7f5ab10" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/finding-waldo-feature-matching-for-opencv-9bded7f5ab10">Finding Waldo — Feature Matching for OpenCV in Python</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Financial Analytics — Exploratory Data Analysis of stock data]]></title>
            <link>https://medium.com/data-science/financial-analytics-exploratory-data-analysis-of-stock-data-d98cbadf98b9?source=rss-16a8c60aa853------2</link>
            <guid isPermaLink="false">https://medium.com/p/d98cbadf98b9</guid>
            <category><![CDATA[stock-market]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Kang Choon Kiat]]></dc:creator>
            <pubDate>Wed, 03 Jul 2019 03:44:58 GMT</pubDate>
            <atom:updated>2019-07-03T23:26:05.132Z</atom:updated>
            <content:encoded><![CDATA[<h3>Financial Analytics — Exploratory Data Analysis of stock data</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*25_I4VdRKG5ZjS0l" /><figcaption>Photo by <a href="https://unsplash.com/@m_b_m?utm_source=medium&amp;utm_medium=referral">M. B. M.</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>With an increase in the penetration of analytics into numerous facets of our lives, Finance is definitely one of the earliest to catch onto this trend. Given the burgeoning market size in fintech and finance, it would be great to impart some financial skills!</p><p><strong>Note: </strong>This article is meant to teach you some basics of manipulating a stock data set and not make you a quant/stockbroker/algorithmic trader. You would surely need more knowledge in trading ,finance and computing in order to advance in those areas.</p><h3><strong>1. Importing the data</strong></h3><p>Most of the financial dataset that you would require can be easily found on <a href="https://sg.finance.yahoo.com/">Yahoo Finance</a>! You just need to key in the stock data, click on <strong>Historical Data</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CTVr1DYdrxgNlDQsCxIPvQ.png" /><figcaption>source: <a href="https://sg.finance.yahoo.com/quote/FB?p=FB&amp;.tsrc=fin-srch">https://sg.finance.yahoo.com/quote/FB?p=FB&amp;.tsrc=fin-srch</a></figcaption></figure><p>After that, set the appropriate time period and remember to click <strong>Apply! </strong>(Heck, when I first started i kept wondering why the csv i downloaded did not give me the right time frame)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rtYu_QaVJvbXF1UM2FUK7g.png" /><figcaption>source: <a href="https://sg.finance.yahoo.com/quote/FB/history?p=FB">https://sg.finance.yahoo.com/quote/FB/history?p=FB</a></figcaption></figure><p>For this tutorial, I will be using Facebook’s data from 16 June 18–16 June 2019. Below are the libraries I will be using to manipulate the data</p><pre>import pandas as pd <br>import numpy as np<br>from scipy.stats import norm</pre><p>As usual, we should always inspect the data and understand the dataframe before we do any further analysis</p><pre>df = pd.read_csv(&#39;stock data/FB_16June18_16June19.csv&#39;)<br>df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/697/1*NTGzlxG27iwX199HImNNDQ.png" /></figure><p>Next we will inspect the datatype of the dataframe using df.info()</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/460/1*9X8-FsHKbo16CGq0BltSmQ.png" /></figure><p>Here, we can see that the ‘Date’ column is displayed as an object instead of datetime datatype. This can be problematic when we are plotting histograms/line chart and so we convert it to a datetime object first</p><pre>df[&#39;Date&#39;] = pd.to_datetime(df[&#39;Date&#39;])</pre><p>Now let’s check the datatype again</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/1*_7cLx6NM__hRhkwdNmBg8w.png" /></figure><p>Nice! We have gotten the ‘Date’ column to be a datetime object now!</p><h3>2. Plotting</h3><p>Time to import the magical stuff</p><pre>import matplotlib.pyplot as plt<br>%matplotlib inline</pre><p>matplot library gives us the power to make powerful plots that gives us insights about the stock</p><pre>plt.figure(figsize=(20,8))<br>plt.plot(&#39;Date&#39;,&#39;Close&#39;,data=df)</pre><pre>plt.xlabel(&#39;Date&#39;)<br>plt.ylabel(&#39;Close Price&#39;)<br>plt.xticks(rotation=45)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SQZCn0m0F_1bcAfdXaV41w.png" /></figure><p>If you have been following financial news, you would know that the huge drop in July is due to Facebook missing their revenue targets and in January it was due to privacy concerns.</p><h3>3. Creation of columns</h3><p>Now, let us create some useful columns for us to make some interesting inferences about the stock</p><p>First, we will create the column ‘daily lag’ which is basically just shifting the ‘Close’ price by one day back. (Note: there are often various metrics we can use but i am choosing ‘Close’ for convenience)</p><pre>df[&#39;Daily Lag&#39;] = df[&#39;Close&#39;].shift(1)<br>df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*aIsWkOIqbCxgfpverRzxnw.png" /></figure><p>The reason for creating the ‘Daily Lag’ is to allow us to create the column ‘Daily Returns’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/0*VtAiLbFvBYuXOlD9.jpg" /><figcaption>source: <a href="http://www.crsp.com/products/documentation/crsp-calculations">http://www.crsp.com/products/documentation/crsp-calculations</a></figcaption></figure><p>Daily returns tells us the returns that we obtain in a day after the stock price closes (Duh!)</p><pre>df[&#39;Daily Returns&#39;] = (df[&#39;Daily Lag&#39;]/df[&#39;Close&#39;]) -1<br>df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/1*_oKd1XNLUuPr-iaZ5WwW3A.png" /></figure><h3>4. More Plotting</h3><p>Now, let us look at the ‘Daily Returns’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*CtSfcKe8wyEhRWwF5Tqksg.png" /></figure><p>We realise that it is hard to tell the shape if we use the <strong>default bin=10. </strong>Let us increase it to 20. Before that let us find the mean and standard deviation.</p><pre>mean = df[&#39;Daily Returns&#39;].mean()<br>std = df[&#39;Daily Returns&#39;].std()<br>print(&#39;mean =&#39;,mean)<br>print(&#39;Std deviation =&#39;,std)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/445/1*kmTmA1FafSA8Bo5K_bMxlA.png" /></figure><p>Ouch,generally returns are negative but remember that this only considers the returns on a day-to-day basis and so it just tells you that in general the return if you <strong>buy and sell on the same day</strong>, you will make a loss.</p><pre>df[&#39;Daily Returns&#39;].hist(bins=20)<br>plt.axvline(mean,color=&#39;red&#39;,linestyle=&#39;dashed&#39;,linewidth=2)<br>#to plot the std line we plot both the positive and negative values <br>plt.axvline(std,color=&#39;g&#39;,linestyle=&#39;dashed&#39;,linewidth=2)<br>plt.axvline(-std,color=&#39;g&#39;,linestyle=&#39;dashed&#39;,linewidth=2)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/535/1*u3IP_OLTz-kgePWn7_wtAg.png" /></figure><p>Lastly, I will introduce kurtosis value to you.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ori8rohC2P7DGPMx" /><figcaption>source: <a href="https://community.plm.automation.siemens.com/t5/Testing-Knowledge-Base/Kurtosis/ta-p/412017">https://community.plm.automation.siemens.com/t5/Testing-Knowledge-Base/Kurtosis/ta-p/412017</a></figcaption></figure><p>Kurtosis tells you the ‘fatness’ of the tail and it is important because it tells you how ‘extreme’ can the values get.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/473/1*Fk7_oP-Fz4745vzSdHLehQ.png" /></figure><p>In our case, the value is positive, so this indicates that the chance of ‘extreme’ values are rare. (Note: the right way to actually ascertain this is using Z-value which I will show in another separarte tutorial!)</p><h3>5. Conclusion</h3><p>This is a very simple walkthrough on some manipulations of stock data for exploration and to unearth some simple insights! There is definitely more to uncover but I think this is a lot for one tutorial already!</p><p>I will be writing more on further statistical analysis and even some trading techniques such as fast-slow signal and Bollinger Bands using python in later tutorials(:</p><p>All the code are from my own Jupyter Notebook and I will may upload it soon to my github so keep a lookout!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d98cbadf98b9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/financial-analytics-exploratory-data-analysis-of-stock-data-d98cbadf98b9">Financial Analytics — Exploratory Data Analysis of stock data</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A simple OpenCV demonstration for facial/face detection]]></title>
            <link>https://medium.com/analytics-vidhya/a-simple-opencv-demonstration-for-facial-face-detection-31c94e7ea05d?source=rss-16a8c60aa853------2</link>
            <guid isPermaLink="false">https://medium.com/p/31c94e7ea05d</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Kang Choon Kiat]]></dc:creator>
            <pubDate>Tue, 02 Jul 2019 03:33:20 GMT</pubDate>
            <atom:updated>2019-07-02T07:57:11.891Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>1. Introduction</strong></h3><p>In recent years, the demand for facial and object detection have increased dramatically due to increased usage in areas such as security, health and even to catch jaywalkers as seen in China.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SxvG6cVTS6FkFPMyMFVMcQ.jpeg" /><figcaption>Facial Recognition to detect Jaywalkers</figcaption></figure><p>As we progress into the 4th Industrial Revolution, the use-case for facial recognition will increase dramatically and evolve into more areas.</p><h3><strong>2. Open CV for Python</strong></h3><p>Here today, I am not going to teach you how to do facial recognition as that would probably require an hour long tutorial in itself but just a simple facial recognition script in python using your own webcam!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S8Il5ethl3YFh0M9XKVz-A.png" /><figcaption>Intel’s OpenCV is a open-source software for facial and object detection</figcaption></figure><p>We are going to use Intel’s OpenCV which is an open-source programming function computer vision!</p><p>For a start, let us install OpenCV on your computer first! Personally I am using jupyter via anaconda and so I installed it via anaconda! You can find the link to download just below!</p><p><a href="https://anaconda.org/conda-forge/opencv">Opencv :: Anaconda Cloud</a></p><p>Once you are done, let us open the notebook an get started! First , we are going to import cv2 as well as the numpy package</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*8EXQa3LJQeg7aOJlnFWyrQ.png" /><figcaption>importing packages</figcaption></figure><p>Next, we are going to download Cascade Classifiers for the face and eye detection</p><h3>3. What are cascades?</h3><p>I always believed that you should understand what you are applying/using in order to fully appreciate the workings and to gain a better understanding and so here I will attempt to simply explain what are cascades!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/565/1*MWjZ2Vt5_Ilkcy7WfiBjJQ.png" /><figcaption>examples of facial cascades</figcaption></figure><p><strong>Note: </strong>Actual Cascades are more complicated than this but this is just an illustration!</p><p>Cascades work like a window that detects features of an object/face that we want to detect. In the example above, we can see 2 different features of a cascade used to detect faces.</p><p>The horizontal feature relies on the fact that the <strong>eyes are normally darker than the cheeks of the face</strong> and hence it is used to detect the position of the eyes.</p><p>The vertical feature relies on the fact that the <strong>bridge of the nose often contrast the eye areas </strong>which is also darker (our nose being more protruded is brighter than the sunken eye area).</p><h3>4. Implementing and downloading cascades</h3><p><a href="https://github.com/opencv/opencv/tree/master/data/haarcascades">opencv/data/haarcascades at master · opencv/opencv</a></p><p>You will need to download the facial and eye cascades for our facial detection exercise today, link is above!</p><p>Once you are done, we will now assign a variable to the cascade classifiers!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/866/1*asxSPxD2Gv3bmiSnzZ_mSQ.png" /></figure><p>Now , we will use videoCapture to turn our webcam on! You should set it at 0 to operate your webcam unless you have more than 1 webcam and in that case you may have to use ‘1’ instead</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*S7xsmnk5OexlkzJUxGDIFA.png" /></figure><p>Here is where the fun starts!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uuq0OFBZPq_cpNLuPGbqLw.png" /></figure><p>What we are doing here , is to first turn the image that we have (from the webcam) into grayscale. <strong>Grayscaling an image ensures easier detection</strong> but we will show the detection on the coloured image/feed.</p><p>Also in the cascade, there are 2 main factors that affects detection</p><ol><li><strong>Scale Factor</strong></li></ol><p>This means that this size of face is detected in the image if present. However, by rescaling the input image, you can resize a larger face to a smaller one, making it detectable by the algorithm<strong>. Or in other words, we tend to increase the scale factor if the image is large</strong>. Normally, 1.1 or 1.2 should be sufficient</p><p><strong>2. minNeighbours</strong></p><p>It is the number of neighbouring rectangles needed to remain detection. Normally 3–6 is sufficient</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*zflKmwWkTi2-bvC6CMQwNg.png" /></figure><p>Here, (x,y,w,h) represents x and y coordinate as well as w for width and h for height.</p><p>cv2.rectangle is a function in cv2 that allows you to draw a rectangle by passing an image, the lower left coordinate, and the upper right coordinate as well as the colour of the rectangle( which is (255,0,0) giving it red, this is a RGB pixel code) and lastly the thickness which I set at 2</p><p>roi_gray is used later again for eye detection! ROI stands for region of image just in case any of you are wondering!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/613/1*wST0mxkh428Ub6QgI_6aZA.png" /></figure><p>This is how the whole code looks like for this part.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SPH7n2N-oiqyG67nRkugmA.png" /></figure><p>the last few parts are just cv2.imshow to show the image and using a waitkey so that you are able to press ‘esc’ key on your window to jump out of the recording and stop the process.</p><p>cap.release also releases the webcam and halts it!</p><p>You can also visit my github for the full code which is basically just a combination of all these snippets. <a href="https://github.com/k-choonkiat/Python-Machine-Learning-/blob/master/OpenCV%20video.ipynb">https://github.com/k-choonkiat/Python-Machine-Learning-/blob/master/OpenCV%20video.ipynb</a></p><p>Alternatively, you can watch sentdex explain it on youtube with a live demonstration of the workings</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fvideoseries%3Flist%3DPLQVvvaa0QuDdttJXlLtAJxJetJcqmqlQq%26start%3D322&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D88HdqNDQsEk&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F88HdqNDQsEk%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/a0ebf1732c77279fb535f0870e3aa455/href">https://medium.com/media/a0ebf1732c77279fb535f0870e3aa455/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=31c94e7ea05d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/a-simple-opencv-demonstration-for-facial-face-detection-31c94e7ea05d">A simple OpenCV demonstration for facial/face detection</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>