<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Alex Attia Blog - Medium]]></title>
        <description><![CDATA[Things I am digging : deep learning, data science, sports, data … - Medium]]></description>
        <link>https://medium.com/alex-attia-blog?source=rss----f9b0441344c5---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Alex Attia Blog - Medium</title>
            <link>https://medium.com/alex-attia-blog?source=rss----f9b0441344c5---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 21:16:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/alex-attia-blog" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Build a first simple Slack Bot with Python]]></title>
            <link>https://medium.com/alex-attia-blog/build-a-first-simple-slack-bot-with-python-5392ef359835?source=rss----f9b0441344c5---4</link>
            <guid isPermaLink="false">https://medium.com/p/5392ef359835</guid>
            <category><![CDATA[slack]]></category>
            <category><![CDATA[star-wars]]></category>
            <category><![CDATA[google-maps]]></category>
            <category><![CDATA[bots]]></category>
            <dc:creator><![CDATA[Alexandre Attia]]></dc:creator>
            <pubDate>Thu, 25 Jan 2018 20:55:45 GMT</pubDate>
            <atom:updated>2018-01-25T21:03:15.699Z</atom:updated>
            <content:encoded><![CDATA[<iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FZwxpIHk5LutMc%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FZwxpIHk5LutMc%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FZwxpIHk5LutMc%2F200w.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="551" frameborder="0" scrolling="no"><a href="https://medium.com/media/53776f4e93761542ac5929579409613e/href">https://medium.com/media/53776f4e93761542ac5929579409613e/href</a></iframe><p>I have always liked the chat bots, even if some of them are overrated and there is often no intelligence behind a chat bot. I like the principle to have an simple assistant in Slack.</p><p>With this post, I quickly explain how to build a simple <strong><em>Yoda bot </em></strong><em>(We are big fans of Star Wars in the team)</em>. Its tasks would be really simple. Most of my work is centered around deep learning and object recognition. So, I sometimes have to manually annotate data and draw bounding boxes in pictures. When I mention the <strong><em>Yoda bot</em></strong>, I want it to tell him how many pictures I have annotated. Moreover, I want it to tell me what’s the duration in traffic to a place (using Google Maps API). It’s a <strong>rule-based</strong> bot (no AI or fancy stuff behind it).</p><p>In the following, there will be some Python code lines, it’s in Markdown so it won’t be very pleasant to read !</p><h4>Requirements</h4><ul><li>Python 3.6, pip (/virtualenv)</li><li>Slack workspace and account</li></ul><p>Install the Slack API via</p><pre>pip install slackclient</pre><h4>Create the Slack App</h4><p>Create your slack app on the <a href="https://api.slack.com/apps?new_app=1">official Slack API website</a> get an API token for the bot. Set a name and the workspace you want to work into.</p><p>Go to the <em>Bot Users </em>part to set the display name and the default username.</p><p>Then go the <em>Install the App</em> part to get the token. Save <em>bot user oauth access token.</em></p><pre>token = ‘xoxb-XXXYYYZZZ’</pre><h4>Set up and basic functions</h4><p>The Slack API really makes things easy. Concretely, we just need to connect <em>(using the token)</em>, check if the bot is connected, then read each message received (i.e posted in the workspace) and eventually, answer to the query.</p><p>We connect with :</p><pre>slack_client = SlackClient(token)</pre><p>We check the connection with :</p><pre>bool = slack_client.rtm_connect(with_team_state=False)</pre><p>We read the message<em> (returns a list)</em> with:</p><pre>events = slack_client.rtm_read()<br># events = [{&#39;type&#39;: XXX, &#39;text&#39;: YYY, &#39;user&#39;: ZZZ}, ...]</pre><p>We post a message with :</p><pre>slack_client.api_call(“chat.postMessage”, channel=channel, text=t)</pre><p><strong><em>NB :</em></strong><em> To post into a channel, you </em><strong><em>must</em></strong><em> invite the bot to this channel (exception for direct message with the bot)</em></p><h4>Coding the Yoda bot : number of annotated data</h4><p>My annotation file is a csv with the format <em>img_name, x1, y1, x2, y2, class. </em>I want to count the total number of rows and the number of rows equal to a particular class.</p><p>The function to post the number of annotated pictures or another message is below :</p><pre><strong>def post_annotation(token, text=None, channel=&#39;bot&#39;, response_to=&#39;&#39;):</strong><br><em>    &quot;&quot;&quot;<br>    Post a message into a channel the number of annotated pictures or a other text.<br>    :param token: Slack API token<br>    :param text: text to post. If None, the text is the information about the annotations<br>    :param channel: channel where the bot post the message<br>    :param reponse_to: if we want to mention some one<br>    &quot;&quot;&quot;</em><br>    <em># connection</em><br>    slack_client = SlackClient(token)<br>    <em># Number of rows and positive samples</em><br>    <strong>with</strong> open(&quot;annotation.txt&quot;, &#39;r&#39;) as f:<br>        annotation = f.read().split(&#39;\n&#39;)[:-1]<br>    tot = len(annotation), <br>    pos =len([p for p in annotation if p.split(&#39;, &#39;)[5] == &#39;1&#39;])<br><em>    # Mention someone (need an user ID, not an username)</em><br>    <strong>if</strong> response_to != &#39;&#39; :<br>        response_to = &#39;&lt;@%s&gt;&#39; % response_to<br>    <em># Pre-defined message</em><br>    <strong>if</strong> not text:<br>        text = &#39;%s _%s_\nYou have annotated *%d* pictures \with *%d*     fractures :+1:&#39; % (response_to, np.random.choice(sw_sentences), tot, pos)<br>    <em># Post message</em><br>    slack_client.api_call(<br>        &quot;chat.postMessage&quot;,<br>        channel=channel,<br>        text=text)</pre><p><em>The sw_sentences is a list of pre-defined Star Wars quote (mostly Yoda quotes) to share some wisdom with my team.</em></p><p><strong>Coding the Yoda bot : duration in traffic</strong></p><p>For the traffic, I used the official Google Maps API <em>(free for &lt;2500 requests/day) </em>and <a href="https://github.com/googlemaps/google-maps-services-python">Python client</a>. I just want to know the duration in traffic between two addresses. For the Google Maps API, you have to register an app <a href="https://console.developers.google.com/">here</a> and get an API token<em> (again!)</em>. Then, install the Python package.</p><pre>pip install -U googlemaps</pre><p>The function to get the duration in traffic (no Slack post yet) is below :</p><pre><strong>def get_directions_duration(address_dep, address_arriv, name_dest):</strong><br><em>    &quot;&quot;&quot;<br>    Get the duration in traffic using Google Maps API<br>    :param address_dep: departure address (as on Google Maps)<br>    :param address_arriv: arrival address<br>    :param name_dest: name of the destination<br>    :return: text with the ETA<br>    &quot;&quot;&quot;</em><br><em>    # Connect to the API</em><br>    gmaps = googlemaps.Client(key=gm_key)<br>    now = datetime.datetime.now()<br><em>    # Compute the directions </em><br>    results = gmaps.directions(address_dep, address_arriv, mode=&quot;driving&quot;,departure_time=now)<br>    directions_result = results[0][&#39;legs&#39;][0]<br><em>    # Get resukts</em><br>    duration = directions_result[&#39;duration_in_traffic&#39;]<br><em>    # Text (fstring) with well formatted addresses</em><br>    t = f&#39;_YodaWaze_ - Time to {name_dest} : %s _(%s to %s)_&#39; % (duration[&#39;text&#39;], directions_result[&#39;start_address&#39;],  directions_result[&#39;end_address&#39;])<br>    return t</pre><p><strong>Coding the Yoda bot : handling messages</strong></p><p>Now we need to differentiate the different type of messages to answer differently depending of the use case. I have three type of events :</p><ul><li><em>“@YodaBot help”</em> : to post the explanations of the bot</li><li><em>“@Yodabot” </em>: to post the number of annotated data</li><li><em>“@Yodabot give me the time from my place to Elie’s place” </em>: to post the ETA in traffic from two places</li></ul><p>I have to take into account only message of the users and not the bot posts <em>(it would result in an infinite loop…).</em></p><p>To match the users and the addresses, I have to define a dictionary with the addresses <em>a</em>nd I want to be able to use “my place” instead of my first name in the post <em>(it would be weird to use the 3rd person for myself) </em>so I have to match the user ids with the user names:</p><pre>adress = {‘alex’ : ‘1 Fake St San Francisco’, ‘elie’: ‘5 Fake St Palo Alto’, ‘eliott’: ’10 Fake St San Jose’}<br>ids = {&#39;UXXYY&#39;: &#39;alex&#39;, &#39;UYYZZ&#39;: &#39;elie&#39;, &#39;UZZXX&#39;:&#39;eliott&#39;}</pre><p>The function to handle message is :</p><pre><strong>def parse_bot_commands(slack_events):</strong><br><em> “””<br> Handling the posts and answer to them<br> :param slack_events: slack_client.rtm_read()<br> “””</em><br> <strong>for</strong> event in slack_events:<br><em>     # only message from users</em><br>     <strong>if</strong> event[‘type’] == ‘message’ and not “subtype” in event:<br><em>         # get the user_id and the text of the post</em><br>         user_id, text_received, channel = event[‘user’],   event[‘text’], event[‘channel’]<em><br>         # the bot is activated only if we mention it</em><br>         <strong>if</strong> ‘@%s’ % yodabot_id in text_received:<br>             <em># Activate help if ‘help’ or ‘sos’ in the post</em><br>             <strong>if</strong> any([k in text_received for k in [‘help’, ‘sos’]]):<br>                   post_annotation(token, text=help_text, channel=channel)</pre><pre><em>             # Activate Google Maps if ‘distance’ or ‘time’ </em><br>             <strong>elif</strong> any([k in text_received for k in [‘distance’, ‘time’]]):<br><em>               # search the users names in the post</em><br>               r = re.compile(r’\bELIE\b | \bELIOTT\b | \bALEX\b | \bMY PLACE\b’, flags=re.I | re.X)<br>               matched = r.findall(text_received)<br>               <em># replace ‘my place’ by the user name</em><br>               <strong>if</strong> ‘my place’ in matched:<br>                    matched[matched.index(‘moi’)] = ids[user_id]<br>               <em># check if we have the addresses of all users</em><br>               <strong>if</strong> all([k in adress for k in matched]):<br>                   name_dest = matched[1].title()<br><em>                   # Compute direction</em><br>                  directions_text = get_directions_duration( adress[matched[0]], adress[matched[1]], name_dest)<br>                  <em> # Post message</em><br>                   post_annotation(token, text=directions_text, channel=channel)<br> <br><em>             # Activate annotations information</em><br>        <strong>     else:</strong><br>                post_annotation(token, channel=channel)</pre><h4>Coding the Yoda bot : wrapping up</h4><p>The remaining part consists only in wrapping up the different part into a <em>slackbot.py</em> file. We need the <em>yodabot_id</em> for the mentions.</p><pre><strong>import</strong> os<br><strong>import</strong> time<br><strong>import</strong> numpy as np<br><strong>import</strong> re<br><strong>from</strong> slackclient <strong>import</strong> SlackClient<br><strong>import</strong> googlemaps<br><strong>import</strong> datetime</pre><pre><strong>if</strong> __name__ == “__main__”:<br>   slack_client = SlackClient(token)<br>   <strong>if</strong> slack_client.rtm_connect(with_team_state=False):<br>       yodabot_id = slack_client.api_call(“auth.test”)[“user_id”]<br>       <strong>while</strong> True:<br>           parse_bot_commands(slack_client.rtm_read())<br>           time.sleep(1)</pre><p><strong>NB</strong> : We need a <em>time.sleep</em> otherwise, it would run indefinitely and use the full CPU even when there isn’t any post. With the <em>time.sleep(1)</em>, we have at least one second of delay.</p><p>To launch it on an Ubuntu server, just use :</p><pre>nohup nice python slackbot.py &amp;  </pre><p>Hope it will be useful !</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_ja1RWLVvrV__LuVJP_7Kg.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5392ef359835" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alex-attia-blog/build-a-first-simple-slack-bot-with-python-5392ef359835">Build a first simple Slack Bot with Python</a> was originally published in <a href="https://medium.com/alex-attia-blog">Alex Attia Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Simpsons characters recognition and detection (Part 2)]]></title>
            <link>https://medium.com/alex-attia-blog/the-simpsons-characters-recognition-and-detection-part-2-c44f9d5abf37?source=rss----f9b0441344c5---4</link>
            <guid isPermaLink="false">https://medium.com/p/c44f9d5abf37</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[image-recognition]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[the-simpsons]]></category>
            <dc:creator><![CDATA[Alexandre Attia]]></dc:creator>
            <pubDate>Fri, 30 Jun 2017 19:41:17 GMT</pubDate>
            <atom:updated>2017-07-07T16:41:04.978Z</atom:updated>
            <content:encoded><![CDATA[<p>Deep Learning : Detect and classify multiple The Simpsons characters in the same picture.</p><p>In <a href="https://medium.com/alex-attia-blog/the-simpsons-character-recognition-using-keras-d8e1796eae36">Part 1</a>, I trained a convolutional neural network to recognize (i.e classify) 20 The Simpsons characters. Giving a picture of a character, the model returns the character on this image. You can still find the <a href="https://www.kaggle.com/alexattia/the-simpsons-characters-dataset">open dataset on Kaggle</a>. My accuracy was pretty high (F1 : 96%) but it’s just a simple classifier. So, it can only recognize one character at a time and it doesn’t return the position of this character.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2Fc1zviFHCf4pq0%2Ftwitter%2Fiframe&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2Fc1zviFHCf4pq0%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2Fc1zviFHCf4pq0%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="295" frameborder="0" scrolling="no"><a href="https://medium.com/media/1ac84240f0077d0e284fda2d1b979ca8/href">https://medium.com/media/1ac84240f0077d0e284fda2d1b979ca8/href</a></iframe><p>Now, I would like to create a model to detect and classify each character in the picture.</p><p>The model would be much more complicated than the previous one and would be able to draw bounding boxes around each character.</p><p>At the beginning, I was thinking of a sliding window which classify multiple windows in a picture. Afterward, to detect a character, we group overlapping boxes with the same character. This algorithm would predict a lot of sub-pictures for each picture so it’s very slow.</p><p>So, I will use a faster and state-of-the-art model which is very interesting : <a href="https://arxiv.org/pdf/1506.01497">Faster R-CNN</a>.</p><p>Again, I will be using <a href="http://www.keras.io/">Keras</a> with TensorFlow backend.</p><p>You can find the code on the related <a href="https://github.com/alexattia/SimpsonRecognition"><strong>GitHub repo</strong></a>.</p><h4>The Faster R-CNN network</h4><blockquote>Object detection networks depend on region proposal algorithms to hypothesize object locations</blockquote><p>This model is based on <strong>Region Proposal Network</strong> : fully convolutional network that simultaneously predicts object bounds and objectness scores at each position.</p><p>Faster R-CNN is an updated version of R-CNN <em>(and Fast R-CNN)</em>. The structure is similar to Fast R-CNN, but proposal part is replaced by a ConvNet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/399/1*imQb4DAmY5nHifFJYb_R9g.png" /><figcaption>Faster R-CNN details</figcaption></figure><p>This is the feed-forward pass stucture :</p><ol><li>Convolutional network to get feature map from the last convolutional layer.</li><li>Region Proposal Network (RPN) with a convnet to propose Region of Interest (RoI) and project them on the feature map</li><li>Each proposed region is passed into a RoI pooling layer.</li><li>Fully connected layer to classify region</li></ol><p>This model has been implemented in Keras by <a href="https://github.com/yhenon/keras-frcnn">Yann Henon</a>.</p><h4>Completing the Dataset</h4><p>The dataset used for the first part wasn’t complete enough. Indeed, we need the bounding boxes for each character of the training set.</p><p>So, using the already label data -you can see explanations of the dataset in the first part or on <a href="https://www.kaggle.com/alexattia/the-simpsons-characters-dataset">Kaggle</a>, I just label bounding boxes for each picture with <em>matplotlib</em> and mouse clicks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/690/1*jbX8I7h5LBX5OY8v14mRPA.png" /><figcaption>Get bounding box coordinates for one character per picture</figcaption></figure><p>So, in addition to pictures, I have a text file with bounding boxes coordinates and classes :</p><pre># pic, x1, y1, x2, y2, class <br>pic_000.jpg,80,31,337,354,homer_simpson<br>...</pre><p>For each character, we have to point the upper left and lower right corner of a bounding box around the character.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/303/1*nt_ZmV6BMOamLARGLFRiCQ.png" /><figcaption>Dataset distribution for 18 characters (on 7/5/2017)</figcaption></figure><h4>Preprocessing</h4><p>Pictures passed into the model can have different ratio/sizes but must be preprocessed</p><p>We resize the image so that smallest side length is 300 pixels but while keeping the same picture ratio. We also normalize pictures by subtracting the dataset’s mean for each channel in order to center the data (we want each feature to have a similar range so that our gradients don’t go out of control.</p><pre>x_img = x_img[:,:, (2, 1, 0)] # BGR -&gt; RGB<br>x_img = x_img.astype(np.float32)<br>x_img[:, :, 0] -= C.img_channel_mean[0]<br>x_img[:, :, 1] -= C.img_channel_mean[1]<br>x_img[:, :, 2] -= C.img_channel_mean[2]</pre><h4>Defining the network</h4><p>The base network is <a href="https://arxiv.org/abs/1512.03385">ResNet</a>. Then, we build the RPN on the base layers. In addition, we have the classifier also built on the base layers.</p><pre><em># base layers</em><br><strong>shared_layers</strong> = nn.nn_base(img_input, trainable=True)</pre><pre><em># define the RPN, built on the base layers</em><br><strong>rpn</strong> = nn.rpn(shared_layers, num_anchors)</pre><pre><em># define the classifer, built on the base layers</em><br><strong>classifier</strong> = nn.classifier(shared_layers, roi_input, C.num_rois, nb_classes=len(classes_count), trainable=True)</pre><pre><em># defining the models + a model that holds both other models</em><br><strong>model_rpn</strong> = Model(img_input, rpn[:2])<br><strong>model_classifier</strong> = Model([img_input, roi_input], classifier)<br><strong>model_all</strong> = Model([img_input, roi_input], rpn[:2] + classifier)</pre><h4>Training</h4><p>For the training, the model is iterating over batches of training set for 90 epochs (the length of each epoch is 1000) .</p><p>It’s impossible to run it on CPU so I run it on GPU with AWS EC2, Tesla K80: 410 seconds per epoch. It took 10 hours.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*nVsbAtpXQUZyaxXuFfUjHA.png" /><figcaption>Accuracy and losses during training</figcaption></figure><h4>Predictions</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5Z5t8IIqJRaIgrza08Zm-A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LOfYUFVEWVMHx5yGnIV9qA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sgIZPlYv7MfLmClGyS51QA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DrdGVPXNFPrm7lmDzfrRcg.png" /></figure><p>For each picture, we are detecting characters and classify them. The network predicts the coordinates of the bounding boxes for each character predicted.</p><p>Characters are well predicted and are often detected but there is too much overlapping. Indeed, bounding boxes are often too large around characters so, when there is more than one character in the picture, the bounding boxes are overlapped. We can improve this by tuning the overlapping_threshold and the <a href="http://www.pyimagesearch.com/2015/02/16/faster-non-maximum-suppression-python/">Non-Maximum Suppression</a> function itself.</p><p>Of course, the accuracy is not as high as on the convnet which predict one character at a time with a full picture containing one character.</p><p>Predictions are very slow on my laptop CPU : 8s per picture. On a GPU (Tesla K80), it’s 0.98 second per picture. It could be interesting to compare with a sliding window a simple convnet (such the one used in Part 1) : for a a 640x460 picture, using a sliding window of 64x128 (moves 8 pixels at a time horizontally and 4 pixels vertically, we have to predict 6000 pictures ! Even with a really quick network which predicts one box in 0.01s, it’s still 60s per picture ! Moreover, with a sliding window, we have only one ratio possible.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fta_hCXKZ_DM%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dta_hCXKZ_DM&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fta_hCXKZ_DM%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/f985d6d47325681123fce1403343850c/href">https://medium.com/media/f985d6d47325681123fce1403343850c/href</a></iframe><h4>Improvements</h4><p>I am still annotating pictures, I will update this post with the new predictions.</p><p>Again, if you have any questions, please feel free to contact me and moreover, if you like this post don’t hesitate to recommend it :-).</p><p>The dataset is <a href="https://www.kaggle.com/alexattia/the-simpsons-characters-dataset">on Kaggle</a>, download it and have fun !</p><p><strong>Alexandre Attia</strong><br><a href="http://www.alexattia.fr/">www.alexattia.fr</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c44f9d5abf37" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alex-attia-blog/the-simpsons-characters-recognition-and-detection-part-2-c44f9d5abf37">The Simpsons characters recognition and detection (Part 2)</a> was originally published in <a href="https://medium.com/alex-attia-blog">Alex Attia Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to take back control and use your Google Maps data ?]]></title>
            <link>https://medium.com/alex-attia-blog/how-to-take-back-control-and-use-your-google-maps-data-683fb5d4043e?source=rss----f9b0441344c5---4</link>
            <guid isPermaLink="false">https://medium.com/p/683fb5d4043e</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[google-maps]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[maps]]></category>
            <dc:creator><![CDATA[Alexandre Attia]]></dc:creator>
            <pubDate>Mon, 05 Jun 2017 18:48:33 GMT</pubDate>
            <atom:updated>2017-06-05T18:48:32.715Z</atom:updated>
            <content:encoded><![CDATA[<p>If you didn’t disable it, Google Maps Timeline stores your location and places you’ve been but fortunately you can download your “timeline”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GtBLgTSA_8Ir3sBReTYjuQ.png" /></figure><p>All my Google Maps data are stored on Timeline and Google processes it in order to create a timeline that I can find in the user interface : <a href="https://www.google.fr/maps/timeline">https://www.google.fr/maps/timeline</a>. I can edit it, visualize it but I cannot compute statistics about it or visualize it in a different way.</p><p>Maps is using raw data (multiple points saved per minutes, depending of your network) and is projecting it on roads, places, etc.<br>You can easily export all of the raw data saved by Maps or you can export one day to KML but it’s impossible to automatically export all of the processed data from the user interface <em>(you can download each day manually … Good luck if you want to do this)</em>.</p><p>Except if you are Google, or have a dataset with many places and roads, it’s really complicated to convert raw data (latitude/longitude and time) to places and way of moving. KML files are well-formatted data for every day, it’s exactly what we can see on the user interface. That’s why I wanted to automatically download each day into KML files and afterwards analyze and visualize them.</p><p>You can see the <a href="https://github.com/alexattia/Maps-Location-History">code here</a>.</p><h3>Download KML files</h3><p>In order to export processed data from Google Maps website from a python script, you need to get your actual cookie <em>(the same you have when you download manually one day)</em>.</p><p>How to get the cURL :</p><ol><li>Go to <a href="https://www.google.fr/maps/timeline">https://www.google.fr/maps/timeline</a></li><li>Inspect the page and go to the Network tab</li><li>Enter this link <a href="https://www.google.fr/maps/timeline/kml?authuser=0&amp;pb=!1m8!1m3!1i2017!2i3!3i16!2m3!1i2017!2i3!3i16">https://www.google.fr/maps/timeline/kml?authuser=0&amp;pb=!1m8!1m3!1i2017!2i3!3i16!2m3!1i2017!2i3!3i16</a> (or another date)</li><li>Save this element as a cURL</li><li>Open the cURL in a text editor</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*67XwQgQMIuc99ipQwegwtQ.png" /></figure><p>You should get something like this:</p><pre>curl &#39;https://www.google.com/maps/timeline/kml?authuser=0&amp;pb=!1m8!1m3!1i2017!2i3!3i16!2m3!1i2017!2i3!3i16&#39;   <br>-H &#39;accept-encoding: gzip, deflate, sdch, br&#39;   <br>-H &#39;accept-language: en-US,en;q=0.8,fr;q=0.6&#39;   <br>-H &#39;upgrade-insecure-requests: 1&#39;   <br>-H &#39;user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) &#39;   <br>-H &#39;x-chrome-uma-enabled: 1&#39;   <br>-H &#39;accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8&#39; <br>-H &#39;authority: www.google.com&#39;   <br>-H <strong>&#39;**cookie**</strong>: gsScrollPos=; _ga=GA1.1.49937635TxMcGmJ-uXXXXX gsScrollPos=&#39;  <br>-H &#39;x-client-data: XXXQ==&#39; --compressed</pre><p>Finally, save the cookie content.</p><p>Then, you need to do a simple request at the good url depending of the day you want to save and save the request as a KML file. You can now loop over the days and months you want. <em>month_url</em> and <em>day_url</em> are numerical values <em>(I have created a python script if you don’t want to do it by your own)</em>.</p><pre>url = ‘<a href="https://www.google.fr/maps/timeline/kml?authuser=0&amp;pb=!1m8!1m3!1i2017!2i{0}!3i{1}!2m3!1i2017!2i{0}!3i{1}&#39;.format(month_url,">https://www.google.fr/maps/timeline/kml?authuser=0&amp;pb=!1m8!1m3!1i2017!2i{0}!3i{1}!2m3!1i2017!2i{0}!3i{1}&#39;.format(month_url,</a> day_url)</pre><h3>Visualizing data</h3><p>I have then created a well-formatted <em>Pandas</em> data frame with my personal data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tqEtzH4NPEqvQkpoeZBHOg.png" /><figcaption>Sample of the data frame for one day</figcaption></figure><p>As you can see, we get the same information than in the user interface : activity/place category <em>(driving, walking, restaurant, grocery store, etc.)</em>. We also have place names, durations and dates.</p><p>We can do many things with this data. The main thing I am currently focused on is the time spent at doing/being something/somewhere.</p><p>During my time living in the US, I lived 1 month in one place then I moved out <em>(to a place farer from my work)</em>. I wanted to see if there were a big difference of driving time and distance each day.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*8k6ZJCrzakhvtyfX-i211g.png" /><figcaption>Driving time and distance</figcaption></figure><p>Although, the distance difference is not really important, the driving time is really different : I spent <strong>30 more minutes</strong> each day in the car. Let’s see if there is an impact on my time at work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/696/1*zK_NXOyflKlyAitgs6Ttfw.png" /></figure><p>I used to spent <strong>8h19</strong> at work before moving out, and now, I spend <strong>8h03</strong> there. <em>(This graph is the time spent at the office, it’s not the difference between arrival and departure, so it doesn’t count time to grab food, restaurants, etc.)</em>.</p><h3>Plotting on maps</h3><p>I didn’t go really further on how to use all this data. My main goal was just to get a little bit more information about my trips and feel to have control on my data <em>(even if I know that’s not really true)</em>. I just did one thing more : plotting points on maps.</p><p>I used multiples libraries to plot data on maps. One using Google Maps, <a href="https://pypi.python.org/pypi/gmplot/1.0.5">gmplot</a> which can be used to draw scatterplots and heat maps and save it as .html files :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/615/1*wOtDEywXsuQURQpGwiAqOg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/1*jg4nQoIE3NPoeXudI1l3nQ.png" /></figure><p>And I also plot on basemaps <em>(probably a geolocalization error because I never went swimming, moreover with my phone) next to Golden Gate bridge) :</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/1*RPib8xbWWsgUv5pYDB_8vA.png" /></figure><p>As you can see, we can do many things with Google Maps Timeline data and the data saved by our phones. I think I can do many more things with this data but the current aim was to recover my location data and compute some stats about it. I will probably continue this project later.</p><p><strong>Alexandre Attia</strong><br>www.alexattia.fr</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=683fb5d4043e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alex-attia-blog/how-to-take-back-control-and-use-your-google-maps-data-683fb5d4043e">How to take back control and use your Google Maps data ?</a> was originally published in <a href="https://medium.com/alex-attia-blog">Alex Attia Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Soccer data and predicting L’Équipe ratings]]></title>
            <link>https://medium.com/alex-attia-blog/soccer-data-and-predicting-l%C3%A9quipe-ratings-7f8761932b78?source=rss----f9b0441344c5---4</link>
            <guid isPermaLink="false">https://medium.com/p/7f8761932b78</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[soccer]]></category>
            <dc:creator><![CDATA[Alexandre Attia]]></dc:creator>
            <pubDate>Fri, 02 Jun 2017 17:52:29 GMT</pubDate>
            <atom:updated>2017-06-03T17:02:29.359Z</atom:updated>
            <content:encoded><![CDATA[<p>Data Mining, visualization and Machine Learning around soccer data</p><p>Soccer is not statistics focused as basketball or baseball can be. Must there are some important data in soccer : goals, assists and possession are the more common but many others could represent what we are watching : tackles, cross, passes, dribbles, etc.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F3o6Yg4GUVgIUg3bf7W%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2F3o6Yg4GUVgIUg3bf7W%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2F3o6Yg4GUVgIUg3bf7W%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="185" frameborder="0" scrolling="no"><a href="https://medium.com/media/d3423360ff3b77833af2a82cc41c71b3/href">https://medium.com/media/d3423360ff3b77833af2a82cc41c71b3/href</a></iframe><p>I have gathered games data for French League 1 games for players and teams. I also have the ratings L’Équipe (famous French sports newspapers) for the 2016/2017 season. The goal of this project is to visualize soccer data, see if there are some interesting insights and try to predict ratings L’Équipe (made by journalists, so those ratings are often subjectives).</p><h3>2016/2017 <em>(not yet Conforama)</em> League 1 season</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/404/1*8SpGt2MHBvbgNjKDwU-MCQ.png" /></figure><h4>Data</h4><p>For each player and each game, I have those features :</p><pre>&#39;Crosses&#39;, &#39;Long Ball&#39;, &#39;ThroughBall&#39;, &#39;AerialsWon&#39;, &#39;Assist&#39;, &#39;BlockedShots&#39;, &#39;Clearances&#39;, &#39;Crosses&#39;, &#39;Day&#39;, &#39;Dispossessed_game&#39;, &#39;Dribbles&#39;, &#39;Fouled&#39;, &#39;Fouls&#39;, &#39;Goal&#39;, &#39;Goal Opponent&#39;, &#39;Goal Team&#39;, &#39;Interceptions&#39;, &#39;KeyPasses&#39;, &#39;Offsides_won_game&#39;, &#39;Opponent&#39;, &#39;PassAccuracy&#39;, &#39;Passes&#39;, &#39;Penalty Missed&#39;, &#39;Home/Away&#39;,<br>&#39;Red Card&#39;, &#39;Shot On Post&#39;, &#39;Shots&#39;, &#39;Shots On Target&#39;, &#39;Team&#39;,<br>&#39;TotalTackles&#39;, &#39;Touches&#39;, &#39;UnsTouches&#39;, &#39;Yellow Card&#39;, &#39;Age&#39;,<br>&#39;Position&#39;, &#39;In LineUp&#39;</pre><p>I use Pandas DataFrames for my preprocessing. I have 10 000 rows (20 teams, 35 games, in average 14 players -subs included-/game). Here are some of the features for few rows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cWK_oXzSMZpy9fyWdsmskA.png" /><figcaption>Players data frame with some of the variables</figcaption></figure><h4>Preprocessing</h4><p>Before training my model, I have to preprocess my data : fill NaN values, convert to float numerical values, convert categorical data to int (with <em>sklearn LabelEncoder</em>).</p><h4>Data Analysis</h4><p>After preprocessing my data and having a well-formatted data frame, I have looked at the data to understand it a little bit more. I wanted to see if there were correlations between features and which features were really important for ratings. The first feature I focused on was the player position, there were too many positions (6 types of midfielder, 5 types of attacking players, 4 types of defender), so I mapped those categories into : defender, attacking, midfielder, goalkeeper and plotted the position distribution and the ratings distribution depending of the position :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*lTmqBc873pWQRr5ZGt7ErA.png" /></figure><p>As you can see, in proportion, it’s hard to get a high rating being a defender compare to attacking and midfield player. In the same way, midfielder have less really bad ratings than the other positions. If you don’t want a L’Equipe bashing, be a midfielder !</p><p>As a PSG fan, I am really disappointed by this season but objectively, Monaco did a great season with great young players that have sometimes amazed me (Mbappé, Silva, Lemar, Mendy …). So I wanted to check if we can see that trend in the ratings. I plotted the ratings distribution for the best French teams and the mean rating for every L1 team.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/611/1*ajXxxKM7Ebm7btTaGceuLg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/1*YtAqjMEYVkVpgJQIzMGKVg.png" /></figure><p>So indeed, Monaco was really the best team this year … And Paris is not even the second for L’Equipe !<br>Intuitively, the team and the positions seems to be two important factors for the L’Equipe ratings(in addition of the number of goals, assists and shots).</p><p>By computing the correlation matrix, we can see than<em> (obviously)</em> some features are correlated : shots on target/shots &amp; goals <em>(yeah, I know we couldn’t score with shots not on target) </em>and<em> </em>keys passes &amp; cross. But those features, are really different so I kept all of them.</p><h4>Learning and evaluating</h4><p>I have benchmarked Random Forest and Ridge Regression models with the super useful <em>scikit-learn</em>.</p><p>I began with the simple Ridge Regression (linear regression where the loss function is the linear least squares function and regularization is given by the L2-norm). To evaluate it, I use 10-fold cross validation (test on partitions of 1/10 of the dataset ten times and average on those ten test errors). The cross validation mean square error (MSE) is <strong>0.83</strong>, which is not really good knowing than the L’Equipe ratings are between 0 and 10 but it isn’t too important. The R2 (coefficient of determination) score is <strong>0.26</strong>. So, our model isn’t really linear though but this regression could help us getting the most important variables for the ratings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*bluKRmhl2nHYbBxeuRhKRA.png" /><figcaption>Miss a penalty is really penalizing for your rating !</figcaption></figure><p>As you can see, the most important features are pretty obvious for a soccer fan: missing a penalty, getting a red/yellow card, conceding goals are important negative factors. Contrary to goals, assists, shots on target/post which are positive factors.</p><p>The second model that I tried is the <em>famous</em> bagging model Random Forest. To evaluate this model, I used the Out of Bag (OOB) score [1] in order to validate the model during training (as accurate as using a test set of the same size as the training set). The OOB score is <strong>0.34 </strong>so this is way better than the regression (R2 for this model is 0.91).</p><p>Both of my models were fine-tuned but I couldn’t get a better score, I think I would need more data. To confirm this, I compute the OOB score for different sizes of the dataset (30%, 40%, etc.). and we can see that the OOB score decreases with the size of the dataset.</p><p>For now, it’s now a true success, even if the RF OOB score isn’t too bad, but I hope to get more data next year and improve this model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/426/1*arlN0UMcPPY895eQ2NwGdg.gif" /><figcaption>FAIL ! <strong><em>(for now !!)</em></strong></figcaption></figure><p>[1]<a href="https://www.stat.berkeley.edu/~breiman/OOBestimation.pdf"> Out-of-Bag Estimation, Leo Breiman</a></p><h3>Soccer data for 2016/2017 for some European leagues</h3><p>I have also gather data for average statistics for players and teams in England, France, Italy, Spain and Germany.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*offA2JC-DF08Td6t7WH78g.png" /><figcaption>Teams Data Frame</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*1fVg4jsfd69afFq21eTMig.png" /><figcaption>Players Data Frame</figcaption></figure><p>I did some preprocessing to gather for data and to have well-formatted <em>Pandas</em> data frames. The aim of this part is to see if there are some interesting insights about this season.</p><p>Moreover, I want to improve my data visualization skills and use <em>Matplotlib</em> and <em>Seaborn</em> to plot graphs. I want to see if I can retrieve the same trends, that I saw this season in game, in the plots : players contributions, wunderkids, etc.</p><h4>Goals contribution</h4><p>I watched almost PSG games this season and one player was really often scoring : Edinson Cavani. He scored many of his team goals so I was wondering what was his contribution compare to other great strikers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qh1eyYddWCdpiY5bElEj5Q.png" /><figcaption>Some of the best strikers contribution to their team goals</figcaption></figure><p>Indeed, Cavani scored <strong>42%</strong> of PSG goals (we really need another striker !) equally to Aubameyang with Dortmund. The MSN (Messi, Suarez, Neymar) has striked again : Messi &amp; Suarez have scored <strong>57%</strong> of their team goals. Another interesting stats, Falcao, the best Monaco striker, has only scored 19% of his team goals.</p><h4>Wunderkids</h4><p>This year was the year of Mbappé’s outbreak. He’s only 18 years old and has incredible stats for his young age but how is he compare to others young players ?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TGWTHv8Kn39cLwQMiwmBgw.png" /><figcaption>Best scorers/game for less than 24 y.o. players (more than 15 games played)</figcaption></figure><p>Indeed, Mbappé is impressive at 18 years old, he’s the <strong>4th</strong> best scorers per game for players of 23 years or less (more than <strong>0.5 goals/game</strong>). With 3 French players, France is the most represented country in this top 10.</p><p><strong>Alexandre Attia</strong><br><a href="http://www.alexattia.fr/">www.alexattia.fr</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f8761932b78" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alex-attia-blog/soccer-data-and-predicting-l%C3%A9quipe-ratings-7f8761932b78">Soccer data and predicting L’Équipe ratings</a> was originally published in <a href="https://medium.com/alex-attia-blog">Alex Attia Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Simpsons characters recognition and detection using Keras (Part 1)]]></title>
            <link>https://medium.com/alex-attia-blog/the-simpsons-character-recognition-using-keras-d8e1796eae36?source=rss----f9b0441344c5---4</link>
            <guid isPermaLink="false">https://medium.com/p/d8e1796eae36</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[the-simpsons]]></category>
            <dc:creator><![CDATA[Alexandre Attia]]></dc:creator>
            <pubDate>Fri, 02 Jun 2017 00:24:20 GMT</pubDate>
            <atom:updated>2017-07-05T20:14:00.000Z</atom:updated>
            <content:encoded><![CDATA[<p>Deep Learning : Training a convolutional neural network to recognize The Simpsons characters.</p><p>As a big Simpsons fan, I have watched a lot (and still watching) of The Simpson episodes -multiple times each- over the years. I wanted to build a neural network which can recognize characters. I don’t know right now what will be the applications of the neural net (<em>perhaps computing the characters presence in each episode)</em>.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FxTiTnwXHKlBpcdWbHG%2Ftwitter%2Fiframe&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FxTiTnwXHKlBpcdWbHG%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FxTiTnwXHKlBpcdWbHG%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="326" frameborder="0" scrolling="no"><a href="https://medium.com/media/d8d17b27ac2d7bee4c68d0e53fc383f7/href">https://medium.com/media/d8d17b27ac2d7bee4c68d0e53fc383f7/href</a></iframe><p>This project is not specially difficult but can be time consuming, because I have to manually label many pictures of each character. I didn’t find any The Simpsons characters database on the Internet so I am building it by myself (I am still labeling pictures when I have time). I think it could be useful for other ones. The dataset is already <a href="https://www.kaggle.com/alexattia/the-simpsons-characters-dataset"><strong>available on Kaggle </strong></a>with exploratory code <em>(in the Kernels section)</em>.</p><p>After learning and using TensorFlow for different projects, I want to use Keras because of its simplicity <em>(compared to TensorFlow for example) </em>and its capacity (TensorFlow backend) for experimentation. <a href="https://keras.io/">Keras</a> is a Deep Learning library written in Python by <a href="https://twitter.com/fchollet">Francois Chollet</a>. My approach to solve this problem will be based on convolutional neural networks (CNNs) : multi-layered feed-forward neural networking able to learn many features.</p><p>You can find the code on the <a href="https://github.com/alexattia/SimpsonRecognition">github repo</a> .</p><h4>Building the image dataset</h4><p>The dataset currently features 18 classes/characters <em>(the data on Kaggle contains 20 classes, but currently I used only 18 characters for training).</em> Please check the image below for the characters used.<em> </em>The pictures are under various size, scenes, could be cropped from other characters and are mainly extracted from episodes <em>(season 4 to 24)</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*64laQlCh-57A6AyTXAeRWQ.png" /><figcaption>The Simpsons characters</figcaption></figure><p>The training set includes about<strong> 1000 images per character </strong><em>(still labeling data to get to this number)</em>. The character is not necessarily centered in each image and could sometimes be with other characters (but it should be the most important part in the picture).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/313/1*qDe8Ne1you46wC4MmH1gvA.png" /><figcaption>Dataset distribution for 20 characters (on 6/19/2017)</figcaption></figure><p>With <em>label_data.py</em>, you can label data from .avi movies : you can get a cropped sub picture (left or right part) or the full picture and then label it by entering a part of the character name (<em>burns</em> for <em>Charles Montgomery Burns</em>).</p><p>To add more data, I also use the Keras model. I capture videos and get 3 pictures for each frame I analyzed (left part, right part, full) and then I ask my algorithm to classify each pictures. Afterward, I check each picture it has classified. It’s still manual but it’s faster and it’s an incremental process that’s more and more fast, particularly for “small” characters.</p><h4>Preprocessing</h4><p>The first step for preprocessing pictures is resizing them. We need to have all pictures with the same size for training. I will convert data as <em>float32</em> to save some memory and normalize them <em>(divide by 255.)</em>. Then, instead of characters name, I use numbers and thanks to Keras, I can quickly convert those categories to vectors :</p><pre>import keras <br>import cv2</pre><pre>pic_size = 64<br>num_classes = 10<br>img = cv2.resize(img, (pic_size, pic_size)).astype(&#39;float32&#39;) / 255. ...<br>y = keras.utils.to_categorical(y, num_classes)</pre><p>I am splitting my dataset into a training and a testing set : for this, I use <em>sklearn train_test_split </em>function.</p><h4>Deep Learning Model(s)</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F13rExn01UnHOEM%2Ftwitter%2Fiframe&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Fsimpson-13rExn01UnHOEM&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2F13rExn01UnHOEM%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="305" frameborder="0" scrolling="no"><a href="https://medium.com/media/27e318631269f04d0f6b1aa775dee88d/href">https://medium.com/media/27e318631269f04d0f6b1aa775dee88d/href</a></iframe><p>Now, let’s begin the “funny” part : defining our model. Right now, we’ll use a feed forward 4 convolutional layers with <strong>ReLU</strong> activation followed by a fully connected hidden layer <em>(see below for a deeper model)</em>. This model is similar to the CIFAR example from Keras documentation. I also use <strong>dropout</strong> layers to regularize and avoid overfitting. The output layer uses <strong>softmax</strong> activation to output the probability for each class. I also tried to replace ReLU by ELU <em>(like ReLU but with a mean closer to zero)</em> but it didn’t work.</p><p>Categorical Cross Entropy loss is -as often- used. And for the optimizer, I use <a href="http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf">RMS Prop</a> which is a stochastic gradient descend where we “divide the learning rate for a weight by a running average of the magnitudes of recent gradients for that weight” .</p><h4>Training the model</h4><p>For the training, the model is iterating over batches of training set (batch size : 32) for 200 epochs.</p><p>As I don’t have a huge data set, I am using <strong>data augmentation</strong> <em>(which is really simple to use with Keras library)</em>. It means doing a number of random variations over the pictures so the model never see the same picture twice. This helps prevent overfitting and helps the model generalize better.</p><pre>datagen = ImageDataGenerator(<br> featurewise_center=False, # set input mean to 0 over the dataset<br> samplewise_center=False, # set each sample mean to 0<br> featurewise_std_normalization=False, # divide inputs by std <br> samplewise_std_normalization=False, # divide each input by its std<br> rotation_range=0, # randomly rotate images in the range <br> width_shift_range=0.1, # randomly shift images horizontally <br> height_shift_range=0.1, # randomly shift images vertically <br> horizontal_flip=True, # randomly flip images<br> vertical_flip=False) # randomly flip images</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/1*HO6ahZBqykFxeLCl-5Lhrg.gif" /></figure><p>This take a while running on CPU (on my computer) so I run it on GPU with AWS EC2, Tesla K80: 8 seconds per epoch. In total, it took 20 minutes <em>(which is really quick for deep learning)</em>.</p><p>As we can on the plot, after 200 epochs, it seems to have reach the asymptote, without an obvious overfitting. Moreover, the accuracy seems good too.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/1*5_mfPWZbHTBBNtyVDCUAvQ.png" /><figcaption>Loss and Accuracy (Validation and Training) during training</figcaption></figure><h4>Classification evaluation</h4><p>Of course, right now, it’s complicated to have a true model accuracy because of the low number of pictures but as the number of pictures will grow, it will be more pertinent. Thanks to <em>sklearn</em> it’s really easy to print a classification report :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/564/1*Uo9yG41KNR-ZzV0jkuCbQw.png" /><figcaption>4 convolutional layers net</figcaption></figure><p>As you can see, the accuracy (f1-sport) is really good : above 90 % for every character except Lisa. The precision for Lisa is 82%. Maybe Lisa is mixed up with other characters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/431/1*rrc9Kdp0yxhMwlRArTHvVA.png" /></figure><p>Indeed, Lisa is often mixed up with Bart. Probably because many pictures of Lisa contain Bart too.</p><h4>Adding a threshold to improve the accuracy</h4><p>In order to improve the precision <em>(so, of course decrease the recall, but I would try to not decrease it too much)</em>, I thought that I can maybe add a threshold.</p><p>Before to talk about a threshold to improve accuracy. I just want to had a famous graph about recall and precision.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/1*IRZaGkaSiNkmb3iYTmpSKA.png" /><figcaption>Graph to define recall and precision</figcaption></figure><p>I compute some statistics about good and wrong predictions : maximum probability predictions, the probability difference between the best two candidates and the std.</p><pre>For good predictions : Max : 0.83, Difference Two First : 0.773, STD : 0.21<br>For wrong predictions : Max : 0.27, Difference Two First : 0.092, STD : 0.07</pre><p>If the probability of the predicted character (1.) is too low, the standard deviation of the prediction (2.) is too high or the probability difference between the two most likely characters (3.) is too low maybe we can say that we don’t want to predict a character at all.</p><p>So I plot those 3 values for the test set to find a line (or a hyperplane) to separate good and wrong predictions. I did it for both characters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rvT_s4m0N6Spb3xZ1HyJKg.png" /></figure><p>As you can see it’s impossible to find a linear separation and to have a simple threshold, for both graphs, between good and wrong predictions. Of course, we can see that wrong predictions are concentrated in the lower left of each graph <strong>but</strong> in this corner, there are too many good predictions too. If I choose a threshold (for example, threshold regarding the probability difference and probability of the best candidate), my recall will be lower.</p><p>Maybe the best thing, to do to improve the accuracy and not affect too much the recall, is to plot those graphs for every character or for a character with a low precision (e.g. Lisa Simpson).</p><p>Moreover, the threshold could be useful for pictures without famous characters or with not character at all. Currently, I do have a “no-character” class in my model but I can probably add with a threshold. I don’t think that we can finding the perfect formula (between the probability of the best prediction, the probability difference and the standard deviation) so I will just focus on the probability of the best prediction.</p><h4>Recall and Precision regarding the probability of the best prediction</h4><p>There is classic trade-off between recall and precision and as often we couldn’t maximize recall and precision at the same time. So, it depends what we want exactly.</p><p>Regarding the probability minimum for the predicted class, we can plot the F1-score, the recall and the precision.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*80zepF4_hROMKZuEYHkg-w.png" /><figcaption>For all classes or a particular class, precision/recall/F1 score regarding the probability minimum for the predicted class</figcaption></figure><p>As we can see, it really depends on the characters. For example, if we focus on Lisa Simpson, it would be interesting to add a probability minimum for predicted class (=0.2), but this threshold will not be really useful for all classes combined.</p><p>So regarding of the application, we should add or not a threshold around 0.2–0.4 for the probability minimum for the predicted class.</p><h4>Improving the CNN model</h4><p>As I said earlier, I have a four convolutional layers models. To make the neural net understands more details and more complexity, we can get deeper and add more convolutional layers.</p><p>It’s what I did. I tried with 6 convolutional layers and going deeper <em>(dimensions of the output space 32, 64, 512 vs 32, 64, 256, 1024)</em> . It has improved the accuracy<em> (precision and recall)</em> as you can see below. The lower precision is 0.89 for Nelson Muntz and we only had 300 training examples for this character.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*wL3KCcqp5kVAO1NIeUOsxA.png" /><figcaption>6 convolutional layers net</figcaption></figure><p>Moreover, this model converge quicker : only 40 epochs <em>(vs 200)</em>. It tooks 15 minutes to train on a Tesla K80.</p><h4>Visualizing predicted characters</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*yVOIhLPlV_qJL0mbTy_GrA.png" /><figcaption>Actual and predicted characters for 12 different characters</figcaption></figure><p>As you can see, the neural network is pretty accurate to recognize and classify characters. Then, I predict characters in a video. Indeed, the predictions are faster enough (less than 0.1 s to predict a picture) to predict multiple frames each second.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgfycat.com%2Fifr%2FHarshCaringIndianpalmsquirrel&amp;url=https%3A%2F%2Fgfycat.com%2Fgifs%2Fdetail%2FHarshCaringIndianpalmsquirrel&amp;image=https%3A%2F%2Fthumbs.gfycat.com%2FHarshCaringIndianpalmsquirrel-size_restricted.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=gfycat" width="576" height="416" frameborder="0" scrolling="no"><a href="https://medium.com/media/554603342ac2fd7dc8e6671a4fe64cb5/href">https://medium.com/media/554603342ac2fd7dc8e6671a4fe64cb5/href</a></iframe><p>If you have any questions, please feel free to contact me and moreover, if you like this post don’t hesitate to recommend it :-).</p><p>The dataset is <a href="https://www.kaggle.com/alexattia/the-simpsons-characters-dataset">on Kaggle</a>, download it and have fun !</p><blockquote>The next steps with a detection model in addition of the classification model are described<strong> </strong><a href="https://medium.com/alex-attia-blog/the-simpsons-characters-recognition-and-detection-part-2-c44f9d5abf37"><strong>in Part 2</strong></a>.</blockquote><p><strong>Alexandre Attia</strong><br><a href="http://www.alexattia.fr/">www.alexattia.fr</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d8e1796eae36" width="1" height="1" alt=""><hr><p><a href="https://medium.com/alex-attia-blog/the-simpsons-character-recognition-using-keras-d8e1796eae36">The Simpsons characters recognition and detection using Keras (Part 1)</a> was originally published in <a href="https://medium.com/alex-attia-blog">Alex Attia Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>