<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Julian Lopez Baasch on Medium]]></title>
        <description><![CDATA[Stories by Julian Lopez Baasch on Medium]]></description>
        <link>https://medium.com/@julianlopezbaasch?source=rss-80846927e92c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*MZqxs7e4TldEp2k-xysTqQ.jpeg</url>
            <title>Stories by Julian Lopez Baasch on Medium</title>
            <link>https://medium.com/@julianlopezbaasch?source=rss-80846927e92c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 17:05:55 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@julianlopezbaasch/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to Leverage GCP’s Free Tier to Train a Custom Object Detection Model With YOLOv5]]></title>
            <link>https://medium.com/swlh/how-to-leverage-gcp-free-tier-to-train-your-custom-object-detection-with-yolov5-c0dde7a3c189?source=rss-80846927e92c------2</link>
            <guid isPermaLink="false">https://medium.com/p/c0dde7a3c189</guid>
            <category><![CDATA[object-detection]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[yolov5]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Julian Lopez Baasch]]></dc:creator>
            <pubDate>Fri, 23 Oct 2020 21:39:34 GMT</pubDate>
            <atom:updated>2020-11-07T15:25:37.448Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*75xeqn-cxgeg3VNFFNdX4w.png" /></figure><h4>Get free labeled data from GCP and fine-tune your custom object detection model with YOLOv5</h4><p>In this tutorial, I’ll show you how to leverage your GPC free tier and get annotated data for training your own model. We’ll see how to build a strong baseline despite using a small dataset (~800 samples!).</p><p>Let’s first take a look at how our resulting model infers on unseen data (remember, only 800 samples were used):</p><p><a href="https://www.linkedin.com/posts/julianlopezba_strong-baseline-after-fine-tuning-a-pre-trained-activity-6726572366618533889-lrug">Julian Lopez Baasch on LinkedIn: Strong baseline after fine-tuning a pre-trained YOLOv5 with only 800</a></p><p>GCP offers $300 credit in a <a href="https://cloud.google.com/free">3-month <strong>free</strong> trial</a> so you can explore and conduct an assessment of its platform. Basically, you can use any of their resources, like BigQuery, GPU instances, Cloud Storage, and (in particular) Cloud AI. This last product offers pre-trained API models for different tasks, like object detection, image classification, emotion recognition, language translation, and so on. In this post, I’ll show you an example with the Vision API, and in particular, its logo detection feature.</p><p>Throughout the tutorial, I’ll provide with useful gists to facilitate each step. However, I won’t get much into detail, so I recommend you to get familiar with both GCP and <a href="https://arxiv.org/abs/1506.02640">YOLO</a>.</p><p>Basically, YOLO (You Only Look Once) is a fast and light model for object detection. Although is not the <a href="https://paperswithcode.com/sota/object-detection-on-coco">most accurate model in this task</a>, it gained considerable popularity among practitioners due to its lightness. The <a href="https://arxiv.org/pdf/1506.02640.pdf">first version </a>came in 2016, with a 24 convolutional layer followed by 2 fully connected layers architecture, and from then it has been improved through 4 different versions. This year, YOLOv5 got open-sourced by Ultralitycs, and the complete<a href="https://github.com/ultralytics/yolov5"> project is shared on their Github</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5qzFR8t9-LoUVeAmcb3J0g.png" /><figcaption>YOLOv1 Architecture (source: <a href="https://medium.com/r?url=https%3A%2F%2Farxiv.org%2Fpdf%2F1506.02640.pdf">https://medium.com/r?url=https%3A%2F%2Farxiv.org%2Fpdf%2F1506.02640.pdf</a>)</figcaption></figure><h4>1) Get your data</h4><p>Once you created a GCP account you need to enable Cloud Vision and Cloud Storage APIs to build a pipeline to download and process your data before training a model. Also, you need to authorize the APIs within your GCP Project and download the credentials in JSON format.</p><p>The next step is to download an interesting video from YouTube to get data from. In our example, I used Formula 1’s videos, in which sponsors and logos are almost protagonists on screen.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F2femix89pTE%3Fstart%3D4&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D2femix89pTE&amp;image=http%3A%2F%2Fi.ytimg.com%2Fvi%2F2femix89pTE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b9fc2c32f064be517ffc42a3ab4336ec/href">https://medium.com/media/b9fc2c32f064be517ffc42a3ab4336ec/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a39c14c2022cc6ac03c3f877fbba24f4/href">https://medium.com/media/a39c14c2022cc6ac03c3f877fbba24f4/href</a></iframe><p>After downloading the video file into our local disk (it can be either in .mp4 or .mkv formats), we upload it to our desired bucket in Cloud Storage. This step is optional, as you can also use VideoIntelligence API with locally hosted videos, although this makes the API call way slower.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4b8c217fe76772db34edfb8eb10a2b24/href">https://medium.com/media/4b8c217fe76772db34edfb8eb10a2b24/href</a></iframe><p>Now, we can call the VideoIntelligenceAPI to process video annotations for logo recognition. It takes it’s time, even with a 10min long video. But don’t worry, VideoIntelligence prices are quite low compared to other Google AI services. So grab a coffee and wait for your video to process :).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e552acef3730be49e9621d05d6b6d42e/href">https://medium.com/media/e552acef3730be49e9621d05d6b6d42e/href</a></iframe><p>The API responses in a JSON alike annotation object, which you need first to serialize it in case you want to store it for later use. Since the annotations are hard to interpret, we build the following parser so that we can convert this information as a pandas DatafFame:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d59688837a2d0fc2ea99825d5cb80947/href">https://medium.com/media/d59688837a2d0fc2ea99825d5cb80947/href</a></iframe><p>Below I display an example of the parsed data into a DataFrame. VideoIntelligence API provides detailed annotations of bounding boxes in seconds and microseconds, together with the confidence of its prediction. And what the anno_to_df.py does is to populate this DataFrame with the corresponding video frame at every instant of the video.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dx7dKE45i3s90-gs-tJonw.png" /></figure><p>Besides the video, we need to extract frames from it and store them on our local disk. The idea is that we can use these frames images as inputs for our object detection model, while the output would the annotation given by Google&#39;s Vision AI. We’ll be building the dataset in the following steps.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/23b5729c0832701cc894a64511994d94/href">https://medium.com/media/23b5729c0832701cc894a64511994d94/href</a></iframe><h4>2) Process Vision AI annotated data</h4><p>But to train a model, not only do we need the bounding boxes of logos, but we should also provide the image path to each video frame. Additionally, we have to deal with the fact that our model needs also examples where logos weren’t detected. That is, if no logo is detected between frame 10 and frame 15, we should fill the DataFrame with 5 samples (rows) with bounding boxes set to 0 and class equal to None.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9fe93c37f1a4fa239f418dab7d442f44/href">https://medium.com/media/9fe93c37f1a4fa239f418dab7d442f44/href</a></iframe><p>Then, we arrive at a DataFrame similar to this one:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bgX-Ybaik3NPuWqctxX1EA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*8ZpHO-jpJzy0kGxgoY-R2A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*73xkLuWQ7YKOo2RJPoAHxg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*CWcw8bBG5J8dzqAxHVpMZg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*Uazqv5NPlpJTRRTM2ysfSA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*h4ZhZdOJJCAUezdPGSuYRw.png" /><figcaption><em>(Source: Formula 1)</em></figcaption></figure><h4>3) Create a custom dataset for YOLOv5</h4><p>You may have guessed that we are not training using ground truth data, but rather we are using Google’s Ground truth, that is, we are trying to fit a model using other’s predictions as input. The problem is that Google’s prediction is not perfect, although they are reasonably good, and with some pre-filtering, we can arrive at reliable data to train with. So to get good data, we keep only frames in which more than 4 logos are detected. That way, we avoid using frames in which the same logo is both detected and undetected, a contradiction that would lead our model to confusion. This filtering means using only ~800 frames out from the ~10000 initial frames. But we will see 800 frames is enough for obtaining a strong baseline with YOLOv5!</p><p>Now that we have our data processed we can proceed to fine-tune an object detection using <a href="https://github.com/ultralytics/yolov5">YoloV5</a>. There are some necessary steps to do before getting to the training part. These steps involve preparing paths and config files so as we build a custom dataset in YOLO/Darknet format.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/adb508ac6db842d121815f7a2a98ed45/href">https://medium.com/media/adb508ac6db842d121815f7a2a98ed45/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/98d957786f8a1bc78d1835683404a0af/href">https://medium.com/media/98d957786f8a1bc78d1835683404a0af/href</a></iframe><p>Let’s have a look at the data path structure:</p><pre>!tree data -L 2</pre><pre>data<br>├── images<br>│   ├── train<br>│   └── val<br>└── labels<br>    ├── train<br>    └── val</pre><pre>6 directories, 0 files</pre><p>Also, YOLO requires to create a dataset config (YAML file) like the following. Additionally, we have to modify the yolov5x.yaml at ./yolov5/models/ by setting the parameter nc to 85 (the number of classes in our dataset).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e6a4117a39653e85c56d5e0df33d6f81/href">https://medium.com/media/e6a4117a39653e85c56d5e0df33d6f81/href</a></iframe><h4>4) Installing and cloning resources</h4><p>Once the train and validation datasets are created, we install the required libraries. These are PyTorch and <a href="https://nvidia.github.io/apex/">APEX</a>, which although optional is highly recommended.</p><p>Finally, we clone the YOLOv5 repo from <a href="https://github.com/ultralytics/yolov5">Ultralytics</a> and we check out a specific commit to ensure reproducibility:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/975bfc1750a9241d09d7d2bd557a82c6/href">https://medium.com/media/975bfc1750a9241d09d7d2bd557a82c6/href</a></iframe><p>I created a <a href="https://colab.research.google.com/drive/1Bi8eb2WvBpHvx0Mv7buG4xgulDSz_IiO#scrollTo=HPwWO9ymKYs3">notebook</a> in Google Colab, which includes these last steps (don’t expect a more detailed explanation than shared in this post).</p><h4>5) Fine-tuning</h4><p>Finally! You made it to the last one step: fine-tuning an object detection model with our custom dataset. This step is as simple as calling the train.py script. We also can tune a couple of parameters:</p><ul><li>img 640 (resize the images to 640x640 pixels)</li><li>batch 4 (4 images per batch)</li><li>epochs 100 (train for 100 epochs)</li><li>data ../data.yaml (path to dataset config)</li><li>cfg ./models/yolov5x.yaml (model config)</li><li>weights yolov5x.pt (use pre-trained weights from the YOLOv5x model)</li><li>name yolov5x (name of our model)</li><li>cache (cache dataset images for faster training)</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fb78d2b0ab76a372217cfa095d496337/href">https://medium.com/media/fb78d2b0ab76a372217cfa095d496337/href</a></iframe><h4>6) Results</h4><p>Ultralitycs open-sourced project includes some useful utils like plot_results() that evaluates the performance of the trained model.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1268720e845edf4e15850cae861cc67d/href">https://medium.com/media/1268720e845edf4e15850cae861cc67d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*fVNpwOdVo8jdpSFeSIGvrQ.png" /><figcaption>(<em>Image by author)</em></figcaption></figure><p>It seems 100 epochs are good enough to get a decent mAP and recall despite having trained with only 800 samples. SUCH A BASELINE!!!</p><h4>7) Predictions</h4><p>We now do inference to see how the model generalizes. We pick 50 images (frames) from the validation set and move them to inference/images in yolov5 path. We the use detect.py script that the project provides to run the model on these frames. We maintain the image size we used for model training.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7321ea7f2e5338916f8016baf6f34af8/href">https://medium.com/media/7321ea7f2e5338916f8016baf6f34af8/href</a></iframe><p>Here are some of the frames, with the bounding boxes and classes that our custom model predicted for each of them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sojv6u3gS3K5-5TM9UinYA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EqTaBZLFlGkzhTeBD5p-cw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xs1Ys_Gvn3iyvjEi3OUQkg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XnjZ43MHme9gS8sYN7Kmgw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R9RGcsUKc-OOvE6LHGOrqg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BUGIliM7x7gCzieCe2svjw.jpeg" /></figure><p><strong><em>Julian Lopez Baasch </em></strong><em>is an Economist turned Data Scientist and passionate about AI. He received his MSc. and BS in Economics from Universidad Torcuato Di Tella, Buenos Aires, and his MSc. in Intelligent Interactive Systems from Universitat Pompeu Fabra, Barcelona. You can reach him on </em><a href="https://twitter.com/JulianLBaasch"><em>Twitter</em></a><em> or </em><a href="https://www.linkedin.com/in/julianlopezba/"><em>Linkedin</em></a><em>, or you can find more of his work at his </em><a href="http://julianlopezb.github.io/"><em>website</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0dde7a3c189" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/how-to-leverage-gcp-free-tier-to-train-your-custom-object-detection-with-yolov5-c0dde7a3c189">How to Leverage GCP’s Free Tier to Train a Custom Object Detection Model With YOLOv5</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mining Twitter discourse on Covid19]]></title>
            <link>https://medium.com/data-science/mining-twitter-discourse-on-covid19-a2b6df66daee?source=rss-80846927e92c------2</link>
            <guid isPermaLink="false">https://medium.com/p/a2b6df66daee</guid>
            <category><![CDATA[social-network-analysis]]></category>
            <category><![CDATA[covid19]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[twitter]]></category>
            <dc:creator><![CDATA[Julian Lopez Baasch]]></dc:creator>
            <pubDate>Tue, 02 Jun 2020 04:24:47 GMT</pubDate>
            <atom:updated>2020-06-09T17:25:21.040Z</atom:updated>
            <content:encoded><![CDATA[<h4>Understanding corona discourse through NLP techniques and Social Network Analysis</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FdoSkCn8rwFLFLmz.jpg" /><figcaption>Credits: eloutput.com</figcaption></figure><h3>Introduction</h3><p>A great activity concerning COVID-19 has emerged on Twitter when on March 11 the <a href="https://time.com/5791661/who-coronavirus-pandemic-declaration/">World Health Organization declared it a pandemic</a>. This is an expected phenomenon, as social media platforms can help track natural disasters in real-time. In this regard, Twitter appears as a preferred place for finding out what happens in this kind of situations, to such an extent that it has updated its <a href="https://blog.twitter.com/en_us/topics/company/2020/covid-19.html#unverifiedclaims">COVID-19 policies</a> claiming that it will remove misleading tweets that could “incite people to action and cause widespread panic, social unrest or large-scale disorder.”</p><p>In this post, we will study Twitter discourse around the novel virus using some data science techniques. For that reason, we used this <a href="https://www.kaggle.com/smid80/coronavirus-covid19-tweets">dataset</a> containing ~27M tweets using hashtags associated with Coronavirus. Namely:</p><p><em>#coronavirus, #coronavirusoutbreak, #coronavirusPandemic, #covid19, #covid_19, #epitwitter, #ihavecorona.</em></p><p>With that in mind, we try to answer the following questions:</p><ul><li>Do more tweets about coronavirus indicate greater public awareness and seriousness and lead to a greater public acceptance of measures to limit the spread, such as social distancing, and therefore fewer infections?</li><li>Are some hashtags spreading more rapidly than others?</li><li>Are tweeters prone to a discussion (tweet replies) or sharing (quoted tweets)?</li><li>Are people tweeting about particular symptoms?</li></ul><p>and we do it with some basic Social Networks Analysis (Part 1) and NLP (Part 2: upcoming). Different techniques and tasks have been implemented in this project, e.g. text processing, measuring graph metrics, using word2vec embeddings for a text classification model, iterative learning with maximum likelihood, applying SVD for dimensionality reduction, and so on. We provide all the results and codes in this <a href="https://github.com/JulianLopezB/covid19-tweets">repo</a> on Github.</p><p>Although results weren’t as I would have expected or desired, the exercise and study overall worth it the time spent on it. I’ve learned a lot, I’ve enhanced my hacking skills and I’ve applied techniques I never used before.</p><h3>Exploratory Analysis</h3><p>First things first, we get the data. Kaggle provides an <a href="http://networksciencebook.com/">API</a> to download datasets and I prepared this <a href="https://github.com/JulianLopezB/covid19-tweets/blob/master/notebooks/download-kaggle-dataset.ipynb">notebook</a> providing a detailed guide with the necessary steps to access the dataset with the API. An excerpt of the data is shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FaUIq1rWGPrVbkDJ.png" /></figure><p>Some basic data exploration confirms that WHO’s announcement on March 11 of the outbreak as a pandemic, triggered a huge activity on Twitter. The following figure shows the daily amount of tweets by country, with the United States counting with most of the tweets posted worldwide. However, this activity started to decrease since then.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fJv2EVOUPrxj4CLw.png" /></figure><p><a href="https://chart-studio.plotly.com/~JULIANLB/5.embed">Amount of tweets by country | line chart made by Julianlb | plotly</a></p><p>There are two properties that a tweet can have. It can be either a reply to another tweet and/or a quote (like a retweet with a personal comment on it). Otherwise, it is just a simple tweet. One can use these properties to see whether debate (replies) or sharing (quotes/retweets) explains an activity on Twitter around a specific topic. We did this, but instead of comparing topics, we compared countries. The following plot shows the quoted, replies, and the total number of tweets by country.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*afIx1krliwIJ_73h.png" /></figure><p>The same insights are displayed in the bubble plot below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uyTtOU98X0Q19sCl.png" /></figure><p>You will find an interactive version of this plot here:</p><p><a href="https://chart-studio.plotly.com/~JULIANLB/52.embed">Replies vs Quotes scatterplot (% of Total) | made by Julianlb | plotly</a></p><h3>Part 1: Social Network Analysis</h3><p>Our first approach involves graph theory. You’ll find all codes for the analysis done in this section in this <a href="https://github.com/JulianLopezB/covid19-tweets/blob/master/notebooks/Covid19-Tweets-Part1%20(SNA).ipynb">notebook</a>. Some cool resources for learning graph theory are:</p><ul><li>Jure Lecsovek’s <a href="https://web.stanford.edu/class/cs224w/">course</a> on Machine Learning with Graphs</li><li>The amusing Albert-László Barabási’s network science digital <a href="http://networksciencebook.com/">book</a>.</li></ul><p>We proceed to preprocess the data. This particularly consists of extracting the hashtags associated with each tweet and dropping the duplicates. In Part 2 we’ll filter out non-English tweets and do some text cleaning (i.e., remove symbols, mentions, URLs, punctuation, and stopwords) so that we can train a Word2Vec model (using the preprocessed corpus will allow us to transfer tweets to the vector space), whereas in Part 3 we’ll train a language model using the original text.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/126caae8e9a3d89bd448717de8138139/href">https://medium.com/media/126caae8e9a3d89bd448717de8138139/href</a></iframe><p>You can have a grasp of how data looks after preprocessing it in the following example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9BWFHur4eXtm0TmW.png" /></figure><p>Now that we have parsed the hashtags associated with each tweet, we can create replies networks from tweets that share a common hashtag. Such networks will consist of a list of edges representing the structure of the graph, i.e., the source and destination users of replies. As an example, the DataFrame below shows that user <em>45993399</em> has replied to user <em>976542720</em> a tweet associated with the hashtag <em>#socialdistancing</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/0*nmZqY3HTKWR-9H98.png" /></figure><p>These links/relationships are displayed as a network in the following graph, with node size varying according to the node degree.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/824c9e313081ead56946fab0c5a1be40/href">https://medium.com/media/824c9e313081ead56946fab0c5a1be40/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/912/0*w6G0SZlCtBQ9L3Hd.png" /></figure><p>Before creating more graphs, we should first look at the most popular hashtags in the corpus (see figure below).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wFtzAr0jqBavYizn.png" /></figure><p>We select <em>‘stayathome’, ‘lockdown’, ‘socialdistancing’, ‘flattenthecurve’, ‘masks4all’, </em>and<em> ‘trump’ </em>as our hashtags of interest for this analysis. However, we don’t use all tweets at once for creating the graphs. Instead, we compute them iteratively adding to the graph the new edges that appear on a daily basis. This way, we can see how hashtag-graphs evolve through time.</p><p>To track the structure evolution of each graph, and compare them with each other, we compute some metrics based on common properties in graph theory:</p><ul><li>Number of nodes: Number of unique users</li><li>Number of edges: Number of links or replies between users (not weighted)</li><li>Diameter: The longest of all the calculated shortest paths in a network.</li><li>Average Degree: Number of edges divided by the number of nodes.</li><li>Degree Rank: Histogram of degrees in log scale</li><li>Assortativity coefficient: <em>Assortativity</em> means a preference for a network’s nodes to attach to others that are similar. In this case, we compute the <a href="https://en.wikipedia.org/wiki/Pearson_correlation_coefficient">Pearson correlation coefficient</a> of degree between pairs of linked nodes. Positive values of <em>r</em> indicate a correlation between nodes of a similar degree, while negative values indicate relationships between nodes of different degrees.</li></ul><p>Many other interesting metrics can be calculated, like <a href="https://www.google.com/search?q=average+path+length&amp;oq=average+path+length&amp;aqs=chrome..69i57j35i39j0l6.2722j0j4&amp;sourceid=chrome&amp;ie=UTF-8">Average Path Lengt</a>h, <a href="https://en.wikipedia.org/wiki/Centrality">Centrality</a> measures, <a href="https://en.wikipedia.org/wiki/Clustering_coefficient">Clustering Coefficient</a>. But these are computationally costly, so we will just stay with the metrics mentioned before.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1ee2b44929a2a6bed93af55d3f65c8e6/href">https://medium.com/media/1ee2b44929a2a6bed93af55d3f65c8e6/href</a></iframe><p>Additionally, we design a specific metric for this analysis, which is formally defined as follows. We call it ‘new edges added’ to the graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/306/0*OGrj3N-n1aHOOY4L.png" /></figure><p>where G_t is the graph at time t and V(G), E(G) the vertices (nodes) and edges of the graph G.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c33e905f5e9517c069f2377c5e603baa/href">https://medium.com/media/c33e905f5e9517c069f2377c5e603baa/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*b6V12cgPHu3EJc3j.png" /></figure><p>The following figure shows the results of our analysis. We have computed each metric daily from March 4 to April 29, for all networks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8uBOglGlGGJoC9tZ.png" /></figure><p>An interactive dash in Plotly has been done to visualize hashtag metrics separately (see link below)</p><p><a href="https://chart-studio.plotly.com/~JULIANLB/14.embed">Hashtags Networks Metrics | line chart made by Julianlb | plotly</a></p><h3>Results</h3><p>First, it is noteworthy that #trump’s network average degree increases considerably compared to the other networks, whilst its number of unique nodes decreases. The same thing happens with the diameter of the networks. Recalling that the diameter is the longest path in the network, we can infer that #trump tweets had a lower reach than, say, #flattenthecurve. One possible reason for this is that nodes in #trump’s network are prone to connect with nodes connected with their neighbors, and we can support this hypothesis by seeing a higher assortativity coefficient. Moreover, #trump’s network degree has fewer nodes with a low degree (number of connections), and more nodes with high degree, than the other networks. This result would have been enriched with an analysis of the <a href="https://en.wikipedia.org/wiki/Degeneracy_(graph_theory)#:~:text=The%20degeneracy%20of%20a%20graph%20is%20a%20measure%20of%20how,the%20arboricity%20of%20a%20graph.&amp;text=k%2Ddegenerate%20graphs%20have%20also%20been%20called%20k%2Dinductive%20graphs.">degeneracy</a> of the networks, which probably would have given us some evidence on the possibility of Trolls explaining partially that activity.</p><p>In the opposite direction, #flattenthecuved had a great spread, reaching the highest diameter, considering its low number of nodes and average degree. Likewise, #mask4all, which is the smallest network, spread so rapidly that in a few days reached the other networks’ diameters on average.</p><p><strong><em>Julian Lopez Baasch </em></strong><em>is an Economist turned Data Scientist and passionate about AI. He received his MSc. and BS in Economics from Universidad Torcuato Di Tella, Buenos Aires, and his MSc. in Intelligent Interactive Systems from Universitat Pompeu Fabra, Barcelona. You can reach him on </em><a href="https://twitter.com/JulianLBaasch"><em>Twitter</em></a><em> or </em><a href="https://www.linkedin.com/in/julianlopezba/"><em>Linkedin</em></a><em>, or you can find more of his work at his </em><a href="http://julianlopezb.github.io/"><em>website</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a2b6df66daee" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/mining-twitter-discourse-on-covid19-a2b6df66daee">Mining Twitter discourse on Covid19</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Winners 2016–2017]]></title>
            <link>https://julianlopezbaasch.medium.com/winners-2016-2017-6dd5dca21b47?source=rss-80846927e92c------2</link>
            <guid isPermaLink="false">https://medium.com/p/6dd5dca21b47</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[stock-analysis]]></category>
            <dc:creator><![CDATA[Julian Lopez Baasch]]></dc:creator>
            <pubDate>Fri, 25 Aug 2017 03:17:50 GMT</pubDate>
            <atom:updated>2020-05-30T16:43:29.769Z</atom:updated>
            <content:encoded><![CDATA[<p>Basándome en un <a href="https://ntguardian.wordpress.com/2016/09/19/introduction-stock-market-data-python-1/">artículo</a> sobre análisis financiero en Python, se me ocurrió replicar el ejercicio que allí desarrollan con acciones de compañías que hace un rato vengo siguiendo: TESLA, NVDIA, AMD y ALI BABA. El motivo del post son dos. El primero es simplemente aprender como se manipula data de acciones utilizando la librería pandas, la cual, como veremos, nos va a permitir acceder a data histórica en (casi) tiempo real a través de una API del Yahoo! Finance. El segundo propósito es el de generar insights sobre esas compañías, las cuales han tenido un gran desempeño en el Nasdaq en los últimos dos años.</p><p>Como es sabido, en los últimos años los principales mercados mundiales han sido dominados por las computadoras. <a href="https://www.ted.com/talks/kevin_slavin_how_algorithms_shape_our_world">Los algoritmos son responsables</a> de un gran porcentaje del volumen operado diariamente, pudiendo tomar decisiones a una velocidad que ningún grupo humano pudiera hacer. La inteligencia artificial y las técnicas de data mining han ganado tal popularidad en el sector financiero que una demora de un segundo en una toma de decisión puede implicar una pérdida de millones de dólares.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f3dde0b05b1340e3fe51478b754d7949/href">https://medium.com/media/f3dde0b05b1340e3fe51478b754d7949/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Fplot.ly%2F%7EJulianLopezBaasch%2F26%2Ftesla%2F&amp;src=https%3A%2F%2Fplot.ly%2F%7EJulianLopezBaasch%2F26.embed%3Fautosize%3Dtrue&amp;type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=plot" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/78d73d8f2a5cd040364a1d72246a9046/href">https://medium.com/media/78d73d8f2a5cd040364a1d72246a9046/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Fplot.ly%2F%7EJulianLopezBaasch%2F16%2Ftime-series-with-rangeslider%2F&amp;src=https%3A%2F%2Fplot.ly%2F%7EJulianLopezBaasch%2F16.embed%3Fautosize%3Dtrue&amp;type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=plot" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/5b4368685fce5b2e9e6653ba2b9b7fc4/href">https://medium.com/media/5b4368685fce5b2e9e6653ba2b9b7fc4/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6dd5dca21b47" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[La Máquina Narcisista]]></title>
            <link>https://julianlopezbaasch.medium.com/salgo-a-la-calle-me-ato-los-cordones-de-las-asics-y-pongo-spotify-6091ed74ecba?source=rss-80846927e92c------2</link>
            <guid isPermaLink="false">https://medium.com/p/6091ed74ecba</guid>
            <category><![CDATA[recommendation-system]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[spanish]]></category>
            <dc:creator><![CDATA[Julian Lopez Baasch]]></dc:creator>
            <pubDate>Fri, 05 Feb 2016 19:53:35 GMT</pubDate>
            <atom:updated>2020-10-23T18:04:22.815Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/180/1*yJLSE4vafMLaNlWclhEF7w.jpeg" /></figure><h3>La máquina Narcisista</h3><p>Salgo a la calle, me ato los cordones de las Asics y pongo Spotify. Elijo una playlist en la categoría Running, que me sugiere ‘Empieza a correr y la música se ajustará a tu ritmo’. La playlist, que se llama Recommended For You, empieza a sonar: The Rolling Stones, Charly García, Chico Buarque, 2Pac, La Vela Puerca, Luis Alberto Spinetta y The Clash. Nada mal, salvo por La Vela Puerca, que no sé qué hacen ahí porque nunca los escuché. El algoritmo debe haber fallado.</p><p>¿Qué es un algoritmo de recomendación? Según Wikipedia, “un <a href="https://es.wikipedia.org/wiki/Sistema_de_recomendaci%C3%B3n">sistema de recomendación</a> forma parte de un tipo específico de técnica de filtro de información, los cuales presentan distintos tipos de temas o ítems de información (películas, música, libros, noticias, imágenes, páginas web, etc.) que son del interés de un usuario en particular. Generalmente, un sistema recomendador compara el perfil del usuario con algunas características de referencia de los temas, y busca predecir el “ranking” o ponderación que el usuario le daría a un ítem que aún el sistema no ha considerado.” En términos más simples, es un robot que te sugiere qué hacer en base a lo que ya hiciste.</p><p>¿Hasta qué punto nos sirve la robotización de las tomas de decisiones? Para muchos, los dudosos, o aquellos que no nacieron con una cuota de criterio suficiente, es un alivio en el angustioso proceso de elegir cual película ver, o a qué restaurante ir a comer. En este sentido el psicólogo Barry Schwartz sostiene que la libre competencia provoca daños en la sociedad, pero no en forma de injusticia, si no en dolores de cabeza. Según su <a href="https://en.wikipedia.org/wiki/The_Paradox_of_Choice">Paradox of Choice</a>, la autonomía y la libertad para elegir son cruciales para el bienestar social, sin embargo el libre albedrío en el siglo XXI poco tiene que ver con doctrinas filosóficas, podríamos decir que encaja mejor en la libertad para elegir un producto entre X marcas, Y modelos y Z lugares dónde comprarlo. Así, el costo de oportunidad (¿y si me equivoqué?) crece a mayor variedad en la oferta. Pongámoslo así: a más opciones, mayor la probabilidad de pifiarla.</p><p><em>Autonomy and Freedom of choice are critical to our well being, and choice is critical to freedom and autonomy. Nonetheless, though modern Americans have more choice than any group of people ever has before, and thus, presumably, more freedom and autonomy, we don’t seem to be benefiting from it psychologically.</em></p><p><em>— quoted from Ch.5, The Paradox of Choice, 2004</em></p><p>Bienvenida la robotización que elija por mí un puto producto, asegurándome que voy a pagar poco y dar pocas vueltas.</p><p>“Hola Julián. Hoy es un buen día para correr!” anuncia la aplicación móvil Runkeeper. Hace cuatro días que no salgo a correr, y si no fuera porque llueve, le haría caso a mi runkeeper. Falló el algoritmo.</p><p>Sabemos que las maquinas reemplazarán al hombre en muchas tareas, sobre todo en aquellas que requieren destreza física o que son sumamente tediosas. Lo que no es de esperar, o nadie esperaba hasta hace poco, es que la tecnología intervenga en el plano de la voluntad. No sé qué opinan los futurólogos al respecto, pero a mí este tipo de inquietudes me vuelan el moño. Siempre me acuerdo del final de esa película de Spielberg, A.I., en donde un grupo de humano-androides evolucionados a tal punto que su piel era casi transparente y su cuerpo casi inerte, ayuda al niño-androide, David, a cumplir su sueño de experimentar ese amor materno que vivían los niños-humanos, una especie de oasis de humanidad en mundo dominado por la inteligencia artificial. Si la tecnología nos va a evitar esfuerzo físico entonces nuestro cuerpo se adaptará darwinisticamente al punto de casi prescindir de él. Lo que no queda bien claro es qué va a pasar si la tecnología comienza a pensar por nosotros.</p><p>Un misterio. Desde el punto de vista técnico es factible que el humano evolucione al punto de terminar, como un perro, lamiéndose su propio miembro. <strong>Luitzen Egbertus Jan Brouwe</strong>, Bertus para los amigos, fue un matemático y filósofo holandés de la primera mitad del sXX, conocido por su famoso <a href="https://en.wikipedia.org/wiki/Brouwer_fixed-point_theorem">Teorema de Punto Fijo</a>. Entre otras aplicaciones, el teorema sirve para encontrar equilibrios en sistemas caóticos — como en el que nos desenvolvemos -, y su uso más destacado fue la prueba de la existencia del famoso Equilibrio de Nash. Algo me dice que vamos en ese sentido, a una contracción de las experiencias, a un encasillamiento y conformismo donde lo fortuito queda de lado. Si nuestras vidas comienzan a responder a esa mega-estructura de publicidad teledirigida inteligentemente artificiada, queda la no-experiencia. Dicen que de los errores uno aprende, y a veces nos marcan las cosas menos esperadas, como cruzarse una chica en una esquina. Claro, para eso está el correcto uso de happn©, para rebobinar y decirle a esa chica que te cruzaste lo que no te animaste en persona. Y mirá si happn© te dice que a los dos les gustan La Vela Puerca.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6091ed74ecba" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>