<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sebastian Delgado Diaz on Medium]]></title>
        <description><![CDATA[Stories by Sebastian Delgado Diaz on Medium]]></description>
        <link>https://medium.com/@sebastiandelgadodiaz?source=rss-a9e037a98cf1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*RYbA1WZXTDiT7a75.jpg</url>
            <title>Stories by Sebastian Delgado Diaz on Medium</title>
            <link>https://medium.com/@sebastiandelgadodiaz?source=rss-a9e037a98cf1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 18:30:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sebastiandelgadodiaz/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Web Scraping & Automation with Selenium and Ruby]]></title>
            <link>https://medium.com/wolox/web-scraping-automation-with-selenium-and-ruby-8211e4573187?source=rss-a9e037a98cf1------2</link>
            <guid isPermaLink="false">https://medium.com/p/8211e4573187</guid>
            <category><![CDATA[web-scraping]]></category>
            <category><![CDATA[selenium]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[data-harvesting]]></category>
            <category><![CDATA[ruby]]></category>
            <dc:creator><![CDATA[Sebastian Delgado Diaz]]></dc:creator>
            <pubDate>Wed, 18 Sep 2019 12:44:41 GMT</pubDate>
            <atom:updated>2019-09-18T14:25:19.724Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CCeDNoM46fXbb0L3ljApOg.png" /></figure><p>In this post we are going to talk about what web scraping is, its uses and when you should choose another option. We are also going to go over some applications of this technique.</p><h3>What is web scraping?</h3><p>Web scraping is a technique to extract information from websites by using software programs to automate the process. These programs usually make HTTP requests directly from the code or simulate human behavior by embedding a browser inside the application. One famous example could be <a href="https://support.google.com/webmasters/answer/182072">GoogleBot</a>, Google’s web scraper, that gathers information from all the web in order to classify and rank sites for their search engine.</p><h3>Let&#39;s scrap some data!</h3><p>In the following example, we are going to get some data out of a well known online marketplace in order to gather some data that we could use later to feed a machine learning algorithm.</p><h4>Before writing code</h4><p>Before we start coding our scraper, it’s necessary to understand the page that we are going to be working on, the flow and the elements that we are going to be interacting with our code.</p><p>Let’s say that our machine learning algorithm is going to be able to predict a product’s category only by its title. In order to train it we are going to need lots of products titles and their corresponding category.</p><ul><li>From here we can infer that our scraper will need to write a term in the search bar (red circle) and then press the search button (green circle).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WJ6q46S0-pgMBzotsICjhg.png" /></figure><ul><li>This will take us to a view where we can see the categories that our search was related to (red circle) and the product title for each of the found results (green line). We will need our scraper to extract that information.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lUstBST-BlWU4oKb3Txl8Q.png" /></figure><ul><li>Only one page of product titles wouldn’t be enough data to feed a robust AI algorithm, so our scraper needs to find the pagination buttons (red line), visit page by page and extract all the titles until the next button (green line) is no longer visible (that means that we are already on the last page).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K_V13c-6lDphX72Nwh5c2w.png" /></figure><h4>Prerequisites</h4><ul><li><a href="https://chromedriver.storage.googleapis.com/index.html?path=2.38/">Download Google’s chrome driver</a>, this is the browser that our script will use.</li><li>After downloading it, move the executable file to /bin folder: $ sudo mv chromedriver /bin. In this way, you won’t need to specify the driver’s URL in your code later.</li><li>Install the Selenium web driver: $ gem install selenium-webdriver</li><li>Install interactor (we will use this gem to keep our code nice and tidy): $ gem install interactor</li><li>Or if you use a bundle like I will in this example, simply create a Gemfile and run $ bundle install</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f04c3db96f37a3767cccf8d9cde331a8/href">https://medium.com/media/f04c3db96f37a3767cccf8d9cde331a8/href</a></iframe><h4>CheatSheets!</h4><p>Finding element(s):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d2bdcae3c14fcbc054293933132c7759/href">https://medium.com/media/d2bdcae3c14fcbc054293933132c7759/href</a></iframe><p>Operations:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/23c537d4074bb77d3fd33dd81816d0a4/href">https://medium.com/media/23c537d4074bb77d3fd33dd81816d0a4/href</a></iframe><h4>Let’s code!</h4><p>We will start by defining our main class. It will contain our configuration, such as what browser we will use, what website we will visit, what we will search for and for how long we will wait until we consider something went wrong.</p><p>We will then call the class that orchestrates our scraper’s logic ScrapingOrganizer:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0a81c59d18c7b392c3bd1af9c9b60368/href">https://medium.com/media/0a81c59d18c7b392c3bd1af9c9b60368/href</a></iframe><p>Our ScrapingOrganizer will only be on charge of calling other 3 classes in order. First, it will search for an item, then find the categories related to it and then start getting the titles of all the items found. It will repeat this last action page by page.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8c0f36f338bf2b2b936c142baaa535bb/href">https://medium.com/media/8c0f36f338bf2b2b936c142baaa535bb/href</a></iframe><p>Now let’s see how each of the three steps described before works:</p><p>SearchItem, as its name says, will:</p><ol><li>Find the search bar.</li><li>Write the configured search term into the search bar.</li><li>Find the submit button.</li><li>Click the submit button.</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/298cba1f42fb3ffb0de4dba6dcfbbfbe/href">https://medium.com/media/298cba1f42fb3ffb0de4dba6dcfbbfbe/href</a></iframe><p>GetCategories will parse the page shown after the search, look for the categories’ breadcrumb, extract and print the texts found in them.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/860f23db1ac25fafd717d2bb4ec65697/href">https://medium.com/media/860f23db1ac25fafd717d2bb4ec65697/href</a></iframe><p>Now that we have the categories in GetTitles we proceed to extract all of the titles from the current page and when we are done, we go to the next page until there are no more pages to visit.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4806b167b5bece327d00ab564ffb697b/href">https://medium.com/media/4806b167b5bece327d00ab564ffb697b/href</a></iframe><p>This is how our program would look while running:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nqQMRt1H9aGj15fM.gif" /></figure><p>You can find the complete code at:</p><p><a href="https://github.com/sebasdeldi/ml_scraper/tree/master">sebasdeldi/ml_scraper</a></p><h3>When is web scraping useful?</h3><ul><li>Web scraping is extremely useful when you don’t have access to an <a href="https://en.wikipedia.org/wiki/Application_programming_interface">API</a> because it gives you the keys to all accessible internet’s data. This is especially important these days when we are constantly improving and implementing algorithms that use data as their raw material.</li><li>If you have an API available always choose it over scraping because <strong>a)</strong> it’s probably going to be easier to consume it rather than writing a web scraping script, <strong>b) </strong>web scraping is <strong>very susceptible </strong>to structural changes within the website you are working on. This means that a change on the site’s HTML structure could leave your program completely obsolete.</li><li>Any tedious job in a website requiring human labor can potentially be replaced by an automation script.</li><li>Automating Quality Assurance processes by using Selenium’s actions to check if styles, properties or HTML elements, change, have a particular value or match any comparison after or before an event in the DOM.</li></ul><p><strong>Conclusion</strong></p><p>In this post we have learned how to get data out of any website, even if it is a one-page application by using <a href="https://www.seleniumhq.org/projects/webdriver/">Selenium webdriver</a> alongside with <a href="https://www.ruby-lang.org/en/">Ruby</a>. We also went through some of the most popular uses of this technique and reviewed some cheat sheets for the most commonly used selectors and actions.</p><figure><a href="http://bit.ly/2lU14ds"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*234JOxeIXU76svp76BDkKA.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8211e4573187" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wolox/web-scraping-automation-with-selenium-and-ruby-8211e4573187">Web Scraping &amp; Automation with Selenium and Ruby</a> was originally published in <a href="https://medium.com/wolox">Wolox</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Teaching basic web development to children in Mexico]]></title>
            <link>https://medium.com/@sebastiandelgadodiaz/teaching-basic-web-development-to-children-in-mexico-5f537502e580?source=rss-a9e037a98cf1------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f537502e580</guid>
            <category><![CDATA[css]]></category>
            <category><![CDATA[mexico]]></category>
            <category><![CDATA[html5]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[teaching]]></category>
            <dc:creator><![CDATA[Sebastian Delgado Diaz]]></dc:creator>
            <pubDate>Tue, 14 Feb 2017 01:11:11 GMT</pubDate>
            <atom:updated>2017-02-14T01:11:11.785Z</atom:updated>
            <content:encoded><![CDATA[<p>As a 19 year old software engineering student, at Eafit University (Colombia), who is passionate about building and learning new things, I wanted to experience a new environment while being able to give back. In November of 2016, I initiated a journey as a volunteer in Mexico. What inspired me to go there, was the opportunity to dedicate my time and knowledge to less fortunate children, in things that would be useful to them, while doing what I love.</p><p>During this experience I had the chance to contribute to the education process of the students by teaching them basic web development using HTML5 &amp; css3, which are the languages used in the basic structure of every website.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/670/1*ye1UONNm5GsnKsJwAzrL3A.png" /><figcaption>Computer’s lab, the place where the classes were held.</figcaption></figure><h3>The process</h3><p>As soon as I got there, it was evident that they had no coding previous knowledge, but, they knew how to surf the web which made it easier to start the guiding process with them:</p><h4>1. Learning how the internet works:</h4><ul><li>What do WWW, HTML &amp; ccs stand for?</li><li>Why is internet a web?</li><li>The human body &amp; css and HTML analogy</li><li>What is an explorer and what is its job</li></ul><p>(Bones = humans as HTML=website &amp; Skin = appearance as css=website styles).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*atE48_r0lMc64WCG5jPOjw.jpeg" /><figcaption>A group of students and me</figcaption></figure><h4>2. Learning how to build a website:</h4><ul><li>Showed them Sublime3 and its purpose</li><li>We built a website where all common HTML tags were used, (I wrote and explained every tag while they did the same on their computers,) so they understood how and what every tag was used for</li><li>Explained them the scope and uses of classes and id’s on HTML tags</li><li>To test if they could identify the tags that we previously learned,I made them visit 9gag.com and tell me the tags that they could identify</li><li>Explained them how to apply styles to tags, classes and ids</li><li>Explained some css properties.</li></ul><p>At this point the kids were pretty eager about building their own websites so they constantly asked how to change “x” or “y” thing or how to make something look different so we started to train and learn how to efficiently use Google to find out programming and coding related questions and answers.</p><h4>3. The final project &amp; the technology fair:</h4><p>When I considered the students had enough knowledge to start building their own sites, I made them research about a topic that they found interesting and draw in their notebooks how they expected their website to look like.</p><p>Based on their design and the information that they researched, they started building the sites by their own, i only assisted them when they had a doubt, and I guided them towards finding the answer rather than just giving it to them.</p><p>At the end the results where pretty impressive, the last day we made a fair, where all the students showed the web sites that they built to their parents, teachers and other students.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1tlTfegVMMLhq2BC5O3WNA.jpeg" /><figcaption>Technology fair</figcaption></figure><p>Overall I consider my experience at Mexico one of the best things i have lived so far, I ate delicious food,visited incredible places, made awesome friends and had the chance to help others by doing what I enjoy the most.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f537502e580" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>