<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Linnet Mbogo on Medium]]></title>
        <description><![CDATA[Stories by Linnet Mbogo on Medium]]></description>
        <link>https://medium.com/@mbogolinnet?source=rss-2433531d9215------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*WO8J3zGeV_x5n4x9</url>
            <title>Stories by Linnet Mbogo on Medium</title>
            <link>https://medium.com/@mbogolinnet?source=rss-2433531d9215------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 08:25:29 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mbogolinnet/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A Simple Guide to Web Scraping COVID-19 Data using Python]]></title>
            <link>https://medium.com/@mbogolinnet/a-simple-guide-to-web-scraping-covid-19-data-using-python-e40313847c34?source=rss-2433531d9215------2</link>
            <guid isPermaLink="false">https://medium.com/p/e40313847c34</guid>
            <category><![CDATA[web-scraping]]></category>
            <category><![CDATA[covid-19-data]]></category>
            <category><![CDATA[beautifulsoup]]></category>
            <category><![CDATA[pandas-dataframe]]></category>
            <category><![CDATA[python-programming]]></category>
            <dc:creator><![CDATA[Linnet Mbogo]]></dc:creator>
            <pubDate>Fri, 05 Jun 2020 01:59:24 GMT</pubDate>
            <atom:updated>2020-06-05T21:06:00.144Z</atom:updated>
            <content:encoded><![CDATA[<p>2020 has been a challenging year and as we come to the close of the first half of the year, it’s important to develop new versatile technical skills. I’ve simplified this code for use when you want to scrap data from any basic Wikipedia table and the steps are beginner friendly.</p><h4>Explore the Data Sources</h4><p>So far, the most visually appealing and accurate central source for <a href="https://www.google.com/search?rlz=1C1GCEA_enUS868US868&amp;q=COVID-19&amp;biw=1280&amp;bih=610">COVID-19</a> cases, deaths and recoveries in countries all over the world has been <a href="https://coronavirus.jhu.edu/map.html">the John Hopkins Univeristy of Medicine Coronavirus Resource Center</a>. However, for this practice, you’ll be scraping the data from a <a href="https://en.wikipedia.org/wiki/Template:COVID-19_pandemic_data">Wikipedia table</a> into Panda dataframes, then exporting it into an excel file. The COVID data available on <a href="https://news.google.com/covid19/map?hl=en-US&amp;gl=US&amp;ceid=US%3Aen">google</a> cites <a href="https://en.wikipedia.org/wiki/Template:COVID-19_pandemic_data">Wikipedia</a> as its data source.</p><h4>Coding Guide</h4><p>Below is a simple guide to web scraping COVID-19 data using python for a beginner:</p><p>Have <a href="https://www.python.org/downloads/">Python</a> installed (I had Python version 3.6.8 and Spyder as the IDE which is included with Anaconda.)</p><ol><li>Import requests, pandas and html. Requests are useful for sending HTTP requests, pandas will be useful for creating our dataframe (you can review more about <a href="https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.DataFrame.html">panda.Dataframe</a>)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*hDPjstU1UTWo5F7VKm5nGA.png" /><figcaption>Import</figcaption></figure><p>2. Copy the URL of the data source, in our case, the base URL represents the path to the search functionality of the website: <a href="https://en.wikipedia.org/wiki/Template:COVID-19_pandemic_data"><em>https://en.wikipedia.org/wiki/Template:COVID-19_pandemic_data</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8l3l8wnxklIA8k2aregSDQ.png" /><figcaption>Requests.get(website_url).text returns the HTML of the website. Copy the website URL which will be now referenced to as ‘website_url’.</figcaption></figure><p>3. Use Beautiful soup for parsing the HTML page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/349/1*WzEWKXrZwVWge_53pg3imQ.png" /><figcaption>Import BeautifulSoup which is a <strong>Python</strong> library that parses HTML and XML files.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*blz2SjFU64T95tKXirJ_0A.png" /><figcaption>Run this to create a Beautiful Soup object which will take HTML content as an input/list of elements and how tags are nested.</figcaption></figure><h4><strong>Explore your data source site and find elements</strong></h4><p>4. Use google chrome to open the data site and inspect your table by right clicking on the table above in Wikipedia and clicking on inspect</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6lZ5ywOdLh_KtMY0cX77YA.png" /></figure><p>5. HTML elements are identified by IDs and are enclosed in tags. The table is contained within the &lt;tbody&gt; tag as shown below…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9q_uQrO97tYHdmawwgoVeA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*vBweVOvqOZ_MzQijL1gbrQ.png" /><figcaption>The table id is ‘thetable’</figcaption></figure><p>…and its id is ‘thetable’ which is obtained from the xpath:</p><p><strong><em>//[</em></strong><a href="http://twitter.com/id"><strong><em>@id</em></strong></a><strong><em>=”thetable”]/tbody </em></strong>(<em>rightclick on tbody in the elements section and select copy xpath to get the xpath</em>)</p><p>findall table rows which are within the tag &lt;tr&gt;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PX6BllqH7PEV35C3X16iaA.png" /><figcaption>The Countries and Territories names are under ‘th’ (table head)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X_gtP9WjZQFquAEcwEdZGg.png" /><figcaption>Cases, Deaths and Recoveries are under ‘td’ (table data)</figcaption></figure><p>6. Create a loop to find results for multiple countries, as below. In this case, we’re only extracting data from the second row to the last row (which is row 230)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O_fndxksS6VYVTleOsPHDQ.png" /><figcaption>A loop defining all rows and their elements.</figcaption></figure><p>7. Use a loop that will create a table for all values in the range<br>specified.</p><p><strong>NOTE:</strong> The strip command truncates the \n that existed to show a new line. The append() command adds items to our list.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/547/1*PmqoC_c_PfjFL9aq-S3YCA.png" /><figcaption>Create a loop to retrieve all country data</figcaption></figure><p>8. Create a Dataframe using Pandas</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/806/1*Ux10uPJW_RmIz28QUMqE2g.png" /><figcaption>Dataframe</figcaption></figure><p>9. Print the Dataframe table</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/142/1*jybkPmubktCIdOhHuPiw3A.png" /><figcaption>Print the dataframe table</figcaption></figure><p>The dataframe will appear as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/746/1*JXc9EN4HJhOhdZbZql5wRQ.png" /><figcaption>Dataframe table.</figcaption></figure><p>10. Export to excel</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LE9IRxzJGA1lhKbEdfBv4A.png" /><figcaption>Export to excel</figcaption></figure><h3><strong>Entire code in Python</strong></h3><blockquote>from urllib.request import Request, urlopen<br>from lxml import html<br>import pandas as pd</blockquote><blockquote>website_url = requests.get(‘<a href="https://en.wikipedia.org/wiki/Template:2019%E2%80%9320_coronavirus_pandemic_data&#39;).text">https://en.wikipedia.org/wiki/Template:2019%E2%80%9320_coronavirus_pandemic_data&#39;).text</a></blockquote><blockquote>from bs4 import BeautifulSoup<br>soup = BeautifulSoup(website_url,’lxml’)<br>print(soup.prettify())</blockquote><blockquote>My_table = soup.find(‘table’,{‘id’:’thetable’})<br>My_table</blockquote><blockquote>tbody = My_table.find(‘tbody’)<br>tbody</blockquote><blockquote><strong>#the number of countries starts from row 2 to 230</strong><br>rows = tbody.findAll(‘tr’)<br>for i in range(2,230):<br> country = rows[i].findAll(‘th’)<br> cell = rows[i].findAll(‘td’)<br> print(‘Countries and territories = ‘+(country[1].text)+’Cases = ‘+(cell[0].text)+’Deaths = ‘+(cell[1].text)+’Recov. = ‘+(cell[2].text))</blockquote><blockquote><strong>#this loop is a Dataframe that will create a table for all values in the range<br>#specified, the strip command truncates the \n that existed to show a new line</strong></blockquote><blockquote>Countries = []<br>Cases = []<br>Deaths = []<br>Recoveries = []</blockquote><blockquote><strong>#the number of countries starts from row 2 to 230</strong><br>for i in range(2,230):<br> row=rows[i]<br> country_cell = row.findAll(‘th’)<br> country=country_cell[1]<br> Countries.append(country.text.strip())<br> <br> cells = row.findAll(‘td’)<br> case = cells[0].text.strip()<br> Cases.append(case)<br> <br> death=cells[1].text.strip()<br> Deaths.append(death)<br> <br> recov=cells[2].text.strip()<br> Recoveries.append(recov)<br> <br>df1 = pd.DataFrame(Countries, index= Countries, columns = [‘Countries’])<br>df1[‘Cases’] = Cases<br>df1[‘Deaths’] = Deaths<br>df1[‘Recov.’] = Recoveries</blockquote><blockquote><strong>#this command prints the dataframe table</strong><br>df1.head(10)<br><strong>#end of table dataframe</strong></blockquote><blockquote><strong>#export dataframe to excel</strong><br>df1.to_excel (r’C:\Users\LM\Desktop\PROJECTS\Covid-19\export_dataframe.xlsx’, index = False, header=True)</blockquote><p>PS: If you’re interested in learning how to code, I recommend starting with <a href="https://www.w3schools.com/default.asp">w3 schools</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e40313847c34" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>