<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vicky Bee Wu on Medium]]></title>
        <description><![CDATA[Stories by Vicky Bee Wu on Medium]]></description>
        <link>https://medium.com/@vickyyy-wu?source=rss-f805b9c13258------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*Fo3MX7gafKbg5gn1</url>
            <title>Stories by Vicky Bee Wu on Medium</title>
            <link>https://medium.com/@vickyyy-wu?source=rss-f805b9c13258------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 04:06:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vickyyy-wu/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Beginner’s guide for MapReduce with MRJob in Python]]></title>
            <link>https://medium.com/datable/beginners-guide-for-mapreduce-with-mrjob-in-python-dbd2e7dd0f86?source=rss-f805b9c13258------2</link>
            <guid isPermaLink="false">https://medium.com/p/dbd2e7dd0f86</guid>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[mapreduce]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[hadoop]]></category>
            <dc:creator><![CDATA[Vicky Bee Wu]]></dc:creator>
            <pubDate>Sun, 07 Mar 2021 14:33:22 GMT</pubDate>
            <atom:updated>2021-03-07T21:59:15.741Z</atom:updated>
            <content:encoded><![CDATA[<h3>Hitchhiker guide to MapReduce with MRJob in Python</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q39d9M88lHxcHHwcuK8jsg.png" /></figure><ol><li><strong>What is MapReduce?</strong></li></ol><p>MapReduce is a programming paradigm for big data processing, where data is partitioned into distributed chunks and processed by a series of transformations.</p><p>The MapReduce programming paradigm processes data in 2 operations: map() and then reduce(). map() is a user-defined function that maps each data record in the data collection. reduce() groups the output of map() with another user-defined function.</p><p><strong>2. MapReduce Pipeline</strong></p><p>MapReduce works on (key, value) pairs by performing the below steps:</p><p><em>INPUT: list of key-value pairs (k1, v1)</em></p><p><em>MAP: (k1, v1) &gt;&gt; [list of (k2, v2)]</em></p><p><em>SHUFFLE: combine (k2, v2) &gt;&gt; (k2, [list of v2])</em></p><p><em>REDUCE: (k2, [list of v2]) &gt;&gt; (k3, v3)</em></p><p><em>OUTPUT: list of (k3, v3)</em></p><p>To put things into perspective, let’s go through these steps with a simple example of WordCount. Say we have a document that contains two lines:</p><pre>the article is about the mapreduce paradigm</pre><pre>mapreduce is part of big data ecosystem</pre><p>We want to compute the frequency of each word in this document. We will work out this problem from the bottom up by first identifying what we want as the end output.</p><p>INPUT: (document: text line)</p><p>OUPUT: (unique word, frequency)</p><p>The end output we want is a list of (unique word, frequency) pairs, and to achieve that we need an intermediate output of each word in the document and a count of 1 — (word, 1). Having a list of (word, 1) will allow us to combine by words and add all the 1s of the same word together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*18QSkDjGy3DM_xNzJFWjGQ.jpeg" /></figure><p>Next, SHUFFLE is performed on intermediate output 1. It groups all the (word, 1) pairs by words and produces intermediate output 2:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gZiHMqytGidGRA_Ypblswg.jpeg" /></figure><p>Finally a Reducer is used to sum up all the 1s together, which gives us the end output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7deUXsrEdHOrZX0i1_Tc3w.jpeg" /></figure><p><strong>3. MapReduce application in Python — Introducing mrjob</strong></p><p><a href="https://mrjob.readthedocs.io/en/latest/guides/why-mrjob.html#overview"><strong>mrjob</strong></a> is a library that allows you to write Python programs that run on Hadoop. With mrjob, you can test your code locally without installing Hadoop or run it on a cluster of your choice. “If you don’t want to be a Hadoop expert but need the computing power of MapReduce, mrjob might be just the thing for you.”</p><p>First install mrjob by running:</p><pre>pip install mrjob</pre><p>Then import MRJob</p><pre>from mrjob.job import MRJob</pre><p>Afterwards you can write a <strong>job</strong> that is defined by a class and contains a series of steps which include mappers, combiners and a reducers. Mappers, combiners and reducers are optional, though you must have at least one.</p><p><strong>Example 1</strong>: Assuming we have a text file named <em>book.txt</em>. Our task is to count the frequency of words in the first 1000 lines of this document.</p><p>First we create a class called WordCount, remember to put ‘MRJob’ in the argument. Then we create a mapper function and a reducer function inside the class</p><pre>class WordCount(MRJob):</pre><pre> def mapper(self, key_1, value_1):<br>  yield (key_2, value_2)</pre><pre> def reducer(self, key_2, value_2):<br>   yield (key_3, value_3)</pre><p>Inside the mapper function, our (key, value) input pairs are (_, line) and our output pairs should be (word, 1). Note that (_, line) tells MRJob to ignore the key and take each line of the document as the value.</p><pre>def mapper(self, _, line):<br> for word in line:<br>  if length(word) &gt; 0:<br>   yield (word, 1)</pre><p>Inside the reducer function, our (key, value) input pairs are (word, count) and our output pairs should be (word, sum(count)).</p><pre>def reducer(self, word, count):<br> yield (word, sum(count))</pre><p>This class is then complete. Finally we just need to assign this class to a task.</p><pre>task0 = WordCount(args = [])</pre><p>To print out the output, we will read the txt file in streaming fashion which means we will read it row by row, and perform task0 on each row.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7a4df360726a9474e70ff2df03b6746f/href">https://medium.com/media/7a4df360726a9474e70ff2df03b6746f/href</a></iframe><p><strong>Example 2</strong>: We have a file ‘citibike.csv’ which contains the data of citibike trips in NYC. Each row of the data represents a unique trip by a biker from station A to station B. The columns we need for this task are “start_station_name”, “end_station_name” and “gender”. Each row of the data represents a unique trip.</p><p>Now we are asked to compute the stations with the most riders started from, per each gender. Meaning, what was the station name with the highest number of bike pickups for female riders, for male riders and for unknown riders.</p><p>The output will be a list of tuples, each includes a gender label (as indicated below) and another tuple consisting of a station name, and the total number of trips started at that station for that gender.<em> ie: (0: (station_A, 100))</em></p><p>The label mapping for the gender column in citibike.csv is: (‘0’=<strong>Unknown</strong>; ‘1’=<strong>Male</strong>; ‘2’=<strong>Female</strong>).</p><p>Same as before, we should start thinking from the bottom up. The output we want in the end is a nested tuple:</p><pre>#desired output<br><strong>(gender_label, (max_start_station_name, max_number_of_trips))</strong></pre><p>where <em>gender_label</em> is the key and <em>(start_station_name, max_number_of_trips) </em>is the value. This can be achieved by using a reducer to extract the max value out of all values if we have an intermediate key-value pair:</p><pre>#intermediate output1<br><strong>(gender_label, (start_station_name, number_of_trips_for_station))</strong></pre><p>In intermediate output 1, we have <em>gender_label</em> as the key and <em>(start_station_name, number_of_trips_for_station) </em>as the value. This means for each gender, we keep a count of each station’s number of pickups, which will look like this: (numbers are made up)</p><pre>(0, (station_A, 4))<br>(0, (station_B, 1))<br>(0, (station_C, 210))<br>(1, (station_A, 1))<br>(1, (station_B, 378))<br>(1, (station_C, 3))<br>(2, (station_A, 5))<br>(2, (station_B, 4))<br>(2, (station_C, 0))</pre><p>This can be rewritten (mapped) into intermediate output2 by just switching the key from <em>gender_label</em> to (<em>gender, station)</em> pair.</p><pre># intermediate output2<strong><br>((gender_label, start_station_name), number_of_trips_for_station))</strong></pre><p>Now we have something easy to achieve. We can just keep a record of each row’s <em>gender_label</em> and <em>start_station_name </em>with a count of 1, and sum up all the count by key (<em>gender_label, start_station_name) like the below:</em></p><pre>((0, station_A), 1 1 1 1 ))<br>((0, station_B), 1))<br>((0, station_C), 1 1 1 ...))<br>((1, station_A), 1))<br>((1, station_B), 1 1 1 ...))<br>((1, station_C), 1 1 1))<br>((2, station_A), 1 1 1 1 1))<br>((2, station_B), 1 1 1 1))<br>((2, station_C), ))</pre><p>This can be easily done using a reducer on our record of of each row’s <em>gender_label</em> and <em>start_station_name </em>with a count of 1. That will be our intermediate output 3:</p><pre># intermediate output3<br>((0, station_A), 1))<br>((0, station_A), 1))<br>((0, station_A), 1))<br>((0, station_C), 1))<br>((1, station_B), 1))<br>((0, station_B), 1))<br>((0, station_C), 1))<br>((2, station_A), 1))<br>((1, station_B), 1))<br>((1, station_C), 1))<br>...</pre><p>To code this in MRJob, we will need a job with 2 steps and each step containing a mapper and a reducer. This requires us to add a step() function into the mrjob class to tell mrjob which mapper and reducer are used together.</p><pre>def steps():<br> return [MRStep(mapper=self.mapper1,<br>                reducer=self.reducer1),<br>         MRStep(mapper = self.mapper2,<br>                reducer=self.reducer2)]</pre><p>Now we can write our mappers and reducers from the top down. For mapper1, we want to achieve intermediate output3:</p><pre>def mapper1(self, key, row):<br> yield ((row[&#39;gender&#39;], row[&#39;start_station_name&#39;]), 1)</pre><p>Then we can use a reducer to sum up all the counts to achieve intermediate output2:</p><pre>def reducer1(self, gender_station, count):<br> yield (gender_station, sum(count))</pre><p>Then we use a mapper to restructure the key-value pair so that we have only gender as the key and station-count pair as the value. This will be out intermediate output 1:</p><pre>def mapper2(self, gender_station, count):<br> gender, station = gender_station<br> yield (gender, (station, count))</pre><p>Finally we just need to pick out the maximum value from the key-value pair by applying a reducer to intermediate output1:</p><pre>def reducer2(self, gender, station_count):<br> genderMap = {&#39;0&#39;:&#39;Unknown&#39;, &#39;1&#39;:&#39;Male&#39;, &#39;2&#39;:&#39;Female&#39;} <br> yield (genderMap[gender], max(station_count, key = lambda x : x[1]))</pre><p>The entire code for this task:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bd664af90f422cf0f37bbbc63ecd6d0b/href">https://medium.com/media/bd664af90f422cf0f37bbbc63ecd6d0b/href</a></iframe><p>You can find more information on how to write a job in MRJob <a href="https://mrjob.readthedocs.io/en/latest/guides/writing-mrjobs.html#defining-steps">here</a>. Thanks for reading and have fun creating jobs!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dbd2e7dd0f86" width="1" height="1" alt=""><hr><p><a href="https://medium.com/datable/beginners-guide-for-mapreduce-with-mrjob-in-python-dbd2e7dd0f86">Beginner’s guide for MapReduce with MRJob in Python</a> was originally published in <a href="https://medium.com/datable">Datable</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Share of Individuals Using the Internet Visualizations]]></title>
            <link>https://vickyyy-wu.medium.com/share-of-individuals-using-the-internet-visualizations-fc91c6fa8f5a?source=rss-f805b9c13258------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc91c6fa8f5a</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[plotly]]></category>
            <dc:creator><![CDATA[Vicky Bee Wu]]></dc:creator>
            <pubDate>Tue, 23 Feb 2021 03:33:48 GMT</pubDate>
            <atom:updated>2021-02-23T15:35:18.017Z</atom:updated>
            <content:encoded><![CDATA[<p>The article contains three visualizations of the dataset share-of-individuals-using-the-internet from ourworldindata.org. The dataset describes the percentage of the population in a country or a region that uses the Internet from 1990 to 2017. The Entity column contains both country-level data and region level data. This dataset is displayed in the table below.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Finternetshare%2Fembed%2F&amp;display_name=Datapane&amp;url=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Finternetshare%2F&amp;image=https%3A%2F%2Fstorage.googleapis.com%2Fdatapane-files-prod%2Fdp%2Fthumbnails%2Fd2951675-89c9-47f5-8c27-e3b9ebf0ef88.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=datapane" width="800" height="625" frameborder="0" scrolling="no"><a href="https://medium.com/media/93224ad9dfe659219849cf2778ede751/href">https://medium.com/media/93224ad9dfe659219849cf2778ede751/href</a></iframe><ol><li>In the first visualization, I took out the 2016 data for 10 aggregated regions including Arab World, Caribbean small states, Central Europe and the Baltics, East Asia &amp; Pacific, Europe &amp; Central Asia, Latin America &amp; Caribbean, Middle East &amp; North Africa, Pacific island small states, South Asia and Sub-Saharan Africa. To compare the percentage of the population using the Internet in these regions, I used a bar chart with each bar representing a region. Europe and Central Asia had over 70% of the population using the Internet while in Sub-Saharan Africa, Internet was still not available to most people in 2016.</li></ol><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Finternet-usage-2016-regions%2Fembed%2F&amp;display_name=Datapane&amp;url=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Finternet-usage-2016-regions%2F&amp;image=https%3A%2F%2Fstorage.googleapis.com%2Fdatapane-files-prod%2Fdp%2Fthumbnails%2F0b16aba6-7c7c-4bf5-a34c-92cea346546b.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=datapane" width="800" height="625" frameborder="0" scrolling="no"><a href="https://medium.com/media/3b12825363102559e05a86bfde2f1909/href">https://medium.com/media/3b12825363102559e05a86bfde2f1909/href</a></iframe><p>2. In the second visualization, I wanted to compare the Internet usage for these same regions but for 3 different years (2000, 2010, 2015). I chose to use a stacked bar chart with each bar representing a region and each color portion representing a year. We can see that all 10 regions saw a growing number of the population using the Internet. However, one problem with this chart is that the y-axis shows the summed up values for each year which makes it confusing. (How can I solve this?)</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Finternet-usage-of-3-years%2Fembed%2F&amp;display_name=Datapane&amp;url=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Finternet-usage-of-3-years%2F&amp;image=https%3A%2F%2Fstorage.googleapis.com%2Fdatapane-files-prod%2Fdp%2Fthumbnails%2Fa609f3e5-63e1-4ac2-8d2e-81fc3001fc6b.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=datapane" width="800" height="625" frameborder="0" scrolling="no"><a href="https://medium.com/media/fbd965116fc19fc08d6c7469cbe967f4/href">https://medium.com/media/fbd965116fc19fc08d6c7469cbe967f4/href</a></iframe><p>3. In the third visualization, I created a bar chart with a drop down menu to display the Internet usage for the most recent year available (2016) for three regions (East Europe and Central Asia, Arab World, South Asia), as well as all countries inside these regions. South Asia, compared to Arab World and East Europe and Central Asia, has the lowest Internet accessibility in 2016, with less than half of the population using the Internet.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Finternet-usage-of-three-selected-regions-overtime%2Fembed%2F&amp;display_name=Datapane&amp;url=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Finternet-usage-of-three-selected-regions-overtime%2F&amp;image=https%3A%2F%2Fstorage.googleapis.com%2Fdatapane-files-prod%2Fdp%2Fthumbnails%2F5c85b539-dd32-4878-bedb-74d49970dc3d.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=datapane" width="800" height="625" frameborder="0" scrolling="no"><a href="https://medium.com/media/aca9fb6adb1d1f522644dd98c36d5dc7/href">https://medium.com/media/aca9fb6adb1d1f522644dd98c36d5dc7/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc91c6fa8f5a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gapminder Cellphone Dataset Visualizations]]></title>
            <link>https://vickyyy-wu.medium.com/gapminder-cellphone-dataset-visualizations-d18abe800409?source=rss-f805b9c13258------2</link>
            <guid isPermaLink="false">https://medium.com/p/d18abe800409</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[gapminder]]></category>
            <category><![CDATA[plotly]]></category>
            <dc:creator><![CDATA[Vicky Bee Wu]]></dc:creator>
            <pubDate>Tue, 23 Feb 2021 02:29:27 GMT</pubDate>
            <atom:updated>2021-03-01T15:01:47.688Z</atom:updated>
            <content:encoded><![CDATA[<p>This article contains 3 visualizations for the cellphone_per_100_people dataset from gapminder.org. Below is the data table, you can search for a specific value by typing it in the box under the variable name.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Fcell-phone-per-100-people%2Fembed%2F&amp;display_name=Datapane&amp;url=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Fcell-phone-per-100-people%2F&amp;image=https%3A%2F%2Fstorage.googleapis.com%2Fdatapane-files-prod%2Fdp%2Fthumbnails%2F5e12a873-cdb1-41b6-b4c0-ca50c32dd8a4.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=datapane" width="800" height="625" frameborder="0" scrolling="no"><a href="https://medium.com/media/74017d6414af7603e8fb00e0cfc4bd5b/href">https://medium.com/media/74017d6414af7603e8fb00e0cfc4bd5b/href</a></iframe><p>Here I randomly selected one country — Brazil to observe its cellphone ownership per 100 people over time. Notice that in 2014, Brazil reached its peak of cellphone prevalence at an average of 1.38 phone per person. In 2019, the average ownership dropped down to around 1 phone per person.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hJcjljZsAY6V04q9xGdEsA.png" /></figure><p>2. For this second visualization, I selected the four countries that have the highest average cellphone ownership overtime, which are United Arab Emirates, Italy, Luxembourg and Finland. We can see from the chart that Finland was the first country among the four to adopt cellphones in the lates 80s. Around 50% of Finland’s population had a cellphone in the late 90s. UAE was the last country to reach around 1 phone per person but was leading in 2019 with an average of 2 phones per person.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Fcellphone-ownership-per-100-people-in-4-countries-with-the-most-cellphone-usage%2Fembed%2F&amp;display_name=Datapane&amp;url=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Fcellphone-ownership-per-100-people-in-4-countries-with-the-most-cellphone-usage%2F&amp;image=https%3A%2F%2Fstorage.googleapis.com%2Fdatapane-files-prod%2Fdp%2Fthumbnails%2F961c59c8-60c8-48d0-8ffe-e9101c62ee5a.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=datapane" width="800" height="625" frameborder="0" scrolling="no"><a href="https://medium.com/media/2d058ff8469fad7cf2f222c92810526f/href">https://medium.com/media/2d058ff8469fad7cf2f222c92810526f/href</a></iframe><p>3. In this third chart, I included all the countries from the dataset. You can select a country to see how its cellphone ownership changes overtime.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Fcellphone-ownership-per-100-people-by-country%2Fembed%2F&amp;display_name=Datapane&amp;url=https%3A%2F%2Fdatapane.com%2Fu%2Fvickybwu%2Freports%2Fcellphone-ownership-per-100-people-by-country%2F&amp;image=https%3A%2F%2Fstorage.googleapis.com%2Fdatapane-files-prod%2Fdp%2Fthumbnails%2Faa817fcf-b81a-4028-b16e-3f2010fd5e10.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=datapane" width="800" height="625" frameborder="0" scrolling="no"><a href="https://medium.com/media/c1cb1392291424e3fea3826250f1b33f/href">https://medium.com/media/c1cb1392291424e3fea3826250f1b33f/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d18abe800409" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data visualization learning journal 1]]></title>
            <link>https://medium.com/datable/data-visualization-learning-journal-1-7b8ca41b1067?source=rss-f805b9c13258------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b8ca41b1067</guid>
            <category><![CDATA[data-visualization]]></category>
            <dc:creator><![CDATA[Vicky Bee Wu]]></dc:creator>
            <pubDate>Mon, 08 Feb 2021 21:39:22 GMT</pubDate>
            <atom:updated>2021-03-07T14:43:18.852Z</atom:updated>
            <content:encoded><![CDATA[<h3>Data Visualization Fundamentals</h3><h3>1. Using Data Visualization to Find Insights in Data</h3><p>Data is invisible.</p><p>Data tables are also a type of data visualization, but tables alone don’t allow us to immediately identify patterns within the data</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/0*nvGLYb1VU3M-m6Ea" /></figure><h3>2. How to visualize?</h3><h3>- types of visualization</h3><ol><li>Tables are very powerful when you are dealing with a relatively small number of data points and data with a single variable. (Edward Tufte suggested including small chart pieces within table columns.)</li><li>Bar charts are perfect for categorical comparison.</li><li>Line charts are especially suited for showing temporal evolutions. Like time-series data (where one variable is time/date/numbers of the same intervals).</li><li>Scatter plot. (Date/Time data of different intervals is not time-series data and should be visualized in scatter plots, using line chart in this situation could be misguiding because it ignores the potential changes during the intervals, see the below example)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*fwBRAYWD0IrrH1pYT3kjPQ.png" /></figure><ol><li>Mapping graphs connect data to the physical world.</li><li>Graphs are all about showing the inter-connections (edges) in your data points (nodes)</li></ol><h3>- analyze and interpret what you see</h3><ul><li>What can I see in this image? Is it what I expected?</li><li>Are there any interesting patterns?</li><li>What does this mean in the context of the data?</li></ul><h3>- document insights and steps along the way</h3><p>It’s a good idea to start the documentation by writing down these initial thoughts.</p><ul><li>Why have I created this chart?</li><li>What have I done to the data to create it?</li><li>What does this chart tell me?</li></ul><h3>- transform data</h3><p>Most common transformation is log(). There are also other transformation depending on the data.</p><p>An example: when you have extreme values in your data, your barchart will look very unbalanced, you can barely tell the difference between b,c,e.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/464/1*-1d6R8XCPxCXp6r8Y-cjpQ.png" /></figure><p>But if we transform the values to be log2 based, all the values will fall below 10. The chart looks much better.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*qv_0w0mNyPWtB9DC97bu1w.png" /></figure><h3>- zooming</h3><p>To have look at a certain detail in the visualization</p><h3>- aggregation</h3><ul><li>To combine many data points into a single group</li></ul><h3>-filtering</h3><p>To (temporarily) remove data points that are not in our major focus</p><h3>- outlier removal</h3><p>To get rid of single points that are not representative for 99% of the dataset.</p><h3>Which Tools to Use</h3><p>Visualization and data wrangling should be easy and cheap, better done in the same software than separate softwares.</p><h3>3. Data visualization DIY: Our Top Tools</h3><h3><a href="http://www.tableausoftware.com/public">Tableau Public</a></h3><p>you can make pretty complex visualizations simply and easily with up to 100,000 rows.</p><p>Tableau is designed for PCs, although a Mac version is in the works</p><h3><a href="http://www.google.com/google-d-s/spreadsheets/">Google spreadsheet charts</a></h3><p>including the animated bubbles used by Hans Rosling’s <a href="http://www.gapminder.org/">Gapminder</a></p><p>They are pretty design-neutral, which is useful in small charts</p><h3><a href="http://www-958.ibm.com/software/data/cognos/manyeyes/">Many Eyes</a></h3><p>it was a unique exercise in allowing people to simply upload datasets and visualise them</p><p>you can’t edit the data once you’ve uploaded it</p><h3><a href="http://colorbrewer2.org/">Color Brewer</a></h3><p>Not, strictly speaking, a visualization tool</p><p>is really for choosing map colors</p><p>There are lots of others out there too, including:</p><ul><li><a href="http://chartsbin.com/">Chartsbin</a> A tool for creating clickable world maps</li><li><a href="http://www.icharts.net/">iCharts</a> Specialises in small chart widgets</li><li><a href="http://geocommons.com/">Geocommons</a> Shares data and boundary data to create global and local maps</li></ul><h3>4 . Information Is Beautiful Awards</h3><p><a href="https://www.informationisbeautifulawards.com/showcase?page=1&amp;pcategory=winner&amp;type=awards">https://www.informationisbeautifulawards.com/showcase?page=1&amp;pcategory=winner&amp;type=awards</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/0*gx0BQ95BanJR-bZo" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*apbeEp9XQgOYeVpw" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HZGE0VsyQWIsoOSH" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/0*rn5sdKiIhDw_SKIO" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/0*Z45-TPX95DbJ68bF" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*I85aYRrI7SgE82Bl" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*R9WXhZRvWzhrqbJ_" /></figure><p><a href="https://www.informationisbeautifulawards.com/showcase/2508-all-the-government-barbecues">All the government barbecues</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b8ca41b1067" width="1" height="1" alt=""><hr><p><a href="https://medium.com/datable/data-visualization-learning-journal-1-7b8ca41b1067">Data visualization learning journal 1</a> was originally published in <a href="https://medium.com/datable">Datable</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>