09.15.2018
After reading Jiachao’s learning journals, I found that her learning strategy is very efficient to learn a new programming language. Then I found a similar tutorial in GitHub:
This tutorial contains ten exercises for learning Pandas and each exercise has a corresponding practical dataset, which can help me to go through those commonly used Pandas functions. Instead of looking every single function in the documentation, this tutorial provides me a practical way and make me more familiar with these basic functions and operations. From now on, I am going to get my hands on these ten exercises.
09.19.2018
The first exercise will show how to use Pandas command to get a general picture of the dataset. This exercise takes chipotle sales data as the example. The dataset is a .csv format data file and has five columns, which include order_id, quantity, item_name, choice_description, item_price.
In a Python script, the first thing we need to do is to import the necessary libraries that will be used later, and apparently, I will use NumPy and Pandas all the time. So, I just import Numpy and Pandas at the very top of the script.
import pandas as pd
import numpy as npThe practices in the first exercise are relatively simple and basic and mainly show how to use Pandas to create a DataFrame and retrieve different kinds of DataFrame information by Python command.
For example, in this dataset, if I want to know the names of columns and the number of entities, I just need to run *DataFrame*.info( ), and the result will be shown as below:
Data retrieving operations is pretty straightforward to understand, and what is a little complicated is the GroupBy command. After looking through the raw dataset, I found that there were repeated item_name with different order_id, so if I want to get the quantity of a certain item, I need to merge those different order_id, and GroupBy command can help me to finish that in one single command line.
09.21.2018
Today, I will get into the second and third exercise. These two exercises are similar to the first exercise. The second exercise is going to do the filtering and sorting and the third exercise is going to group data entities.
In the second exercise, besides those sort functions I have used in the first exercise, there are three more useful functions call .str, .loc, and .iloc.
Get Hanning Su’s stories in your inbox
Join Medium for free to get updates from this writer.
When I am trying to pull out some data entities with a certain string from dataset, I can just use this ‘str’ function and set the string as a parameter. Then Python will select these certain entities and form a new DataFrame.
.loc function provides a method to access a group of rows and columns by label(s) or a boolean array. The difference between .loc and .iloc is that .loc uses index label(s) for searching and .iloc uses the number of rows or columns to retrieve the group.
The third exercise is about how to use GroupBy. GroupBy can be summarized as Split-Apply-Combine. There are many countries from different continents in this dataset. The GroupBy can help us to group these countries by continent, and then get different kinds of statistics references.
09.26.2018
Exercise 4 is about how to transfer raw data to an integral data frame and merge raw data or data frames into one data frame.
The raw data includes some lists with list names. If I want to transfer the raw data to an integral data frame, what I need to do is using .DataFrame and indicating which list is the column in the data frame.
In join and merge practices, I used .concat and .merge to combine different data frame into one. The difference between .concat and .merge is that if there is no index parameter in the data frame, .concat will add the second data frame to the end or right of the first data frame. If axis=0, the second data frame will be added to the end and if axis=1, it will be added to the right.
On the other hand, .merge can merge the data frames following the demand users want. There are two most frequently used parameters, ‘on’ and ‘how’. Users can select which column to be the index by giving ‘on’ values and decide which way they want to merge by giving ‘how’ values. For example, when the index doesn’t exist in both two data frames, how=inner indicates only keeping intersection, and how=outer indicates keeping all of the union.
09.29.2018
Today I am going to finish the last exercise in my first module. This exercise is about how to visualize the dataset. This exercise is based on the Titanic Disaster dataset available at Kaggle. There are a lot of different kinds of data charts can be drawn by Pandas, such as pie chart, line chart, scatter plot, and histogram. It is not difficult to draw these charts in Python. Those parameters are pretty straightforward to understand, and you only need to make sure that you have entered them correctly.
For example, the parameters of pie chart include proportions, which is most important for a pie chart, labels, and colors etc.
After doing this Pandas exercise, I think I have had a general picture of how Pandas works with different kind of datasets and some basic functions of modifying data frame. This tutorial is very logical and easy to follow. However there are still a lot of commands and I need to practice more on other datasets. I think practicing more will be very helpful for me to have a deeper comprehension of these commands and functions and to be able to apply Pandas to practical problems.
