Sitemap
TDS Archive

An archive of data science, data analytics, data engineering, machine learning, and artificial intelligence writing from the former Towards Data Science Medium publication.

Accessing Census Data with Python

4 min readApr 20, 2020

--

With the 2020 Census already underway and the coming redistricting accessing and understanding public Census data is more relevant than ever. This post will guide you through getting your hands on the data using a Python package called CensusData to import the data into Pandas

Press enter or click to view image in full size
Different sizes, colors and genders of people grouped together to shape the USA

Census Data Sets

There are 5 data sets available through CensusData including the classic Decennial Census as well as 4 different American Community Survey (ACS) estimates.

What is the American Community Survey?

The ACS started in 2005 and replaced the long form Census that used to be sent every ten years to a subset of households along with the traditional Census. Instead, the ACS is conducted on a rolling basis and sent to roughly 3.5 million households each month. This subset is then used to create estimates of the entire population. The questions on the ACS are also more in-depth and include topics like education, employment and internet access.

The Five Data Sets

  • ACS 1-year estimates (2012–2018) For areas with populations 65,000+, most frequently updated but with the lowest “resolution” since it excludes areas with low population and has the smallest sample size
  • ACS 1-year supplemental estimates (2014–2017) Supplemental dataset that focuses on lower population areas of 20,000+
  • ACS 3-year estimates (2010–2012 to 2011–2013) For areas with populations 20,000+, very much the middle ground between the 1 and 5 years. Currently discounted by the Census Bureau but old versions can still be accessed.
  • ACS 5-year estimates (2005–2009 to 2014–2018) Data for all areas, highest resolution and largest sample size but the least current
  • Census 2010 Summary File 1 Counts every resident of the US, updated every 10 years.

Notes

  • Estimates that span multiple years are aggregates
  • You should never compare estimates from overlapping ranges, for instance comparing ACS 3-year 2010–2012 to 2011–2013

Setup and Searching for Tables

Installing is easily done using pip through your terminal.

pip install censusdata

Now you can either look up the exact table you’re interested in using documentation on the census site, for instance here for the ACS. Or you can use the search method of CensusData

import pandas as pd
import censusdata
sample = censusdata.search('acs5', 2015,'concept', 'transportation')

Here are the details of the Search Parameters taken from the documentation:

  • src (str) — Census data source: ‘acs1’ for ACS 1-year estimates, ‘acs5’ for ACS 5-year estimates, ‘acs3’ for ACS 3-year estimates, ‘acsse’ for ACS 1-year supplemental estimates, ‘sf1’ for SF1 data.
  • year (int) — Year of data.
  • field (str) — Field in which to search.
  • criterion (str) — Search criterion.
  • tabletype (str, optional) — Type of table from which variables are drawn (only applicable to ACS data). Options are ‘detail’ (detail tables), ‘subject’ (subject tables), ‘profile’ (data profile tables), ‘cprofile’ (comparison profile tables).

So in the above search query, we’re looking for 5 year ACS estimates from 2015 with ‘transportation’ in the concept. Now this search will return a list of 3 tuples containing variable names, concepts, and labels matching the search criterion. This list can be quite large.

print(len(sample))>>>3630

Let’s examine two of the first tuples in detail to understand what they’re telling us.

print(sample[0])>>>('B08006_001E', 'B08006.  Sex of Workers by Means of Transportation to Work', 'Total:')print(sample[2])>>>('B08006_002E', 'B08006.  Sex of Workers by Means of Transportation to Work', 'Car, truck, or van:')

The first element is the Variable Name and contains two elements, the parent table and the sub table designation. Both entries share the same parent table ‘B08006’ which corresponds to the second element the Concept, ‘B08006. Sex of Workers by Means of Transportation to Work’. The final element is the label which corresponds to their subtable.

Printing Tables and Find Geographies

Once you know the parent table you’re interested in you can use the print table command to get a clean readout of all of the subtables.

censusdata.printtable(censusdata.censustable('acs5', 2015, 'B23025'))
Press enter or click to view image in full size
Screenshot of readout of printtable statement
Included as an image since it doesn’t format well with Medium

The final piece you’ll need in order to download some Census data is the geography codes of the region you’re interested in. We can use the geographies method in order to explore this information.

states = censusdata.geographies(censusdata.censusgeo([('state', '*')]), 'acs5', 2015)

This query will return a dictionary object where each key is the name of a state. Below is a sample of the first two elements.

{'Alabama': censusgeo((('state', '01'),)), 
'Alaska': censusgeo((('state', '02'),))}

So in order to find the code for New York, we would need to print the corresponding value.

print(states['New York']
>>>Summary level: 040, state:36

We’re interested in the second value, 36, which can then be used to get all the county codes for that state.

counties = censusdata.geographies(censusdata.censusgeo([('state', '36'), ('county', '*')]), 'acs5', 2015)print(counties)
>>>Summary level: 040, state:36
{'Queens County, New York':
censusgeo((('state', '36'), ('county', '081'))),
'Rensselaer County, New York':
censusgeo((('state', '36'), ('county', '083'))),
'Richmond County, New York':
censusgeo((('state', '36'), ('county', '085'))),
...}

Downloading Data

Now we have everything we’ll need to download our first set of Census data!

data = censusdata.download('acs5', 2015,
censusdata.censusgeo([('state', '36'),
('county', '081'),
('block group', '*')]),
['B23025_001E', 'B23025_002E', 'B23025_003E',
'B23025_004E', 'B23025_005E',
'B23025_006E', 'B23025_007E'])

That query will store the requested data as a Pandas dataframe that can be accessed using standard Pandas methods.

print(data.head)

Congratulations! You’ve officially accessed the public US Census Data.

References:

https://jtleider.github.io/censusdata/

Get Jackson Gilkey’s stories in your inbox

Join Medium for free to get updates from this writer.

https://www.census.gov/en.html

--

--

Written by Jackson Gilkey

Data Scientist. Studied at Flatiron in NYC. Strong interest in Graph Theory, Mapping, GIS, NLP, Non-linear Modeling, Python, Urban Agriculture and Mycology.

Published in TDS Archive

An archive of data science, data analytics, data engineering, machine learning, and artificial intelligence writing from the former Towards Data Science Medium publication.