Week 2: Collecting Data, Sourcing Data and Big Data

Additional Videos
Exploring the Role of AI in Optimizing the Data Pipeline
A Bite of Me
Nicholas Felton: A Quantified Life
The House that Spied on Me
Darknet (The Most Connected Human)

Data Miner
Python Beautiful Soup

Guest Speaker
Olivia Robinson, Data Director at Color of Change. Watch video below.

This week we will be jumping into the world of data and looking at what is openly available to us. We will also be talking about all the different types of data (there are lots) and all the different formatting types it could possibly be saved as. We will also be talking about some great tools to collect your own data. Finally we will be demystifying what “Big Data” actually is.

Lecture Slides


  1. Find an interesting open data set. Present it to the class in the class Slack channel giving your reasons why you chose it and what interesting data points you noticed in it. This can be a website or open data set, lets avoid APIs for now.
  2. Watch the video of the Week 3 lecture and come with questions next week.
  3. Reading: Chapter “Why Visualize: From Information to Wisdom”, PagesĀ 5-19
  4. Start collecting your own personal data that you can use in your final project. You must have at least three different data sets to work with. Some examples could be location data, exercise data, type of food you eat, who you interact with, what time you wake up and go to sleep, your mood for the day, etc.

    Here are a few tools to help you do this:
    Gyroscope, best main tool.
    Reporter App simple diary app created by Nicholas Felton.
    Daytum track your daily events on this website.
    Apple Data, download all the data Apple has on you.
    Facebook Data, download some of the data Facebook has on you.
    iPhone Health App track steps, calories burned, goals, etc.
    And of course devices like the Apple Watch, FitBit, Nike Fuelband, etc.