Objectives

In this project, you will create a Python class called Data that will extract data from a CSV file. Beginning in the second week, you will use the numpy Python library to perform computations on numeric data, selected and organized by variable name, and use the matplotlib Python library to create a variety of different visualizations. In the final step of the project, you will use the tools that you implement to analyze a real world dataset of your choice. Throughout, you will practice common operations that will come up throughout the semester.

The goals are to:

Parse and work with data stored in CSV format.
Perform computations on data based on variable name rather than numeric index.
Get accustomed to numpy's ndarray data type and matplotlib.
Practice writing vectorized code.
Review programming in Python.
Practice turning in a CS251 project and rubric on Google Classroom.

Instructions

Please read all these instructions carefully!

Due date

Your first goal is to write a draft of the Data class to submit on Google Classroom lab next week. In general, Mondays will be the day that you also turn in projects to get graded (on alternate weeks).

Try to complete Task 1 below by Monday; there might be bugs in your Data class and that's ok — you will have the chance to fix them in Week 2 without penalty. We will be grading your progress on Task 1:

1 point if you make substantial progress
0 otherwise

Get started immediately on Task 1 — the first lecture of CS251 should be all that you need to complete it.

Task 0: Download data and code templates

To start the project, download the following code template (data.py), test code, and data CSV files.

NOTE: This week we will use Python test script files to check your code, but next week and beyond we will move to Jupyter Notebooks.

Check-in submission

Download the Task 1 code templates and test code below.

Code templates	Test code
data.py	test01_read_data.py
	test02_read_data_errors.py
	test03__str__.py
	test04_get.py

Data

Download data used in the project below. Place the data in a subfolder of your working directory called data.

Dataset	Files
Iris	iris.csv
Bad Iris	iris_bad.csv
Anscombe Quartet	anscombe.csv
Spaces test	test_data_spaces.csv
Complex test	test_data_complex.csv

Don't skip reading this section

Important: this class, it is important that you write code that conforms to the code template methods specifications exactly. This means that the parameter count, data types, return types must match the docstring specifications 100%. We grade by running test code, which includes what we give you in these notebooks. We give this to you to give you rapid feedback to help you make progress. We may also have other test code that we use to grade, which makes similar assumptions. You will lose points if test code fails to run or returns incorrect results.

You should never have to modify test code!

Installing software

If you are working on your own computer, visit the Software page for installation instructions on Windows and macOS. Here is the minimum of what you need for Week 1:

Python 3.9 (install via Python website).
Numpy package (install via pip).

Task 1: Data class

The Data class:

Reads in and stores tables of data contained in .csv files.
Allows the user to select and return variables by their string name, rather than their column index.

Overview of CSV files

Your Data class parses .csv data files. Here is the assumed structure of each .csv file:

1st row: headers (name of each data variable; corresponds to the columns).
2nd row: Data type. Possible values: numeric, string, enum, or date. Numeric types can be either integers or floating point values; strings are arbitrary strings; enum implies there are a finite number of values but they can be strings or numbers; a date should be interpreted as a calendar date.
3rd row+: Actual data.

Your Data object should only hold numeric data variables (ignore non-numeric columns of data).

1a. Overview of Data class

Implement the following methods in data.py (also see Task 1b. Testing your code).

Constructor: Declare/initialize instance variables, start parsing .csv file if its path is provided.
read(filepath): Reads data from the specified .csv file into the code Data object.
__str__(): Prepares a nicely formatted string for printing Data objects
get_headers(): returns a list of all of the headers.
get_mappings(): returns a list of all of the dictionary mappings between variable name and column index.
get_num_dims(): returns the number of variables (columns).
get_num_samples(): returns the number of samples in the data set.
get_sample(rowInd): returns the rowInd-th data sample.
get_header_indices(headers): returns the variable (column) indices of the str variable names in headers

1b. Testing your code

As you go, run the following test scripts to check the correctness of your code (Download above). Check them in order — verify test01_read_data.py produces the expected results before moving onto test02_read_data_errors.py and so on.

test01_read_data.py
test02_read_data_errors.py
test03__str__.py
test04_get.py

Note about Test 2

Running test02_read_data_errors.py should produce a crash. The goal is to understand why this happens and to inform the user what problem occurs and how to correct it.

Task 2+: (Week 2)

Starting this week, all project instructions are in a Jupyter notebook. Download the following notebook and code template to get started on the next few tasks.

Remember: final submissions should include updated versions of your draft code.

Notebooks	Code templates
analysis.ipynb	analysis.py

Corrections

Please check here for any corrections to the project instructions or test code.

Submission

Checklist

Before you turn in your project, did you...

generate the requested outputs and plots inline in the appropriate notebook cells?
answer all the questions?

Turning in your project

To turn in check-in and final project submissions, you should create a .ZIP file and upload it to Google classroom. The .ZIP file should contain:

Notebooks (.ipynb)
Code (.py)
Data under 10MB, data used in extensions

Submissions should be named as follows:

yourUserName.zip.