Objectives

In this project, you will create a Python class called Data that will extract data from a CSV file. Beginning in the second week, you will use the numpy Python library to perform computations on numeric data, selected and organized by variable name, and use the matplotlib Python library to create a variety of different visualizations. In the final step of the project, you will use the tools that you implement to analyze a real world dataset of your choice. Throughout, you will practice common operations that will come up throughout the semester.

The goals are to:

Parse and work with numeric and categorical data stored in CSV format.
Handle missing numeric and categorical data.
Perform computations on data based on variable name rather than numeric index.
Get accustomed to numpy's ndarray data type and matplotlib.
Practice writing vectorized code.
Review programming in Python.
Practice turning in a CS251 project and rubric on Google Classroom.

Instructions

Please read all these instructions carefully!

Due date

Your first goal is to write a draft of the Data class to submit on Google Classroom next week. This is referred to as the Project 1 check-in. Please refer to the post on Google Classroom for the Project 1 check-in due date.

Try to complete through Task 1 below for the check-in. There might be bugs in your Data class and that's ok — you will have the chance to fix them in Week 2 without penalty. We will be grading your progress on Task 1:

1 point if you make substantial progress
0 otherwise

Get started immediately on Task 1 — the first lecture of CS251 should be all that you need to complete most of it. The first lab will cover the rest (how to parse categorical data).

Task 0: Download data and code templates

To start the project, download the following code template (data.py), test code, and data CSV files.

NOTE: This week we will use Python test script files to check your code, but next week and beyond (Week 2+) we will move to Jupyter notebooks.

Check-in submission

Download the Task 1 code templates and test code below.

Code templates	Test code	Notes
data.py	test01_read_numeric_data.py	Tests handling of numeric data only
	test02_read_data_errors.py
	test03_read_cat_data.py	Tests handling of both numeric and categorical data
	test04_read_missing_data.py	Tests handling of missing data
	test05__str__.py	Tests the "toString" method
	test06_get.py	Tests the Data class get methods.

Data

Download data used in the project below. Place the data in a subfolder of your working directory called data.

Dataset	Files
Iris (Numeric variables only)	iris_no_species.csv
Bad Iris	iris_bad.csv
Iris	iris.csv
Anscombe Quartet	anscombe.csv
Spaces test	test_data_spaces.csv
Complex test	test_data_complex.csv
Numeric and categorical data test	test_data_mixed_spaces.csv
Missing data test	test_data_missing.csv
Spotify dataset	spotify.csv

Don't skip reading this section

Important: In this class, it is important that you write code that conforms to the code template methods specifications exactly. This means that the parameter count, data types, return types must match the docstring specifications 100%. We grade by running test code, which includes what we give you in the above Python test scripts (and Jupyter notebooks in Weeks 2+). We give this to you to provide you with rapid feedback to help accelerate your progress. We may also have other test code that we use to grade, which makes similar assumptions. You will lose points if test code fails to run or returns incorrect results.

You should never have to modify test code!

Installing software

If you are working on your own computer, visit the Software page for installation instructions on Windows and macOS. Here is the minimum of what you need for Week 1:

Python 3.14 (install via Python website).
Numpy package (install via pip).

Task 1: Data class

The Data class:

Reads in and stores tables of data contained in .csv files.
Allows the user to select and return variables by their string name, rather than their column index.

Overview of CSV files

Your Data class parses .csv data files. Here is the assumed structure of each .csv file:

1st row: Headers (name of each data variable; corresponds to the columns).
2nd row: Data type. Possible values: numeric, categorical, string, or date. Numeric types can be either integers or floating point values; categorical values are strings that come in a discrete set of possible values called levels; strings are arbitrary strings; a date should be interpreted as a calendar date.
3rd row+: Actual data.

Your Data object should only hold numeric and categorical data variables (ignore other columns of data in the CSV file).

1a. Overview of Data class

Implement the following methods in data.py (also see Task 1b. Testing your code).

Constructor: Declare/initialize instance variables, start parsing .csv file if its path is provided.
read(filepath): Reads data from the specified .csv file into the code Data object.
__str__(): Prepares a nicely formatted string for printing Data objects
get_headers(): returns a list of all of the headers.
get_mappings(): returns the dictionary that maps between variable name (key) and column index (value).
get_cat_level_mappings(): returns the mapping between categorical variable name (key) and a list of the respective unique level strings (value).
get_num_dims(): returns the number of variables (columns).
get_num_samples(): returns the number of samples in the data set.
get_sample(rowInd): returns the rowInd-th data sample.
get_header_indices(headers): returns the variable (column) indices of the str variable names in headers

1b. Testing your code

As you go, run the following test scripts to check the correctness of your code (Download above). Check them in order — verify test01_read_numeric_data.py produces the expected results before moving onto test02_read_data_errors.py and so on.

test01_read_numeric_data.py
test02_read_data_errors.py
test03_read_cat_data.py
test04_read_missing_data.py
test05__str__.py
test06_get.py

Note about Test 2

Running test02_read_data_errors.py should produce a crash. The goal is to understand why this happens and to inform the user what problem occurs and how to correct it. -->

Task 2+: (Week 2)

Starting this week, all project instructions are in a Jupyter notebook. Download the following notebook and code template to get started on the next few tasks.

Remember: final submissions should include updated versions of your draft code.

Notebooks	Code templates
analysis.ipynb	analysis.py

Create a subfolder in your Project 1 working directory named images. Download the following image and copy it into the images subfolder.

Images	Description
pairplot_bottomleft.png	Format of the bottom-left pair plot.

Corrections

Please check here for any corrections to the project instructions or test code.

Submission

Naming

Recall the naming scheme for project and lab submissions:

Place your lab in a folder whose name includes your Colby user name. For example, I would place my project folder inside one called owlayton.
Create a ZIP file out of the folder that has your Colby username in it. For example, mine would be called owlayton.zip.

Following these naming guidelines greatly helps us grade your labs and projects! We may take points off your project if you repeatedly do not follow this convention when submitting.

Checklist

Before you turn in your project, did you...

generate the requested outputs and plots inline in the appropriate notebook cells?
answer all the questions?

Turning in your project

To turn in check-in (draft) and final project submissions, you should create a .ZIP file and upload it to Google classroom. The .ZIP file should:

contain code (.py) and notebooks (.ipynb).
have a file size of under 10MB. If the .ZIP file is over 10MB, exclude data files until you get it under 10MB.

Extensions

You should not turn in any code related to extensions in your base project submission. This is important because project and extensions will be graded by different people. Turn in your video extensions (5 minute video recording + code) via the separate Google Classroom Assignment under the Project Extensions section.