Charts Lab · Lecture 7

Mean, median and spread: feel what each one does

Edit the numbers or drag the largest one far away. Watch which summary moves and which stays put.

values

largest value =

Drag to push the largest value up to 10 times further away.

mean

median

std (pandas)

By hand, as on the slide

What to notice

One extreme value drags the mean; the median hardly moves.
Standard deviation is the typical distance from the mean: square the gaps, average them, take the square root.
pandas std() divides by count − 1, so it is a little larger than the by-hand (divide by count) value.

Try this

Homework 7.1: enter 3, 5, 7, 9, 11. Check the by-hand standard deviation. Make a list where mean and median are exactly equal. Make two lists with the same mean but very different standard deviations.

Histogram: the shape of a column, and the bins you choose

A histogram cuts the range into bins and counts the values in each. The same data can tell a different story with different bins.

Data

bins = 40

same bin edges for both groups drop duplicates

Histogram

Point at a bar to see its range and count.

What to notice

Too few bins hide the shape; too many make it look noisy. Try 5, 15 and 80.
Two separate plt.hist calls choose their own bin edges, so bars of ham and spam do not line up.
matplotlib silently skips missing values: check isna().sum() so you know how many were left out.

Try this

Find the number of bins where the two peaks of the SMS data are easiest to see. Draw attendance with 15 bins (slide 15, "Your turn"). Describe its shape in one sentence.

Scatter and correlation: how tightly two columns move together

Build your own cloud of points, or look at real columns from students.csv. Then test your eye in the guessing game.

Points from

slope =

noise =

points =

add one outlier

correlation r

points used

Guess the correlation

your guess r = 0.00

rounds

average miss

What to notice

r measures how close the points are to a straight line, from −1 to +1. It is not the slope.
One outlier can change r a lot when there are few points.
In students.csv, hours and prior_gpa are almost unrelated (r near 0), yet hours clearly differ between passed and failed students (see tab 4). Correlation only looks for straight-line patterns between numbers.
Correlation is not causation: a hidden third factor can drive both columns.

Try this

Set noise to 0. What is r for slope 2? For slope 0.1? Why is it the same? With 10 points and little noise, how much does one outlier lower r? Play 5 rounds of the game and get your average miss below 0.15.

Box plot: the middle half, the median and the unusual values

The box holds the middle 50% of values, the line is the median, the whiskers reach the furthest values within 1.5 box-lengths, and dots are outliers.

Compare

whis = 1.5

show every value as a dot behind the boxes

What to notice

Half the values are inside the box; a quarter are below it and a quarter above.
"Outlier" here only means "beyond the whiskers": change whis and the number of dots changes.
Box plots compare groups quickly; tick "show every value" to see what the box hides.

Try this

Homework 7.3: box plot of hours by passed. Describe the difference in one sentence. Why does ham have so many outliers but spam has fewer?

Works offline: no internet needed. The whole dataset is built into this page. Every number is calculated the same way as pandas and matplotlib and was checked against the real libraries, so it matches what you will see in Jupyter or Colab.