Feature Prep Lab · Lecture 12

Missing values: drop or fill?

students.csv has 150 invented students; some hours and GPAs are missing. Choose what to do with them and watch what happens to the data and to the model.

What to do

The first 8 rows

NaN missing filled value we guessed

Hours of study, all students

rows used

5-fold CV accuracy (scaled k-NN pipeline)

What to notice

Filling with 0 says "this student studied 0 hours". That is a strong, usually wrong, claim, and the model gets worse.
Median and mean give similar results here; the median is safer when a column has extreme values.
Dropping keeps only real values but throws 14 students away, and the score is then measured on different rows.

Try this

Which choice puts a tall orange bar in the histogram? Why is that a warning sign? Write one sentence for your report that says which choice you made and why.

Categories to numbers: one-hot or 1, 2, 3?

A model can only do arithmetic with numbers. Compare two ways to turn dept into numbers, and look at the distances a k-NN model would see.

Encoding

What the model receives

Distance between departments

5-fold CV accuracy (scaled k-NN pipeline)

What to notice

Numbering invents an order: with CSE = 1, EEE = 2, Math = 3, Math is "twice as far" from CSE as EEE is. Nothing in the data says so.
One-hot puts every pair of departments at the same distance, √2 ≈ 1.414.
Change the order of the numbers and the model changes. A good encoding should not depend on alphabetical luck.

Try this

Find an order where numbering scores lower than one-hot. Homework 12.5: explain in your own words why one-hot is better than 1, 2, 3.

Scaling changes the neighbours

Click any student (a dot). The three nearest neighbours are joined to it. Switch between raw values and StandardScaler values and see which students count as "near".

raw values scaled (StandardScaler)

passedfailedthe ring shows the distance to the 3rd neighbour

The chosen student and its 3 nearest neighbours

Does it matter for accuracy? k-NN with and without the scaler

StandardScaler by hand

Numbers

What to notice

Raw: attendance runs from 41 to 100, hours only from 0 to 12. A 5-point attendance gap outweighs a 4-hour study gap, so neighbours are chosen almost only by attendance (the flat ring).
Scaled: each column has mean 0 and standard deviation 1, so both count fairly (a round ring).
On these students, k-NN with the scaler wins for every k (slide 13: 0.687 versus 0.893 for k = 5).

Try this

Find a student whose three neighbours change completely after scaling. Homework 12.1: type 2, 4, 6 in the calculator after you have done it by hand.

Data leakage: a score too good to be true

60 students, thousands of columns of random numbers, and labels from a coin toss. Nothing can be learned. Watch what the wrong order of steps reports.

Random columns

Wrong: choose the 20 "best" columns using all rows, then cross-validate

Right: the selection is a step inside the pipeline

The same mistake with the scaler on students.csv

Spot the leak

correct

What to notice

With all rows, some random columns happen to match the coin tosses. Choosing them first means the test folds helped choose them.
The more columns, the more lucky matches, and the bigger the lie: about 90% on pure noise.
Inside a pipeline, each training fold chooses its own columns; the honest score is about 50%, the coin toss.
On students.csv the leaky scaler gives the same score. You cannot see leakage by looking at the score, so always use the right order.

Try this

Press "New random data" several times. Does the wrong method ever look honest? Homework 12.4: describe a leak that could happen in your own project.

Works offline: no internet needed. The data and the trained models are built into this page. Every number was calculated with scikit-learn and checked, so it matches what you will see in Jupyter or Colab.