Responsible ML Lab · Lecture 14

Check every group, not just the average

First the per-department check from slide 5. Then an invented experiment: what happens to a small group when the training data has few of its members?

Slide 5: accuracy per department (students.csv, invented)

Experiment: how much of group B is in the training data?

group B share:

group A accuracy

group B accuracy

gap

Before trusting any model (slide 7)

What to notice

An overall accuracy can hide a group for whom the model works much worse.
In the experiment both groups are equally easy to predict; group B just follows a slightly different pattern. With 2% of the training data it gets about 66% right, group A about 84%.
More group B data closes the gap. Collecting better data is often the real fix, not a cleverer model.

Try this

At what share does the gap fall below 5 percentage points? Homework 14.2: repeat the slide 5 check with a decision tree. Do the groups differ more or less?

Privacy: is "anonymous" data really anonymous?

150 invented students (names and numbers are made up). Choose which columns to share in a data set. The lab counts how many people are the only one with their combination of values, so anyone who knows them a little can find their row.

Columns to share

Make some columns less exact

birth year → "2004 or earlier" / "2005 or later" home district → "Barishal division" / "other"

The shared table (first 6 rows)

people who are unique (identifiable)

smallest group of look-alikes

What to notice

Removing names and phone numbers is necessary but not enough: department, year, gender, district and birth year together single out most people.
Making values less exact (an age band, a division) puts people in larger groups of look-alikes.
Collect only what the question needs. A column you never collected cannot leak.

Try this

Remove name, phone and student ID. How many people are still unique? Find a set of columns where nobody is unique but the table is still useful for studying marks.

One neuron, by hand

Slide 9: inputs times weights, plus a bias, then a simple rule. Move any slider and follow the numbers.

Inputs

Weights and bias

Function

What to notice

A weight says how much an input matters; its sign says in which direction.
The bias moves the threshold: a lower bias needs more evidence before the output becomes 1.
"Learning" means a computer adjusts these weights, a little at a time, to make fewer mistakes.

Try this

Homework 14.1: find a student (hours, attendance, GPA) for whom the output is 0. Check the total here. Change the weights so that more of the 150 invented students are classified like their real result.

A small network learning

A network with one hidden layer, trained on 200 invented points. Move through the training rounds (epochs) and change the number of hidden neurons.

Data Hidden neurons

epoch

training accuracy

loss (error)

Slide 11: the digits, same split

What to notice

At first the border is almost a straight line. Each epoch nudges every weight and the border bends.
The loss falls fast at first, may rest on a plateau while the border searches for the curve, then flattens (slide 12).
One hidden neuron can only draw a straight-ish border. More neurons allow curved borders, and also more memorising (Lecture 11).

Try this

On the ring, find the smallest number of hidden neurons that separates the two classes. Homework 14.3: train MLPClassifier on the digits with 8, 32 and 128 neurons in Colab.

Works offline: no internet needed. The data and the trained models are built into this page. Every number was calculated with scikit-learn and checked, so it matches what you will see in Jupyter or Colab.