Unsupervised learning

The road ahead: machine learning

Data without labels

200 invented points. No colours, no answers. Do you see groups?

Codecopy

import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs
X, _ = make_blobs(n_samples=200, centers=3,
                  cluster_std=1.2, random_state=42)
plt.scatter(X[:, 0], X[:, 1], s=14, color="gray")
plt.show()

k-means in four steps

By hand: k-means on six numbers

Points 1, 2, 3, 10, 11, 12. k = 2. Start the centres at 1 and 2 (a poor start, on purpose).

# Round 1: all but 1 are
#   nearer to centre 2.
#   New centres: 1 and 7.6
# Round 2: 2 and 3 switch.
#   New centres: 2 and 11
# Round 3: no change: stop.

Watch it: k-means finding three groups

k-means in scikit-learn

import numpy as np
from sklearn.cluster import KMeans
km = KMeans(n_clusters=3, n_init=10, random_state=0)
labels = km.fit_predict(X)
print(km.cluster_centers_.round(2))
print(np.bincount(labels))

[[-6.72 -6.85]
 [-2.69  8.95]
 [ 4.63  2.13]]
[66 67 67]

See the clusters

Colours are the clusters k-means found; crosses are the centres.

Codecopy

plt.scatter(X[:, 0], X[:, 1], c=labels,
            s=14, cmap="viridis")
c = km.cluster_centers_
plt.scatter(c[:, 0], c[:, 1], marker="X",
            s=250, color="red")
plt.show()

Choosing k: the elbow method

Inertia = total squared distance of points to their centre. It always falls as k grows; look for the bend.

Codecopy

ks = range(1, 9)
inertia = []
for k in ks:
    km_k = KMeans(n_clusters=k, n_init=10, random_state=0)
    inertia.append(km_k.fit(X).inertia_)
plt.plot(list(ks), inertia, marker="o")
plt.xlabel("k")
plt.ylabel("inertia")
plt.show()

Do clusters match real classes? Iris without labels

import pandas as pd
from sklearn.datasets import load_iris
iris = load_iris()
cl = KMeans(n_clusters=3, n_init=10,
            random_state=0).fit_predict(iris.data)
species = [iris.target_names[t] for t in iris.target]
print(pd.crosstab(pd.Series(species, name="species"),
                  pd.Series(cl, name="cluster")))

cluster      0   1   2
species               
setosa       0  50   0
versicolor  48   0   2
virginica   14   0  36

Limits of k-means

Common clustering mistakes

Read the last line of an error first. It names the problem.

KMeans(n_clusters=3).fit(pd.read_csv("students.csv").dropna())

ValueError: could not convert string to float: 'Math'

KMeans(n_clusters=10).fit([[1], [2], [3]])

ValueError: n_samples=3 should be >= n_clusters=10.

km.predict([[1.0]])

ValueError: X has 1 features, but KMeans is expecting 2 features as input.

Today's toolkit

Homework 13

Due before Lecture P3 · [date]