Foundations of numerical computing · Lecture 2

Floating-Point Numbers

A computer keeps about 7 or 16 significant digits, in binary. That one fact explains why 0.1 + 0.2 ≠ 0.3, and sets the accuracy limit of every method in this course.

Md Mahbub E Noor · Assistant Professor
CSE-3107 Numerical Methods · B.Sc. 3rd Year 1st Semester
Department of CSE, University of Barishal

Today's goals

Convert decimal fractions to binary and see why 0.1 never ends
Write numbers in normalised floating-point form and count significant digits
Read the three fields of an IEEE 754 single and double number
Define and compute machine epsilon; explain rounding vs chopping
Recognise overflow, underflow and safe ways to compare floating-point numbers

Last time

Numerical methods compute approximate answers by iteration; we stop when the change is small enough.

Today

How small can "small enough" be? The answer is set by how numbers are stored: about 16 significant decimal digits in double precision.

Foundations of numerical computing · you are here

L1 · Why Numerical Methods?→L2 · Floating-Point Numbers→L3 · Taylor Series and Order of Accuracy→L4 · Errors and Their Propagation→L5 · Conditioning and Stability

Builds on: L1

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

2 / 16

Start here

The idea in one picture

Between 1 and 2 this toy computer has only nine numbers.A real number usually falls between two of them.It is replaced by the nearest storable number.The small difference is the round-off error.Real computers do the same in base 2, with far more digits.

In plain words

A computer has a fixed number of digits for every number
So it can store only some numbers; all others are rounded to a neighbour
That tiny rounding is where numerical error begins

New words today

floating point: digits × a power of the base, like 1.25 × 2³
mantissa / fraction: the digits of the number
exponent: where the point is placed
round-off error: true number − stored number

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

3 / 16

Decimal fractions in binary: multiply by 2

Problem 1Convert

0.625_{10}

and

0.1_{10}

to binary. Rule: multiply the fraction by 2; the integer part is the next bit.

1

0.625 \times 2 = 1.25 \Rightarrow

bit 1, keep 0.25

2

0.25 \times 2 = 0.5 \Rightarrow

bit 0;

\;0.5 \times 2 = 1.0 \Rightarrow

bit 1, remainder 0: stop. So

0.625_{10} = 0.101_2

(exact)

3

0.1 \times 2 = 0.2 \to 0,\; 0.4 \to 0,\; 0.8 \to 0,\; 1.6 \to 1,\; 1.2 \to 1,\; 0.4 \to 0, \dots

4

The remainder 0.4 has appeared before, so the pattern repeats forever:

0.1_{10} = 0.0\overline{0011}_2

A fraction is exact in binary only if its denominator is a power of 2 (½, ¼, ⅝ …).

0.1 = \tfrac{1}{10}

is not, so the computer must cut it off: a round-off error before any arithmetic happens.

step	fraction × 2	bit
1	0.2	0
2	0.4	0
3	0.8	0
4	1.6	1
5	1.2	1
6	0.4	0
7	0.8	0
8	1.6	1
…	…	…

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

4 / 16

Floating-point form and significant digits

Normalised floating-point form (decimal): $x = \pm\, 0.d_1 d_2 \dots d_t \times 10^{e}$ with $d_1 \ne 0$
The digits $d_1 \dots d_t$ are the mantissa; $e$ is the exponent; $t$ is the precision
Example: $-347.62 = -0.34762 \times 10^{3}$ , $\;0.000123 = 0.123 \times 10^{-3}$
Significant digits: all digits from the first non-zero digit onward. $0.000123$ has 3; $2.500$ has 4; $3070$ has 3 or 4 (ambiguous: write $0.3070\times10^4$ )
A computer stores only $t$ digits, so most real numbers are replaced by a nearby representable one

Chopping

Cut off extra digits

$\tfrac{2}{3} = 0.666666\dots \to 0.6666$ (4 digits). Error up to one unit in the last place.

Rounding

Round to nearest

$\tfrac{2}{3} \to 0.6667$ . Error at most half a unit in the last place. IEEE uses round-to-nearest, ties to even.

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

5 / 16

IEEE 754: the format inside every CPU

The same idea as scientific notation, but in base 2: $x = (-1)^{s} \times 1.f \times 2^{\,E - \text{bias}}$

single (C `float`), 32 bits

1

8 exponent

23 fraction

bias 127 · about 7 decimal digits · range ≈ $10^{\pm 38}$

double (C `double`, Python `float`), 64 bits

1

11 exponent

52 fraction

bias 1023 · about 16 decimal digits · range ≈ $10^{\pm 308}$

1

Example:

-6.25 = -110.01_2 = -1.1001_2 \times 2^{2}

2

sign

s = 1

; exponent field

E = 2 + 127 = 129 = 10000001_2

; fraction

f = 1001000\dots0

(the leading 1 is not stored)

3

Bits: 1 10000001 10010000000000000000000 = 0xC0C80000

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

6 / 16

Explore: what is really stored?

Type 0.1: the stored value is not 0.1
Switch to double: closer, still not exact
Try 0.5 or −6.25: exact, because they are sums of powers of 2
Try 16777217 in single: the integer itself cannot be stored ( $2^{24}+1$ )

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

7 / 16

Machine epsilon: the precision of the machine

Definition: the gap between 1 and the next larger float

\varepsilon_{\text{mach}} = (\text{next float after } 1) - 1

Values

\begin{aligned} \varepsilon_{\text{single}} &= 2^{-23} \approx 1.19\times10^{-7} \\ \varepsilon_{\text{double}} &= 2^{-52} \approx 2.22\times10^{-16} \end{aligned}

Rounding error bound

\text{fl}(x) = x(1 + \delta), \qquad |\delta| \le \tfrac{1}{2}\,\varepsilon_{\text{mach}}

Relative spacing of floats is about $\varepsilon$ everywhere: the gap near $10^{6}$ is about $2.2\times10^{-10}$
So a double holds about $\log_{10}(1/\varepsilon) \approx 16$ significant decimal digits
No tolerance smaller than about $\varepsilon\,|x|$ can ever be met
$1 + \varepsilon/2$ rounds back to exactly 1

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

8 / 16

Find machine epsilon by experiment

Halve $\varepsilon$ until $1 + \varepsilon/2$ is no longer different from 1
The C version also does it in float: about $10^{-7}$
Python has sys.float_info.epsilon; C has DBL_EPSILON in <float.h>

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

9 / 16

0.1 + 0.2, and how to compare floats

Neither 0.1, 0.2 nor 0.3 is exact; the rounding errors do not cancel
Never test floats with ==; test abs(a - b) < tol
Use a relative tolerance for large numbers: math.isclose(a, b)
Adding 0.1 ten times also misses 1.0: errors accumulate

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

10 / 16

float vs double: accumulation in C

Add 0.1 one million times. The true answer is 100000
float drifts by about 1000: wrong in the 3rd significant digit
double is wrong only after about 10 digits
Lesson: use double for numerical work; keep sums in double even if data are float

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

11 / 16

Floats are not evenly spaced

A toy system: base 2, 3 fraction bits, exponents −2 to 2
Each colour is one exponent: inside it, numbers are equally spaced
The gap doubles at each power of 2
But the relative gap stays about the same: that is machine epsilon

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

12 / 16

At the edges: overflow, underflow, special values

Overflow

Too big

$10^{308} \times 10 = \infty$ (inf). In Python, 2.0**1024 raises OverflowError; math.exp(1000) too.

Underflow

Too small

$10^{-320}$ is stored as a subnormal with fewer digits; below $5\times10^{-324}$ it becomes 0.

NaN

Not a Number

$0/0$ , $\infty - \infty$ , $\sqrt{-1}$ in C give NaN. NaN ≠ NaN: even x == x is false.

🧠

When training "blows up"

A loss that becomes NaN or inf is usually overflow in exp (softmax) or a division by a tiny number. Libraries compute softmax as $e^{x_i - \max x}$ to avoid overflow: a numerical-methods trick.

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

13 / 16

Checkpoint

Checkpoint · think, then discuss with your neighbourIn double precision, which of these is stored exactly?

A0.1

B0.375

C1/3

D0.2 + 0.1

B:

0.375 = \tfrac{3}{8} = 0.011_2

. The others have denominators that are not powers of 2, so their binary expansions never end.

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

14 / 16

Practice

Practice problem

Problem 2"Discuss the importance of numerical methods in computing science. Define the floating-point form of numbers, significant digit of a number and round-off error."

How to write a complete answer

Floating-point form: $\pm 0.d_1d_2\dots d_t \times \beta^{e}$ , $d_1 \ne 0$ ; name mantissa, base, exponent; give one example
Significant digits: digits from the first non-zero digit; example $0.004560$ has 4 significant digits
Round-off error: true value − stored value, caused by keeping only $t$ digits; example $2/3 \to 0.6667$ , error $3.3\times10^{-5}$
Mention chopping vs rounding and the bound $\tfrac{1}{2}\times10^{1-t}$ (relative) for rounding
One sentence of importance (Lecture 1) to open the answer

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

15 / 16

Summary

Key points

Only fractions with power-of-2 denominators are exact in binary; 0.1 is not
Floating point = sign × mantissa × base^exponent, normalised; IEEE double has 53 significant bits
$\varepsilon_{\text{mach}} = 2^{-52} \approx 2.2\times10^{-16}$ : about 16 significant digits; rounding error ≤ ε/2 relative
Never compare floats with ==; use a tolerance, preferably relative
Use double; beware accumulation, overflow (inf), underflow and NaN

Homework

Convert 13.375 and 0.3 to binary. Which one is exact?
Write 0.000452 and 45200 in normalised floating-point form and state the number of significant digits of each.
In the lab: find the stored double for 0.7 with the bit explorer. Is it above or below 0.7?
Run the float-vs-double C program with 10 million additions. Describe what happens to the float sum.

Next lecture

Taylor series and order of accuracy. Round-off is one source of error; the other is truncation. The Taylor series predicts it, and every method in this course is built on it.

Md Mahbub E Noor

Lecture 2: Floating-Point Numbers

University of Barishal

16 / 16

Floating-Point Numbers

Today's goals

The idea in one picture

Decimal fractions in binary: multiply by 2

Floating-point form and significant digits

Cut off extra digits

Round to nearest

IEEE 754: the format inside every CPU

single (C float), 32 bits

double (C double, Python float), 64 bits

Explore: what is really stored?

Machine epsilon: the precision of the machine

Find machine epsilon by experiment

0.1 + 0.2, and how to compare floats

float vs double: accumulation in C

Floats are not evenly spaced

At the edges: overflow, underflow, special values

Too big

Too small

Not a Number

When training "blows up"

Checkpoint

Practice problem

Summary

single (C `float`), 32 bits

double (C `double`, Python `float`), 64 bits