In Python and JavaScript, 0.1 + 0.2 returns 0.30000000000000004. Yet 0.5 + 0.25 gives exactly 0.75. Both calculations use the same number format, so why does only one need an approximation?

I found the usual answer, "floating point is inherently imprecise," unsatisfying. Computers execute exact instructions on exact bit patterns. Something specific happens when we turn a decimal number into bits, and calling it imprecision doesn't explain the mechanism.

Numbers such as 0.5, 0.25, 1024, and 6.75 have exact binary representations. Others, including 0.1 and 6.1, have to be rounded before we can store them.

I ran into this question again while looking into automatic mixed precision (AMP) for training neural networks. Training adjusts numbers called weights to reduce the model's error. AMP uses lower precision for some of that computation, and the PyTorch docs and NVIDIA papers show loss curves, plots of training error over time, that mostly look fine. But what changes when you cut a number's bit budget in half? Are we losing precision, or range, or both? What decides what we lose?

I couldn't answer those questions. Knowing that floats were "approximate" didn't help me choose between FP16 and BF16, two formats that both use 16 bits per number. I needed to understand what each part of the format was for.

Rounding also changes arithmetic. An identity such as $(a + b) + c = a + (b + c)$ can fail because the two groupings round different intermediate results.

The design problem

An integer such as 13 is 1101 in binary, and we can recover it exactly. With 32 bits, we have $2^{32}$ = 4,294,967,296 distinct patterns to assign to numbers.

There are infinitely many real numbers between 1.0 and 2.0, and still infinitely many between 1.00 and 1.01. Unlike integers, real numbers have no next value: between any two, we can always find another.

We have about 4.3 billion bit patterns to cover that continuous line. Imagine assigning each pattern to a tick mark; choosing the format means choosing where those marks go.

A value between two marks has to be approximated. With round-to-nearest, we store the nearer mark. This is rounding, and its error depends on how far apart the marks are.

The same thing happens in decimal. $1/3 = 0.333...$ repeats forever. Keep six digits and we get $0.333333$; add three copies and the result is $0.999999$, not $1$. The calculation is exact for the numbers we kept, but we rounded each input first.

Some fractions terminate in decimal: $1/2 = 0.5$, $1/4 = 0.25$, and $1/5 = 0.2$. Others repeat, including $1/3 = 0.333...$ and $1/7 = 0.142857...$. For a fraction in lowest terms, the distinction comes from the prime factors of its denominator. Since $10 = 2 \times 5$, a decimal expansion terminates if the denominator contains only factors of 2 and 5. Another prime factor, such as 3 or 7, makes it repeat.

Binary has only one prime factor in its base: 2. A fraction in lowest terms therefore terminates in binary only when its denominator is a power of two, as with halves, quarters, and eighths.

That explains 1/10. Its decimal form is 0.1, but its binary form, $0.0\overline{0011}_2$, repeats forever because 10 contains a factor of 5.

Both 0.1 and 0.2 are therefore stored as nearby binary approximations. Their errors exist before the addition runs; the addition can introduce another rounding error.

The spacing also has to suit the quantities we compute with:

• A bridge architect working at millimeter precision across 0.5 to 500 meters.
• Machined parts like hard disk heads, where a micrometer (0.001 mm) matters, but the whole part fits inside a few hundred millimeters.
• Orbital dynamics, where one equation mixes $G \approx 6.67 \times 10^{-11}$ with Earth's mass $M \approx 5.97 \times 10^{24}$.

With a fixed bit budget, evenly spaced marks force a choice between a wide range and small gaps. Floating point instead places marks close together at small magnitudes and farther apart at large magnitudes.

Same program, different answers

That still leaves several choices: how to divide the bits between scale and significant digits, how to encode negative numbers, and what to do when a result falls outside the range or exactly between two marks.

In the 1960s and 70s, manufacturers made those choices differently. DEC's VAX used one format, IBM's System/360 used base 16, and Cray's supercomputers used another format.

The same Fortran program could give different numerical answers on these machines. They disagreed about rounding, intermediate precision, and overflow, so moving scientific software from a VAX to a Cray could change its results.

In the late 1970s, Intel was designing the 8087 floating-point coprocessor, and mathematician William Kahan helped develop a standard for floating-point arithmetic. The aim was to make basic operations agree bit for bit when implementations used the same format and rounding mode.

After nearly eight years of debate, the committee published IEEE 754 in 1985. It specified formats and arithmetic rules, including what happens near zero and how rounding breaks ties. By the 90s, mainstream CPUs had adopted it, and languages commonly expose its formats as float/double or float32/float64.

---

Part 1: Representation

Bits and Unsigned Integers

A bit is either 0 or 1. Eight bits make a byte:

$0\;1\;1\;0\;1\;0\;1\;1$

To read this as a number, each position gets a power of two. The rightmost bit is $2^0 = 1$, the next is $2^1 = 2$, then $2^2 = 4$, and so on up to $2^7 = 128$ on the left.

Add up the columns where there's a 1:

$0 \times 128 + 1 \times 64 + 1 \times 32 + 0 \times 16 + 1 \times 8 + 0 \times 4 + 1 \times 2 + 1 \times 1 = 107$

Eight bits cover 0 (00000000) through 255 (11111111), giving $2^8 = 256$ possible values.

Binary addition proceeds from right to left, carrying whenever a column totals 2, just as decimal addition carries at 10. For 42 + 23:

$\begin{array}{r} 00101010 \quad (42) \\ + \; 00010111 \quad (23) \\ \hline 01000001 \quad (65) \end{array}$

With 32 bits, the largest value is $2^{32} - 1 = 4{,}294{,}967{,}295$. These are unsigned integers: zero and positive whole numbers, with no bit assigned to a minus sign.

---

The Limits of Integers

Adjacent integers are exactly 1 apart. That spacing works for counts, but it gives us no control over the precision we spend at different magnitudes.

The Sun's mass is roughly $2 \times 10^{30}$ kg. We don't measure it to the kilogram, so a representation that counts every kilogram up to that value spends bits on distinctions we don't need. Scientific notation separates the significant digits ($2$ here) from the exponent ($30$ in $10^{30}$). We want a similar separation between precision and range in our bit pattern.

Our running example has a more immediate problem: an integer can't store 6.75. The nearest choices are 6 and 7.

---

Fixed Point

One way to represent fractions is to split the 32 bits in half: 16 bits for the integer part and 16 for the fraction.

The integer side works as before: powers of two going up ($2^0, 2^1, 2^2, \ldots, 2^{15}$). The fractional side mirrors it, going down: $2^{-1} = 0.5$, $2^{-2} = 0.25$, $2^{-3} = 0.125$, and so on. Where decimal has tenths, hundredths, thousandths, binary has halves, quarters, eighths.

The positions on either side follow the same rule: $1, 2, 4, 8, \ldots$ to the left, and $0.5, 0.25, 0.125, \ldots$ to the right. To encode a fraction, choose the positions whose values add up to it.

6.75: the integer part is $6 = 110_2$ ($4 + 2$). The fractional part is $0.75 = 0.5 + 0.25 = 2^{-1} + 2^{-2}$, which is 0.11 in binary. Together:

$0000000000000110.1100000000000000$

This is also a scaled integer. The scale factor is $2^{16} = 65{,}536$, so 6.75 is stored as $6.75 \times 65{,}536 = 442{,}368$. Divide by 65,536 to read it back.

The maximum value has dropped from over 4 billion to roughly 65,535. Every value gets the same 16 integer bits and 16 fractional bits, even if a different split would be more useful.

For 3.14159, only two integer bits are needed; the unused integer bits can't provide more fractional precision. 50,000 fits, but 100,000 doesn't, even if we don't need any fractional digits for it.

---

The Floating Point Idea

We could store the position of the binary point alongside the digits, allowing a different scale for each value.

Set aside 8 bits for the point position and use the remaining 24 for digits. For example, store the digits 11 and a position that places the point after the first digit. Together, the fields describe the binary value $1.1$.

A value such as 0.000125 needs the point far to the left, while 6,000,000 needs it far to the right. Both can use the same number of significant digits.

For 6.75, the digits are 11011 and the point goes after the third digit: 110.11. One field specifies the scale; the other holds the significant digits, called the significand or mantissa. I'll use "mantissa" to match the diagrams. Allowing a different point position for each value gives us floating point.

Within a fixed bit budget, more bits for the point position leave fewer for the mantissa. We gain range but lose precision. Assigning more bits to the mantissa reverses that tradeoff. This 8+24 split is close to the float32 layout.

This is scientific notation in base 2, with both the digits and the scale stored in the bit pattern.

---

The Sign Bit

To include negative numbers, assign one of the 32 bits to the sign: 0 for positive, 1 for negative.

With one bit assigned to the sign, the layout is:

$\underbrace{1}_{\text{sign}} \;\; \underbrace{8}_{\text{exponent}} \;\; \underbrace{23}_{\text{mantissa}}$

The scale field, called the exponent, keeps 8 bits, leaving 23 for the mantissa: 1 + 8 + 23 = 32.

The patterns for 6.75 and -6.75 differ only in the sign bit. Using 0 for a positive sign also lets positive zero have an all-zero pattern (00000000...).

---

The Redundancy Problem and Normalization

Storing the point position separately allows several encodings of the same number. For 6.75 (110.11_2), these all have the same value:

• 1.1011 × 2^2 (point after first digit)
• 11.011 × 2^1 (point after second digit)
• 110.11 × 2^0 (point after third digit)
• 0.11011 × 2^3 (point before first digit)

If we allow all four forms, we spend four patterns on one value. Requiring a single form would leave more patterns available for distinct values.

Zero also has redundant forms: moving its point never changes its value.

Scientific notation avoids this redundancy for nonzero values by requiring exactly one nonzero digit before the point:

• $6{,}750 = 6.75 \times 10^3$
• $0.00675 = 6.75 \times 10^{-3}$

With the decimal point after the first nonzero digit, we exclude forms such as $67.5 \times 10^2$ and $0.675 \times 10^4$, even though both describe the same value. The exponent compensates for moving the point.

We use the same rule in binary: place the binary point after the first non-zero digit and adjust the exponent to compensate. This is normalization.

For 6.75:

$110.11_2 = 1.1011 \times 2^2$

Moving the point two positions left divides the mantissa by 4, so we multiply by $2^2$ to preserve the value. The mantissa is 1.1011 and the exponent is 2. This rule gives each representable nonzero magnitude one normalized form.

---

The Implicit Leading 1

In decimal, the first digit of a normalized number can be anything from 1 through 9, so we have to store it.

In binary, the first nonzero digit can only be 1. Normalization therefore makes it predictable:

$1.\text{something} \times 2^n$

We can leave that 1 out of the stored mantissa and restore it when reading the value. The 23-bit field then provides 24 bits of precision.

For 6.75, the normalized form is $1.1011 \times 2^2$. We store 10110000000000000000000 in the mantissa field (23 bits, the part after the binary point). When computing, the hardware reconstructs 1.10110000000000000000000 (24 bits).

The omitted digit is the implicit leading 1, also called the hidden bit.

---

Negative Exponents and the Bias

The 8-bit exponent field holds integers from 0 to 255, but values below 1 need negative exponents: $0.5 = 1.0 \times 2^{-1}$ and $0.125 = 1.0 \times 2^{-3}$.

We could give the exponent its own sign bit and use the other 7 bits for its magnitude. That covers -127 through +127, but assigns two patterns to exponent zero: 00000000 and 10000000.

IEEE 754 instead adds a fixed offset before storing the exponent:

$e_\text{stored} = e_\text{real} + 127$

Subtract 127 to recover the exponent:

• Stored 1 → real exponent -126 (smallest normal)
• Stored 127 → real exponent 0
• Stored 254 → real exponent +127 (largest normal)

The offset is the bias, and the stored value is the biased exponent. Adding a constant preserves ordering, so comparing stored exponents also compares their real exponents.

Stored exponents 0 and 255 are reserved for special values. For normal numbers, the stored range is 1 through 254; subtracting the bias of 127 gives real exponents from -126 through +127.

---

Exponent as a Window, Mantissa as an Offset

For a normal float32 value, the fields combine as follows:

$(-1)^s \times \left(1 + \frac{m}{2^{23}}\right) \times 2^{(e-127)}$

When I first saw this formula, it didn't help me picture the spacing. I found it easier to think of the exponent as selecting a range and the mantissa as selecting a position inside it.

For positive normal numbers, whose stored exponent is 1 through 254, that range is a window between consecutive powers of two. Negative values mirror it across zero.

Strip the bias and call the real exponent $n$. Then:

• The window spans $[2^n, 2^{n+1})$.
• The window is $2^n$ wide.
• The mantissa divides that window into $2^{23} = 8{,}388{,}608$ evenly spaced slots.

Here are a few windows to make this pattern concrete:

$[0.5, 1), \;\; [1, 2), \;\; [2, 4), \;\; [4, 8), \;\; [8, 16), \;\; \ldots \;\; [2^{127}, 2^{128})$

Each window has $2^{23}$ slots. As the exponent increases by one, the width doubles, so the distance between adjacent floats doubles too.

If the mantissa bits decode to an integer $m$ in $[0, 2^{23}-1]$, the value is:

$2^n + m \times \frac{2^n}{2^{23}}$

This writes the value as the left edge of the window plus an offset. The sign bit determines whether the value is positive or negative.

6.75 lies in $[4,8)$. Its position as a fraction of the window width is:

$\frac{6.75 - 4}{8 - 4} = 0.6875$

At 68.75% of the window width, 6.75 lands exactly on a slot, so it needs no rounding.

Now try 6.1, also in $[4,8)$:

$\frac{6.1 - 4}{8 - 4} = 0.525$

Multiply that fraction by $2^{23}$ to get the slot index: $0.525 \times 2^{23} = 4{,}404{,}019.2$. The index isn't an integer, so 6.1 falls between slots and must be rounded.

The same rule applies when moving from $[1,2)$ to $[2,4)$: $2^{23}$ slots cover twice the distance.

Some concrete slot sizes to give you a feel for the scale:

• In $[1,2)$: slot size is $2^{-23} \approx 1.19 \times 10^{-7}$.
• In $[2048,4096)$: slot size is $2^{-12} \approx 2.44 \times 10^{-4}$.

The absolute spacing grows with magnitude, but the spacing relative to the value stays roughly constant. Normal floats therefore provide about the same number of significant bits at each scale.

These normal values use the middle range of the exponent field:

Normal numbers cover stored exponents 1 through 254. Stored 0 and 255 are still empty.

---

The Zero Problem

A normal number starts with 1.something, which prevents it from representing zero. No exponent makes $1.\text{anything} \times 2^n$ equal zero.

When the stored exponent and mantissa are both zero, IEEE 754 assigns the value zero. The sign bit gives us +0 and -0.

For comparison, 6.75 is a normal value: $1.1011 \times 2^2$, with biased exponent $2 + 127 = 129 = 10000001_2$, mantissa field 10110000000000000000000, and sign bit 0.

$\underbrace{0}_{\text{sign}} \;\; \underbrace{10000001}_{\text{exponent} = 129} \;\; \underbrace{10110000000000000000000}_{\text{mantissa}}$

Zero occupies the pattern with stored exponent 0 and mantissa 0:

One gap remains: between 0 and the smallest normal number.

---

Subnormal Numbers

The smallest positive normal uses the lowest normal exponent and an all-zero mantissa field.

Sign = 0, stored exponent = 1 (the lowest allowed for normals; 0 is reserved for zero and subnormals), mantissa = all zeros:

$0 \;\; 00000001 \;\; 00000000000000000000000$

Decode the exponent: $1 - 127 = -126$. The implicit leading 1 gives a significand of exactly 1.0:

$1.0 \times 2^{-126}$

Since $126 \times \log_{10} 2 \approx 126 \times 0.301 \approx 37.9$, this is roughly $10^{-38}$; more precisely, $\approx 1.175 \times 10^{-38}$.

Without smaller representable values, there would be a gap between this number and zero. A result in that gap couldn't retain its value.

Take $1.5 \times 2^{-126}$ and $1.25 \times 2^{-126}$, both representable normal values. Their difference, $0.25 \times 2^{-126}$, falls into the gap and rounds to zero. We would get a - b == 0 even though a != b.

Stored exponent 0 with a nonzero mantissa is still unused. Its $2^{23} - 1$ mantissa patterns can provide values in the gap.

For these patterns, IEEE drops the implicit leading 1. Instead of $1.\text{mantissa} \times 2^{-126}$, the value is $0.\text{mantissa} \times 2^{-126}$. Keeping the scale fixed while allowing a leading zero extends the format below the smallest normal.

These are subnormal numbers, also called denormals.

The smallest positive subnormal has only its last mantissa bit set: 00000000000000000000001. That gives a significand of $2^{-23}$. With no leading 1 to restore, the significand remains $2^{-23}$ and the exponent stays at $-126$:

$2^{-23} \times 2^{-126} = 2^{-149}$

In decimal: $149 \times 0.301 \approx 44.8$, so roughly $10^{-45}$. More precisely, $\approx 1.4 \times 10^{-45}$.

Subnormals have a constant spacing of $2^{-149}$, forming a uniform grid between zero and the smallest normal. As values shrink, that fixed step becomes a larger fraction of the value, so relative precision decreases.

Flushing results below $1.0 \times 2^{-126}$ to zero is called abrupt underflow. Subnormals instead let results decrease in fixed steps toward zero, losing relative precision along the way. This is gradual underflow.

For distinct finite values in the same format, gradual underflow preserves the guarantee that a != b implies a - b != 0.

Exponent 0 is fully assigned now: mantissa 0 means zero; mantissa non-zero means subnormal.

---

Floats as Buckets

Round-to-nearest assigns an interval of real inputs to each representable float. Think of that interval as a bucket around the stored value.

Take 6.75. The next representable float is one grid step away, about $4.8 \times 10^{-7}$. Every real number within half that step rounds to the same bit pattern. A neighborhood of real values all become "6.75."

The bucket boundaries lie halfway between adjacent values, with the rounding rule deciding which bucket owns an exact tie.

Interactive placeholder: zoomable number line around a selected value showing adjacent floats, half-ULP bucket boundaries, and which reals round to the same stored value.

Negative zero. A negative result that is too small to represent can round to -0, preserving its sign. Signed zero also occurs in exact operations; it doesn't always mean a tiny value was rounded away.

Infinity. Overflow can produce infinity when a result exceeds the finite range. Unlike an ordinary rounding bucket, infinity has special arithmetic rules.

$x / 0 = \infty$ (for finite nonzero $x$) is one of those rules, with the signs determining which infinity. The bucket picture can suggest a limiting value, but it doesn't make division by exact zero ordinary real arithmetic.

---

Infinity

The largest positive finite float uses stored exponent 254 and an all-ones mantissa field:

$0 \;\; 11111110 \;\; 11111111111111111111111$

Decode the exponent: $254 - 127 = +127$. Reconstruct the significand: $1.111...1_2$ with all 23 fractional bits on. Each additional bit gets you halfway closer to 2. With all 23 set, you're $2^{-23}$ short:

$1.\underbrace{111...1}_{23} = 2 - 2^{-23}$

Equivalently: the mantissa encodes $m = 2^{23} - 1$, so the significand is $1 + (2^{23} - 1)/2^{23} = 2 - 2^{-23}$. Multiply by the window:

$(2 - 2^{-23}) \times 2^{127} = 2^{128} - 2^{104}$

Since $2^{104}$ is negligible compared to $2^{128}$: $128 \times 0.301 \approx 38.5$, so roughly $3.4 \times 10^{38}$. More precisely, $\approx 3.4028235 \times 10^{38}$.

That is a 39-digit number. For comparison, estimates put the number of atoms in the observable universe around $10^{80}$, well beyond float32's range.

Overflow needs a separate result. Stored exponent 255 with an all-zero mantissa means infinity, and the sign bit distinguishes $+\infty$ from $-\infty$:

$+\infty: \quad 0 \;\; 11111111 \;\; 00000000000000000000000$ $-\infty: \quad 1 \;\; 11111111 \;\; 00000000000000000000000$

Infinity has defined arithmetic rules, such as $\infty + 1 = \infty$ and $-\infty < x$ for every finite $x$. It lets a calculation continue after overflow, though the finite magnitude has been lost.

Stored exponent 255 has $2^{23}$ possible mantissa patterns. Infinity uses only the all-zero pattern.

---

NaN

$0/0$ has no defined real result. Neither do $\sqrt{-1}$ in real arithmetic or $\infty - \infty$. These need a result distinct from both ordinary numbers and infinity.

IEEE 754 assigns these invalid results NaN (Not a Number): stored exponent 255 with a nonzero mantissa. With traps disabled, the computation can return NaN and continue.

There are $2^{23} - 1$ nonzero mantissa patterns and two sign-bit choices, giving $2 \times (2^{23} - 1) = 16{,}777{,}214$ NaN patterns. Some mantissa bits can hold a payload that identifies the source of an invalid result. Whether that information is recorded and preserved depends on the implementation.

NaN propagates through basic arithmetic, so an invalid result can travel through subsequent operations until the program checks it:

The product $0 \times \infty$ has no unique limiting value. If zero and infinity came from rounded values, their bit patterns no longer tell us the relative magnitudes needed to determine the product.

By contrast, IEEE defines $1 / 0$ as infinity with the appropriate sign when traps are disabled. This division-by-zero result is separate from an invalid operation such as zero divided by zero.

NaN == NaN is false. That makes x != x a test for NaN under IEEE comparison rules, though a language's isnan function states the intent more clearly.

What "not equal to anything" means in practice:

• If x is NaN, x == y is false for every y (including NaN).
• If x is NaN, x != y is true for every y.
• Every ordered comparison (<, <=, >, >=) with NaN returns false.

Practical consequence: x < y is not the same as !(x >= y) when NaN is involved. When writing min and max, choose explicitly whether NaNs should propagate or be ignored.

A zero-finding algorithm can use this behavior when it probes a function outside its domain, for example at $\sqrt{-1}$. The probe returns NaN, and the algorithm can detect it and choose another point. The algorithm still needs that check; propagation alone doesn't recover from the invalid probe.

Stored exponent 255 now covers infinity and NaN:

The counts add up to all 4,294,967,296 bit patterns.

---

Encoding 6.75

The complete encoding of 6.75 combines the choices made above:

Step 1: Convert to binary.

$6 = 110_2$ and $0.75 = 0.11_2$ ($0.5 + 0.25$), so:

$6.75 = 110.11_2$

Step 2: Normalize.

Move the binary point to after the first 1:

$110.11 = 1.1011 \times 2^2$

The normalized mantissa is 1.1011 and the exponent is 2.

Step 3: Sign bit.

6.75 is positive. Sign bit = 0.

Step 4: Bias the exponent.

Real exponent is 2. Add the bias (127):

$2 + 127 = 129 = 10000001_2$

The exponent field is 10000001, selecting the $[4, 8)$ window.

Step 5: Store the mantissa.

Drop the implicit leading 1. The stored mantissa is the 23 bits after the binary point:

$10110000000000000000000$

The hardware puts the 1 back when it reads this.

Step 6: Assemble.

$\underbrace{0}_{\text{sign}} \;\; \underbrace{10000001}_{\text{exponent}} \;\; \underbrace{10110000000000000000000}_{\text{mantissa}}$

Full 32-bit string: 0 10000001 10110000000000000000000.

Hexadecimal: 0x40D80000.

To check the encoding, decode it: sign 0 is positive, exponent $129 - 127 = 2$, and significand $1.10110...0 = 1.6875$. The value is $1.6875 \times 4 = 6.75$.

---

Part 2: Arithmetic

Compute, Then Round

For basic IEEE arithmetic in the default rounding mode, the rule is:

Compute the exact result, then round to the nearest representable float.

A correctly rounded operation returns the same result as exact arithmetic followed by rounding. Hardware only needs enough information to decide that rounding; it doesn't need to store an infinitely precise intermediate.

In a chain of operations, each rounded result becomes the next input. The next operation computes from the bits it receives, with no record of the rounding error in earlier steps. Its result may need rounding too.

Changing the grouping changes those intermediate results. $(a + b) + c$ doesn't always equal $a + (b + c)$, so floating-point addition isn't associative.

On a number line, each addition can land between marks and round to one of them. A different order can reach a different intermediate mark, and therefore a different final result.

Addition also needs the binary points to line up. To add 42 and 0.001, the hardware shifts the smaller operand's significand until the exponents match, adds the significands, then normalizes and rounds the result.

With a large exponent gap, the smaller operand's significant bits lie far below the larger operand's stored precision. They may be too small to affect the rounded sum, which then equals the larger operand.

For example, 1e20f + 1.0f == 1e20f in float32. The spacing near $10^{20}$ is on the order of $10^{13}$, so adding 1 doesn't reach the midpoint to the next float.

Multiplication and division use different steps because neither needs exponent alignment.

IEEE 754 requires correct rounding for the five basic operations (+, -, *, /, √). In round-to-nearest mode, a finite result is at most 0.5 ULP from the exact answer, where a ULP is the spacing at that scale. The tie-breaking rule chooses between equally close values.

IEEE also specifies correct rounding for remainder and integer-float conversions. Decimal conversion (printing floats as decimal strings and parsing them back) is harder; efficient exact conversion is a research problem in its own right, and behavior varies across libraries.

For transcendental functions (sin, cos, exp, log, ...), the standard is less strict. Math libraries usually aim for errors within a few ULPs, and results can differ across platforms and library versions.

How Far Apart Are Adjacent Floats? (ULP)

A ULP measures the spacing between adjacent values at a given exponent.

The window model gives $2^{23}$ evenly spaced slots per exponent band. In the float32 formula:

$(-1)^s \times \left(1 + \frac{m}{2^{23}}\right) \times 2^{(e-127)}$

where $m$ is the 23-bit mantissa interpreted as an integer.

Increment $m$ by 1 (keeping $e$ fixed) and you step to the next representable float. The size of that step:

$2^{(e-127)} \times 2^{-23} = 2^{(e-150)}$

That step is one ULP (unit in the last place) at this exponent.

With $2^{23}$ slots per window, the spacing at a few real exponents is:

• Around 1.0 (exponent 0): spacing is $2^{-23} \approx 1.19 \times 10^{-7}$.
• Around 6.75 (exponent 2): spacing is $2^{-21} \approx 4.8 \times 10^{-7}$, putting the next float at roughly 6.7500005.
• Around 1024 (exponent 10): spacing is $2^{-13} \approx 0.000122$.

Subnormals are the exception: constant spacing of $2^{-149}$, because there is no implicit leading 1 and the exponent is fixed.

The abrupt change at a power-of-two boundary caught me off guard. Just below 4.0, the spacing is $2^{-22}$; just above, it is $2^{-21}$. An error measured in ULPs therefore needs a reference scale, especially near a boundary.

In C/C++, nextafterf(x, +∞) returns the next representable float above x. Subtracting x gives the upward spacing; at a power-of-two boundary, the downward spacing can differ.

The spacing at 1.0 comes up so often in error analysis that it has its own name: machine epsilon. The distance from 1.0 to the next representable float. For float32: $\varepsilon = 2^{-23} \approx 1.19 \times 10^{-7}$. In C: FLT_EPSILON. In C++: std::numeric_limits<float>::epsilon().

FLT_EPSILON is fixed, while spacing depends on magnitude. Near 1.0, the spacing is about $10^{-7}$; near 1024, about $10^{-4}$; near $10^8$, it is 8. Using FLT_EPSILON alone as a comparison tolerance ignores those differences.

Rescaling can keep a computation away from overflow and the subnormal range. It doesn't change the number of mantissa bits: normal values near $10^9$ or $10^{-30}$ still have the same significant-bit budget as values near 1.

In decimal terms, float32 gives about 7 significant digits of precision. Float64 gives about 16.

Printing float32 to 20 decimal places exposes the decimal expansion of the stored value. It doesn't give the original quantity 20 digits of accuracy; the extra digits describe the binary approximation.

The practical rule is about round-trips: 9 decimal digits are enough to recover any float32 exactly when parsing back to binary. For float64: 17 digits. These are about unique recovery, not about how many digits of the original quantity are meaningful.

Rounding Modes

Round-to-nearest chooses the closer of two representable values. If the exact result is halfway between them, it needs a tie-breaking rule.

An exact binary tie has a 1 immediately after the retained bits and zeros in every lower position. Ties-to-even chooses the neighbor whose last retained bit is 0.

Always rounding ties upward introduces a directional bias. Ties-to-even can round either way, depending on the retained bits. It avoids always favoring one direction, though it doesn't guarantee that rounding errors cancel in a particular calculation.

IEEE 754 defines five rounding modes total. Ties-to-even is the default, and it is what you get unless you explicitly change it. The other four: ties-away-from-zero (less commonly exposed in hardware), round toward zero (truncation), round toward $+\infty$ (ceiling), and round toward $-\infty$ (floor). The directed modes exist mainly for interval arithmetic, where you round one endpoint up and the other down to trap the true value in a provable bracket.

Switching modes changes more than the last bit. It can affect overflow behavior and whether an exact zero comes out as +0 or -0.

I didn't expect rounding modes to affect conversions too. In C, rint rounds to an integer-valued float and lrint returns an integer. Both follow the active rounding mode: round-toward-$-\infty$ behaves like floor, and round-toward-$+\infty$ like ceil. Plain casts usually truncate toward zero. Overflow also depends on the mode: the default produces $\pm\infty$, while a directed mode can return a largest finite value instead.

Measuring Error

We can describe an error in units of the local spacing or as a fraction of the true value.

ULP error counts grid steps: how many adjacent-float hops separate your result from the true value. "Off by $N$ ULPs" means about $\log_2 N$ trailing bits are unreliable. It is intuitive because it measures distance in the float's own units.

Relative error normalizes by magnitude:

$\frac{|x_{\text{approx}} - x_{\text{true}}|}{|x_{\text{true}}|}$

The nearest float32 to $\pi$ is 3.1415927410125732. Its gap from real $\pi$ is about $8.742278 \times 10^{-8}$, or 0.37 ULP, with relative error about $2.8 \times 10^{-8}$. For a function whose value at $\pi$ is zero, rounding $\pi$ can change the result noticeably.

Interactive placeholder: interactive ULP explorer around a selected float32 value (especially near powers of two) showing nextafter, spacing jumps, and relative vs absolute error.

Cancellation

When two nearly equal numbers are subtracted, their matching leading digits cancel. The result depends on the accuracy of the trailing digits.

If both operands are exact, cancellation exposes accurate lower digits. This is benign cancellation.

If the operands contain rounding error from earlier operations, cancellation can leave that error large relative to the result. This is catastrophic cancellation. The subtraction exposes an error already present in its inputs.

The distinction is the accuracy of the operands. Subtracting $x - y$ when both are exact machine values can be harmless even when they are close. In $b^2 - 4ac$, rounding in the two products can corrupt the digits that survive subtraction.

(I once stared at a quadratic discriminant that came back negative for what was obviously a real root. $b^2$ and $4ac$ were both huge, nearly equal, and each slightly rounded from the multiplications. The subtraction wiped out every meaningful bit. The leftover noise happened to be negative. The formula told me the root didn't exist.)

Diagram placeholder: side-by-side comparison of benign vs. catastrophic cancellation. Two float mantissa bars being subtracted in each case. Left (benign): all bits in both operands are accurate; after subtraction, surviving trailing bits are clean. Right (catastrophic): leading bits are accurate but trailing bits carry rounding noise; after subtraction, only the noisy bits survive, now occupying the most significant positions. Same number of canceled bits, opposite outcome.

Cancellation also interacts with the loss of small terms during addition. These examples show how a small term can disappear before a later subtraction would have made it significant.

---

Why 0.1 + 0.2 ≠ 0.3

The float64 calculation of 0.1 + 0.2 has three rounding steps: one for each input and one for their sum. Those steps explain why its result differs from the stored value of 0.3.

Float64, or double precision, is the format used for this expression in Python and JavaScript.

The repeating expansion from the fraction-termination discussion is $1/10$ in binary, analogous to $1/3 = 0.333...$ in decimal:

$\frac{1}{10} = 0.0\overline{0011}_2 = 0.000110011001100110011..._2$

A float64 mantissa stores 52 explicit bits (53 total with the hidden bit). The rest are rounded away. So the float64 labeled "0.1" is actually:

$\text{float64}(0.1) = 0.1000000000000000055511151231257827021181583404541015625$

This is slightly above 0.1. The stored value of 0.2 is:

$\text{float64}(0.2) = 0.200000000000000011102230246251565404236316680908203125$

Both inputs have rounded upward. Their exact sum is:

$0.300000000000000016653345369377348...$

Now look at the two float64 values that bracket the real number 0.3:

• Lower: $0.29999999999999998889776975...$
• Upper: $0.30000000000000004440892098...$

The exact sum is halfway between these neighbors, and ties-to-even selects the upper one. Thus 0.1 + 0.2 rounds to $0.30000000000000004440...$, while the literal 0.3 rounds to $0.29999999999999998889...$. The stored results differ by one ULP.

More mantissa bits reduce the error, but $1/10$ still repeats in binary. Even float128 must approximate it.

The intervals that round to a single float widen with magnitude, as in the window model. That spacing determines both the input approximation and the final rounding of an operation.

---

Comparing Floats

== asks whether two stored values are equal. When the values came from different rounded calculations, we may instead need to test whether their difference is small enough for the application.

Attempt 1: absolute epsilon. $|a - b| < \varepsilon$.

If the tolerance is FLT_EPSILON, it matches the spacing above 1.0. At 2.0, adjacent floats are already farther apart; at 1024, the gap is roughly $1000 \times$ FLT_EPSILON. A fixed absolute tolerance can be appropriate, but choosing it from spacing at 1.0 only makes sense when that scale fits the problem.

Attempt 2: relative epsilon. $|a - b| < \varepsilon \cdot \max(|a|, |b|)$.

This scales the tolerance with magnitude. Near zero, however, a tiny absolute difference can still be large relative to the values. Compare $1 \times 10^{-40}$ and $2 \times 10^{-40}$: the difference is small in absolute terms, but it is half the larger value. A relative test alone may reject a difference the application considers negligible.

Attempt 3: hybrid. Use an absolute term for small magnitudes and a relative term for larger ones:

$|a-b| \le \varepsilon_{\text{abs}} + \varepsilon_{\text{rel}} \cdot \max(|a|, |b|)$

The absolute term provides a tolerance near zero, while the relative term grows with magnitude. NumPy's allclose and PyTest's approx both combine absolute and relative tolerances, though their exact comparison rules differ.

Choose absolute tolerance from the smallest difference that matters in the application: perhaps $10^{-30}$ in one physics problem or $10^{-4}$ for pixel coordinates. Relative tolerance also depends on the algorithm and its inputs. Error may grow roughly as $\sqrt{n}$ when rounding errors are sufficiently independent and unbiased, or linearly when they reinforce each other. A tolerance of about $10 \times$ FLT_EPSILON for 100 operations is only a rough estimate; cancellation or sensitive inputs can invalidate it.

For a comparison measured in representable steps, use ULP distance. Two finite floats are $N$ ULPs apart when $N$ nextafter steps connect them. The bit-pattern ordering described earlier gives this distance directly for positive floats.

If a test passes only after a large increase in tolerance, investigate the discrepancy. Accumulation, cancellation, or sensitivity to the input may explain it; increasing the tolerance alone doesn't establish that the result is acceptable.

Handle special values explicitly. As above, NaN == NaN is false, infinities compare by their values, and $+0 = -0$ despite their different bit patterns.

---

Accumulation Error: Time

A 100 Hz simulation runs for 10 minutes, updating its clock with t += 0.01 for 60,000 frames. In float32, that clock ends at about 600.27 seconds.

Float32 stores 0.01 as 0.009999999776482582. Repeated addition rounds that increment and the running total differently from computing elapsed time directly:

1. Accumulate: start at t = 0, do t += Δt each frame. Rounds 60,000 times.
2. Compute from a counter: keep an integer frame index i, compute t = i * Δt. Rounds once.

After 60,000 frames, repeated addition gives t = 600.2744140625, while t = i * Δt gives 600.0. The counter avoids feeding the rounded total from one frame into the next.

If your timestep is constant, compute from an integer counter. If it is variable, accumulate in float64, or store time as integer ticks and convert at the edges.

A smaller timestep can make this harder to notice. At about 810 kHz, each step is roughly 1.2 microseconds, but that says little about the accuracy of a clock updated with t += Δt. Hundreds of thousands of rounded additions per second accumulate error, and the drift can change direction at exponent boundaries.

The timestep is the size of one update; clock accuracy concerns the accumulated result. A small $\Delta t$ added repeatedly can give a less accurate clock than a larger $\Delta t$ multiplied by an integer counter.

---

Stable Algorithms: Triangle Area

Rounding can cause a large error even in a formula evaluated only once. A thin triangle exposes this in Heron's formula:

$s = \frac{a+b+c}{2}$ $\text{area} = \sqrt{s(s-a)(s-b)(s-c)}$

The formula is mathematically correct, but it is numerically fragile when two sides are nearly equal and the third is much smaller.

Take a triangle with sides $a = 100{,}000{,}000$, $b = 100{,}000{,}000$, $c = 1$. The true area is about 50,000,000.

In float32, naive Heron's formula returns zero.

The semiperimeter s and side a round to the same float. Their difference s - a is then exactly zero, making the product and computed area zero too.

Kahan gave a stable rearrangement. Sort the sides so $a \ge b \ge c$, then compute:

$\text{area} = \frac{1}{4}\sqrt{(a+(b+c))(c-(a-b))(c+(a-b))(a+(b-c))}$

Evaluating this rearrangement in float32 gives 50,000,000 for the same triangle. The changed grouping preserves the small terms that the original formula lost.

Float64 has enough precision for both formulas to round to 50,000,000 in this example. That doesn't make the original formula stable: sufficiently extreme side lengths can expose the same problem in float64.

Grouping terms of comparable magnitude can preserve contributions that would otherwise round away. The parentheses determine the intermediate values, so they are part of the numerical algorithm.

---

When Algebra Breaks

An algebraic rearrangement changes the intermediate values in a computation. That can change its rounding or introduce overflow even when the real-number result is unchanged.

As the addition example showed, a + (b + c) can differ from (a + b) + c. Distributivity can fail for the same reason: (a + b) * c and a*c + b*c round different intermediates.

For an overflow example, take $x = 10^{30}$ and $a = 10^{10}$ in float32:

$x \cdot (a/a) \quad\text{vs.}\quad (x \cdot a) / a$

The first expression gives $x \cdot 1 = 10^{30}$. The second forms $10^{40}$ as an intermediate and overflows to infinity before the division can reduce it.

I would have assumed that dividing by 7 and multiplying by 1/7 were interchangeable. In float32 with $x = 12345.678$, however, $x / 7$ gives 1763.668212890625 while $x \cdot (1/7)$ gives 1763.6683349609375. Rounding the reciprocal first changes the result.

A compiler may substitute reciprocal multiplication when its floating-point settings allow it. The flags therefore affect both execution speed and numerical behavior.

---

Compilers and Hardware Knobs

Changing compiler flags can change a floating-point result even when the source code and inputs stay the same.

The language, compiler, hardware, and math library all affect which operations run and how they round.

C++ (as of C++23) doesn't require IEEE 754 conformance. That surprised me when I learned it. Most implementations follow IEEE, but portable code has to check the implementation and its settings rather than rely on the language name alone.

Strict floating-point modes restrict transformations that change observable numerical behavior. Options such as -ffast-math allow more assumptions and rearrangements. I've seen that flag passed around Makefiles as a free speedup, even though it can change which numerical tests pass.

Several settings matter here:

Fused multiply-add (FMA). a*b + c can be evaluated with a single final rounding instead of a rounded product followed by a rounded sum. FMA is usually more accurate, but it may differ from the two-operation result. Whether the compiler contracts the expression into FMA affects reproducibility.

Flush-to-zero. The FTZ and DAZ modes described earlier change behavior near zero. A build that enables them can disagree with one that preserves subnormals.

Extended precision intermediates. Some implementations evaluate an expression in a wider format than its declared type and round it when storing. Comparing a stored result with an unstored intermediate can then fail, as in the 3/7 example below.

Algebraic rewrites. Reciprocal multiplication, reassociated sums, and changes such as x*y - x*z to x*(y - z) alter where rounding occurs. They need numerical justification for the inputs in use.

The chosen type also affects performance. Some GPUs execute float32 much faster than float64. Vector instruction sets such as SSE, AVX, and NEON add platform-specific behavior that reproducible code needs to account for.

Choosing Your Number Type

Choose a number type from the required range, acceptable error, and cost:

• Integers. Use these for exact discrete quantities, with an explicit unit such as cents or clock ticks. Convert to other formats where the application needs them.
• Float64. About 16 significant decimal digits make it a useful default for many scientific and engineering calculations.
• Float32. About 7 significant decimal digits, with lower storage costs and often higher throughput. Common uses include GPU shaders, machine learning inference (running a trained model), and bulk sensor data.
• Software multiprecision (Boost, mpfr). I'd reach for this when float64 is insufficient and the extra computation is affordable.
• Fixed-point. Uniform spacing over a chosen range. A 60.40 format has 60 integer bits and 40 fractional bits; if the unit is seconds, its resolution is well below a nanosecond over a long duration.

Wider types use more memory and reduce the number of values that fit in cache. Their arithmetic cost also depends on the hardware, especially on GPUs.

---

Testing and Debugging

A floating-point error can produce a plausible value, so it may survive a casual check. The first time I chased a NaN backward through a computation, I wished I had set up my debugging differently.

Testing strategy. Compare approximate results using tolerances justified by the problem or a ULP bound. Use == when exact equality is the requirement. Work through at least one case by hand and keep it as a test.

Implicit promotion. In C/C++, an operation combining a double and a float promotes the float operand to double. Storing the result back in a float can round it again. Check the types of individual operations, especially inside a larger expression.

Floating-point exceptions. During debugging, trapping on overflow, division by zero, or invalid operations can locate the first bad result. Underflow traps can help when tiny values are unexpected. C/C++ provide fenv.h for parts of the floating-point environment; trap controls are platform-specific.

Traps vs. NaN propagation. Traps stop at the operation that raises the exception. With traps disabled, a program can propagate NaN and check for it later. I use traps for debugging and NaN propagation for production.

Backward error analysis asks how much the inputs would need to change for the computed result to be exact. An algorithm is backward stable when those changes stay small relative to the input. The forward error also depends on how sensitive the problem is to such changes.

Record tolerances, expected input ranges, and compiler settings alongside the tests. Those assumptions help explain a failure when the code or environment changes.

---

Float32 Layout and Other Formats

Float32 stores its fields in these positions:

The reserved exponent values determine how to interpret the mantissa:

These fields determine the available values. Rounding to them during arithmetic explains results such as $(a + b) + c \ne a + (b + c)$.

Precision Cheat Sheet

Other binary formats divide their bits differently between exponent and mantissa. These five formats are common in numerical code and machine learning:

Interactive placeholder: side-by-side format explorer for FP16/BF16/TF32/FP32/FP64 showing field splits, epsilon, min normal/subnormal, and representable range.

Float64 expands the exponent from 8 to 11 bits and the stored mantissa from 23 to 52 bits. Most of the additional bits provide precision rather than exponent range.

bfloat16 keeps float32's 8 exponent bits and bias of 127, but stores only 7 mantissa bits. It provides a similar normal range with much coarser spacing, useful in neural-network calculations that tolerate the rounding error.

float16 uses 5 exponent bits and 10 mantissa bits. Its maximum is much smaller ($\sim 6.5 \times 10^4$), but its spacing within a normal exponent window is finer than bfloat16's.

TF32 is a computation format used by NVIDIA Tensor Cores on Ampere and later GPUs. Inputs remain float32 in memory; the hardware reduces their mantissa precision to 10 stored bits for multiplication and accumulates into float32. This gives float32's exponent range with roughly float16's significand precision. PyTorch exposes a control through torch.backends.cuda.matmul.allow_tf32.

The bit allocations have several practical consequences:

• Three formats share the same exponent field. bfloat16, TF32, and float32 all use 8 exponent bits with bias 127, so their normal-range magnitudes are similar (roughly $10^{-38}$ to $10^{38}$). They differ in mantissa width (7, 10, 23 bits), which changes epsilon and the subnormal floor.
• Conversion to bfloat16. Rounding away the low 16 mantissa bits of float32 leaves the 7 stored mantissa bits of bfloat16. Converting back pads those positions with zeros; it cannot recover the discarded information.
• Float16 overflow. The largest finite value is 65,504. Summing 256 values of 255 has an exact result of 65,280; 257 such values have an exact sum beyond the format's range. A naive float16 sum also rounds along the way. Wider accumulation helps preserve both the range and the accuracy of sums.
• Not all hardware implements subnormals for narrow formats. Many GPUs flush bfloat16 subnormals to zero. The subnormal values in the table are theoretical minimums.

This answers the format question I had when looking at AMP. FP16 spends more of its 16 bits on the mantissa, while BF16 spends more on the exponent. FP16 gives finer spacing within the normal range; BF16 covers more magnitudes. Choosing between them requires knowing which limitation affects the computation.

IEEE also defines extended formats. Some FPUs use 80-bit extended internally, not as a user type, but to give implementations extra headroom for functions like exp, log, and decimal conversion before rounding back to float32/float64.

Enter a decimal value below to compare its stored value, neighboring floats, and bit fields across formats:

Interactive placeholder: decimal-to-float snap visualizer with live bit fields, neighboring representable values, and ULP distance to the typed decimal.

---

Reducing the stored mantissa from 23 bits to 10 (float16), 7 (bfloat16), or 3 (FP8 E4M3) leaves fewer representable values in each exponent window. The exponent allocation separately determines which windows are available. For a training computation, both limits matter: rounding must preserve useful updates, and intermediates must stay within range.

References and Further Reading

If you read one thing beyond this post, make it Goldberg's survey. It's dense but foundational. The error model and guard digit examples we used come from there.

The references below include visual explanations, tools for inspecting bit patterns, and the technical sources used here.

Visual guides and explainers

• Fabien Sanglard, Floating Point Visually Explained. Uses number-line diagrams to show how float spacing changes with magnitude and where precision gets lost. This is the one that first made float spacing click for me.

• The Floating-Point Guide: What Every Programmer Should Know. Concise answers to the "why does 0.1 + 0.2 != 0.3?" question, with language-specific guidance.

• Svar.dev, Demystifying Floating Point. Walkthrough from binary fractions to subnormals and special values.

• John D. Cook, Anatomy of a Floating Point Number.

• Bruce Dawson, Comparing Floating Point Numbers, 2012 Edition. Thorough treatment of absolute epsilon, relative epsilon, and ULP-based comparisons. Covers exactly the tradeoffs discussed in our "Comparing Floats" section.

• Bruce Dawson, That's Not Normal: the Performance of Odd Floats. Measured performance costs of subnormals on x86 hardware.

Videos and talks

• Computerphile, Floating Point Numbers.

• John Farrier, Demystifying Floating Point (CppCon 2015).

Interactive tools

• Float Exposed. Type a float, see the exact bits, sign/exponent/mantissa breakdown, and the stored value. I used this constantly while writing this post.

Technical references and papers

• David Goldberg (1991). What Every Computer Scientist Should Know About Floating-Point Arithmetic. ACM Computing Surveys, 23(1), 5-48. Long survey that introduces the standard error model ($\mathrm{fl}(a \circ b) = (a \circ b)(1 + \delta)$), guard digits, and classic examples (including the cancellation example we used earlier).

• Hans-J. Boehm (2017). Small-data computing: correct calculator arithmetic. Communications of the ACM, 60(8), 44-49. Case study of Android’s “Exact Calculator” and demand-driven (“constructive real”) arithmetic for fully accurate displayed results.

• T.J. Dekker (1971). A Floating-Point Technique for Extending the Available Precision. Numerische Mathematik, 18(3), 224-242. The original double-double splitting technique referenced in our "Double-double arithmetic" section.

• IEEE Computer Society (2019). IEEE Standard for Floating-Point Arithmetic (IEEE 754-2019). The standard itself. Defines all the formats, rounding modes, and special values we derived. Surprisingly readable for a standards document.