Skip to content

Integer overflow

Integer overflow occurs when an exact result exceeds a fixed-width integer range. NumPy examples show why widening must happen before arithmetic.

Integer overflow occurs when an arithmetic result lies outside the representable range of its integer data type. In a quant system, an overflowing position counter can turn an increase in holdings into a negative stored quantity. The inputs can each be valid while their sum is not.

A position that changes sign

Suppose a fictional position contains 120 contracts and a trade adds 20. The exact new position is 140 contracts. An 8-bit signed integer can store only −128 through 127, so 140 does not fit.

In NumPy, adding two int8 arrays keeps that dtype and wraps the result to −116. By contrast, built-in Python integers return 140. This example was executed with Python 3.14.4 and NumPy 2.5.1.
import numpy as np position = np.array([120], dtype=np.int8) trade = np.array([20], dtype=np.int8) wrapped = position + trade wide = position.astype(np.int16) + trade.astype(np.int16) too_late = wrapped.astype(np.int16) print(wrapped.tolist(), wrapped.dtype) # [-116] int8 print(wide.tolist(), wide.dtype) # [140] int16 print(too_late.tolist(), too_late.dtype) # [-116] int16 print(120 + 20) # 140
Widen before the arithmetic. Casting the wrapped output to int16 preserves the already incorrect −116; it cannot reconstruct 140. The wider inputs produce 140 because that result fits int16. A wider fixed-width type still has a limit.
Fictional contract quantities 120 plus 20 equal 140 exactly. Adding int8 arrays wraps to minus 116 with bits 10001100; widening the inputs to int16 first gives 140, while widening the wrapped result leaves minus 116.
Fictional contract quantities 120 plus 20 equal 140 exactly. Adding int8 arrays wraps to minus 116 with bits 10001100; widening the inputs to int16 first gives 140, while widening the wrapped result leaves minus 116.
Original fictional contract-count example. A signed 8-bit result has range −128 to 127. NumPy int8 array addition wraps 140 to −116; converting the inputs before addition preserves 140 in int16.

Where does −116 come from?

An 8-bit value has 256 possible bit patterns. The low eight bits of 140 are 10001100. Read as unsigned, this means 140. Read as a signed two’s-complement value, the leading bit contributes −128, giving −128+8+4=−116-128+8+4=-116. Equivalently, 140−256=−116140-256=-116.

For a signed two’s-complement integer of width bb, the range is

−2b−1≤x≤2b−1−1.-2^{b-1} \leq x \leq 2^{b-1}-1.

The corresponding unsigned range is 00 through 2b−12^b-1. For example:

NumPy dtypeMinimumMaximum
int8−128127
uint80255
int16−32,76832,767
np.iinfo(dtype).min and .max report these bounds. Signedness changes how the bit patterns are interpreted; unsigned storage does not remove the upper limit.

Treat the representation as part of the calculation

Overflow behaviour belongs to the particular language, library and operation. Do not infer that every integer implementation wraps as this NumPy array does. NumPy’s current documentation also distinguishes scalar overflow warnings from array arithmetic: these array additions produce no warning. Absence of a warning does not establish correctness.

Python’s built-in int uses arbitrary precision, so it has no fixed 8-bit or 64-bit arithmetic boundary. Larger values need more memory and computation; arbitrary precision does not mean infinite machine capacity. An array created with dtype=int still uses a fixed-width NumPy integer dtype.
For position counts, timestamps or monetary amounts stored in minor units, establish bounds for intermediate results as well as stored inputs. Choose sufficient width before adding or multiplying, and test boundary cases against exact Python-integer calculations. Inspect the dtype used by each operation; reductions can choose a different accumulator dtype. Switching to floating-point arithmetic is not a general fix because it can sacrifice exact integer values.
For array and dtype background, the NumPy lesson has a public preview; the full lesson requires membership.

Check the unsigned case

A fictional unsigned 8-bit counter is 250 and increases by 10. With NumPy uint8 arrays for both inputs, what are the result, the result cast afterward to int16, and the result when both inputs are cast to int16 first?
Answer. The exact total is 260. uint8 wraps modulo 256, giving 4. Casting that result afterward still gives 4. Casting both inputs first gives 260 in int16. The positive wrapped value can look plausible, so a negative-result check alone would miss this failure.

Sources

Last updated
  • integer arithmetic
  • numpy
  • numerical correctness
← Back to the glossaryBack to top ↑