Skip to content

Bitwise operations

Use masks to test, set, clear and toggle individual flags in an integer. See why field width and signedness matter when decoding binary data.

Bitwise operations work directly on the binary digits of an integer. They let a market-data decoder test one flag in a packed field even when other flags are set. The data type determines how the integer is represented. The message schema determines what each bit means. FIX Simple Binary Encoding, for example, can encode several independent choices as bits of one unsigned integer. FIX SBE 1.0, §4.14

Four operations on one flag field

Take a fictional quote monitor with an unsigned four-bit field. Bit 0 means stale, bit 1 crossed, bit 2 sequence gap and bit 3 manual review. Bit 0 is the least significant bit, at the right of the written word. A mask is an integer whose bit pattern selects positions. To select bit kk, use 1 << k for 0≤k<40 \le k < 4.
The starting word is 0101, so the stale and sequence-gap flags are set. Follow the updates in order:
  1. Test for a sequence gap with AND (&). An output bit is 1 only when both input bits are 1. AND 0101 with 0100 and the result is 0100 (decimal 4). Its nonzero value means the flag is set. The stored word stays 0101.
  2. Set the crossed flag with OR (|). An output bit is 1 when at least one input bit is 1. OR 0101 with 0010 to get 0111 (decimal 7).
  3. Clear the sequence-gap flag with AND. Use 1011, which has a zero at bit 2 and ones everywhere else in the four-bit field. AND 0111 with 1011 to get 0011 (decimal 3).
  4. Toggle the stale flag with XOR (^). An output bit is 1 when the two input bits differ. XOR 0011 with 0001 to get 0010 (decimal 2). Only the crossed flag remains set.
The four columns run from bit 3 to bit 0. AND combines input 0111 and mask 1011 into output 0011. Only bit 2 changes, from 1 to 0.
The four columns run from bit 3 to bit 0. AND combines input 0111 and mask 1011 into output 0011. Only bit 2 changes, from 1 to 0.
The zero in the mask clears bit 2. The three ones pass their input bits through.
Here are the same steps in Python:
width = 4 all_bits = (1 << width) - 1 flags = 0b0101 sequence_gap = 1 << 2 crossed = 1 << 1 stale = 1 << 0 tested = flags & sequence_gap print(f"{tested:04b}", tested != 0) # 0100 True flags |= crossed print(f"{flags:04b}") # 0111 clear_mask = (~sequence_gap) & all_bits flags &= clear_mask print(f"{clear_mask:04b}", f"{flags:04b}") # 1011 0011 flags ^= stale print(f"{flags:04b}") # 0010
When one mask selects several flags, (flags & mask) != 0 tests whether any selected flag is set. (flags & mask) == mask tests whether all are set. Set and clear give the same result when repeated on the same flag. A second toggle flips the flag back, which matters if the same update can be replayed.

Width, signedness and byte order

Python's integer inversion ~x equals -(x + 1). In the example, ~sequence_gap is -5. AND with the all-ones mask 1111 keeps only the four bits that belong to the field. Python also allows nonnegative shift counts without imposing a field width. For instance, 0b1001 << 1 gives 10010 in binary. Masking back to four bits gives 0010 if truncation is what the schema calls for.

For an unsigned field, shifting right brings in zeros on the left. In Python, shifting a negative integer right follows floor division by a power of two, so the sign matters. For a field spanning multiple bytes, decode using the schema’s byte order and signedness before applying masks. Byte order tells you how to assemble the integer from bytes. The flag definitions tell you which positions to test.

For how these conditions affect the use of a quote, see the quality-flags section of Trades, Quotes and Market State. The preview is public and the full lesson requires membership.

Sources

Last updated
  • binary representation
  • market data
  • programming
  • python
Keep connecting the ideas
← Back to the glossaryBack to top ↑