Fuzz testing

Fuzz testing, or fuzzing, automatically generates inputs and runs them through software to look for failures. A market-data team can fuzz a feed decoder before its output reaches an order book. A crash can interrupt updates, and an accepted malformed message can pass incorrect data downstream without an obvious failure.

Suppose a fictional decoder takes a buffer that must contain exactly one frame. Its first two bytes hold an unsigned big-endian length. If those bytes have values $b_0$ and $b_1$, they declare $256b_0+b_1$ payload bytes. The decoder accepts only when the payload has exactly that length, and then returns it unchanged. Here 00 03 denotes two hexadecimal header bytes, and A, B, C and D each stand for one ASCII byte.

Start with 00 03 ABC as a seed. Delete the final byte to make 00 03 AB, or append a byte to make 00 03 ABCD. Both edits keep the declared length at three. The first leaves only two payload bytes and the second leaves four.

The framing rule gives the test an oracle, a way to decide what result is correct. Expect ABC from the seed and an ordinary rejection from each malformed input. A decoder that only checks whether enough bytes arrived could accept 00 03 ABCD, return ABC and silently discard D. Checking for crashes alone would miss that error. A stream decoder that returns the unconsumed bytes would have a different contract. This oracle applies to a buffer that must contain exactly one frame.

Mutation creates inputs by editing examples. A fuzzer can try mutations without any feedback. In coverage-guided fuzzing, test coverage tells the engine which parts of the target an input exercised. A mutation that reaches new code can be kept in the corpus, the collection of inputs the engine can mutate again. This is how Go's native fuzzer and LLVM's libFuzzer use coverage to steer the search. The hand-picked edits above illustrate mutation without measuring coverage. Go Fuzzing · LLVM libFuzzer documentation

Coverage guides the search. The oracle decides whether the observed behaviour is wrong. Fuzzing and property-based testing can overlap when the fuzz target checks a rule on every generated case. For this decoder, one rule is that appending a byte to any valid frame must make it invalid when the length header stays fixed.

When a fuzzer finds a failure, save the exact input, minimise it if the smaller input still exposes the same failure, and keep it as a regression test after the fix. Go's fuzzer documents automatic minimisation attempts and replay of saved failures in ordinary test runs. A passing run means the checks found no failure in the cases tried. Go Fuzzing

The SelfMadeQuant lesson Testing and Debugging practises choosing boundary cases from an input contract and reducing a failure to a small reproducible example. The full lesson requires membership.