Skip to content

Divide and conquer and the master theorem

Trace a two-way merge and see how the master theorem turns the cost of each recursive call into the running time of a full merge sort.

Divide and conquer is an algorithm design strategy. Split a problem into smaller instances of the same problem, solve them using recursion until a base case is reached, and combine the answers. The master theorem gives the time complexity for a class of recurrences that arise from this strategy. In a backtest that ingests batches of market events, merge sort illustrates both ideas. Sorting batches separately is useful because the resulting ordered runs can be stitched together with a single pass.

Two sorted runs, one pass

Take two fictional runs of event timestamps, measured in milliseconds from the same origin. The left run contains 2, 5, 9, 13 and the right contains 1, 6, 10, 12. To merge them, compare the first unread value in each run. The smaller one is the smallest remaining value overall because both runs are already sorted. Write it to the output and advance only its run. Once either run is empty, append the remainder of the other run in order.
Two sorted runs, 2, 5, 9, 13 and 1, 6, 10, 12, merge in seven head comparisons. The right run is then empty, so 13 is appended without comparison. The output is 1, 2, 5, 6, 9, 10, 12, 13.
Two sorted runs, 2, 5, 9, 13 and 1, 6, 10, 12, merge in seven head comparisons. The right run is then empty, so 13 is appended without comparison. The output is 1, 2, 5, 6, 9, 10, 12, 13.
The blue column shows which value each comparison writes. The last value needs no head comparison.

Seven comparisons suffice here because the right run runs out just after 12 is written. Across two nonempty runs with nn items in total, there are at most n−1n-1 head comparisons. At least one item is left when the first run empties. Writing every item still takes Θ(n)\Theta(n) time when comparing and writing each record costs constant time. For real event records, the runs must use the same ordering key. If the timestamp alone leaves simultaneous events tied, specify the tie order the backtest needs.

From the last merge to the full sort

The example starts with sorted halves and shows only the combine step. A standard top-down merge sort starts with arbitrary input. It divides the input in half, sorts both halves recursively, then merges them. An empty run or a run with one item is already sorted. With nn a power of two, a constant-time base case, and linear work outside the recursive calls, the running time obeys
T(n)=2T(n/2)+Θ(n).T(n)=2T(n/2)+\Theta(n).
At depth ii there are 2i2^i runs of size n/2in/2^i. The work outside the recursive calls adds up to Θ(n)\Theta(n) at each merge level. There are log⁡2n\log_2 n such levels, so the full sort takes Θ(nlog⁡n)\Theta(n\log n) time. For other input sizes, the two calls use the floor and ceiling of n/2n/2. The splits remain balanced and the same bound holds. MIT 6.006, Lecture 3

What the master theorem compares

In the clean equal-size case, write T(n)=aT(n/b)+f(n)T(n)=aT(n/b)+f(n), where a≥1a\ge1 is a fixed number of recursive calls, b>1b>1 is a fixed shrink factor and f(n)≥0f(n)\ge0 counts all work outside those calls, including dividing and combining. For n=bkn=b^k with integer bb and kk, and a base case T(1)=Θ(1)T(1)=\Theta(1), there are npn^p leaves, where p=log⁡bap=\log_b a. The basic three cases compare f(n)f(n) with that leaf count. MIT 6.006, Recitation 3
Work per callTotal time
f(n)=O(np−ε)f(n)=O(n^{p-\varepsilon}) for some ε>0\varepsilon>0Θ(np)\Theta(n^p)
f(n)=Θ(np)f(n)=\Theta(n^p)Θ(nplog⁡n)\Theta(n^p\log n)
f(n)=Ω(np+ε)f(n)=\Omega(n^{p+\varepsilon}) for some ε>0\varepsilon>0, with af(n/b)≤cf(n)a f(n/b)\le c f(n) for a constant 0<c<10<c<1 and all sufficiently large nnΘ(f(n))\Theta(f(n))

Merge sort lands in the middle case because a=b=2a=b=2 and f(n)=Θ(n)f(n)=\Theta(n). The first and third cases require a polynomial gap. A logarithmic factor alone does not establish either case. Count the recursive calls and the work each one does before applying a bound. A recurrence with unequal sizes, such as T(n/3)+T(2n/3)+f(n)T(n/3)+T(2n/3)+f(n) on the right-hand side, falls outside this equal-size form. A recursion tree lets you track the cost of its branches directly.

For a different combine step, the SelfMadeQuant design-paradigms lesson shows how to aggregate P&L across independent backtest runs. The full lesson requires membership.

Sources

Last updated
  • algorithms
  • recursion
  • sorting
  • time complexity
← Back to the glossaryBack to top ↑