Summary
Highlights
BBT highlights that the most impactful optimizations are often not glamorous or highly visible. Users didn't know BBT existed; they just noticed their machines felt faster and more stable. This 'invisible' improvement represents the highest form of systems engineering, leading to millions of machines performing more efficiently without explicit user awareness.
Dave Plamer introduces BBT, a clandestine optimization tool from his Microsoft days. This tool, also known as Lego, worked on binaries after compilation and linking, secretly enhancing performance by rearranging code without developer intervention. This practice was unknown outside Microsoft for nearly 30 years.
In the 90s, memory was a critical constraint. Operating systems like Windows NT4 and Windows 95, along with applications like Office, were growing in size and complexity, but hardware was limited. Machines often had as little as 12 megabytes of RAM, making memory efficiency paramount to avoid performance bottlenecks like excessive swapping.
Unlike compilers and linkers that operate on source code and object files, BBT intervened at the binary level. It treated basic blocks of machine instructions like 'Lego bricks,' rearranging them based on actual execution profiles. This allowed commonly executed code paths to be grouped together, improving memory locality and reducing the working set size.
The working set refers to the memory pages a process needs to run smoothly. BBT tackled the issue where necessary code was scattered across many memory pages, forcing the OS to load more data than required. By clustering 'hot' code paths and moving 'cold' or rarely used code away, BBT ensured only essential pages were in RAM, reducing page faults and improving responsiveness.
The BBT process involved building the program, analyzing the binary to identify basic blocks, instrumenting it or using profile data from representative runs to understand execution patterns, and then rebuilding the executable's layout. The effectiveness of this profile-guided optimization heavily depended on 'representative' workloads for training, as optimizing for the wrong scenario could yield poor results.
Rewriting binaries safely is extremely complex. It involves meticulously managing entries, import/export tables, relocation records, exception metadata, and debug symbols. Any error could lead to crashes in specific, hard-to-debug scenarios. Microsoft's unique resources and build discipline allowed them to manage this risk effectively.
Plamer emphasizes that binary layout directly impacts the user experience. By optimizing how code was arranged in memory, BBT made applications feel faster and more responsive, indirectly improving the "user interface." This principle of locality remains crucial today, as seen in modern tools like LLVM's Bolt and Propeller, despite advancements in hardware.