Today’s systems ranging from mobile to servers are characterized by deep software stacks and massive instruction working sets. The instruction-heavy workloads exert immense pressure on the critical front-end structures of the CPU, including the instruction cache, branch predictor, BTB, etc. Because these components must operate under stringent latency constraints, they cannot simply be scaled up to match the explosive growth of modern binaries. Consequently, the front-end is the primary performance bottleneck in contemporary CPUs including those in our pocket and in the cloud. Notably, this is not a new problem, having first been identified in server workloads over 3 decades ago!
In this series of lectures, we will dissect the front-end through the lens of modern architectural and microarchitectural trade-offs. We will trace the evolution of ideas in this space and the industry’s response to the front-end problem. We will conclude by examining the latest research and the outlook for the future.