Power, Thermal, and Physical Design

This chapter is about the physics. Earlier chapters treated CPUs at the architectural and microarchitectural level: cycles, pipelines, caches, predictors. Underneath is silicon — billions of transistors switching, drawing current, dissipating heat. The physical layer constrains everything above it. Modern CPUs are designed at the limits of what semiconductor manufacturing and thermal engineering can sustain, and many architectural choices (frequency, core count, vector width, cache size) are driven by power and thermal constraints more than by any other factor.

We cover the basics of CMOS power, voltage and frequency scaling, dynamic power management, thermal design, manufacturing, and the slow death of Moore's Law and Dennard scaling.

01. Where the Power Goes

A CMOS transistor switches by charging and discharging the capacitance at its output. Each transition (0 → 1 or 1 → 0) dissipates energy. The dynamic power is approximately:

$P_{dyn} = \alpha \cdot C \cdot V^2 \cdot f$

Where:

• $\alpha$ is the activity factor: the fraction of clock cycles that actually switch this gate.
• $C$ is the load capacitance: the gate's output drives some downstream gates and wires.
• $V$ is the supply voltage.
• $f$ is the clock frequency.

The $V^2$ term is critical: doubling voltage quadruples dynamic power. Conversely, halving voltage cuts dynamic power by 75%, if the circuit can still operate correctly at that voltage (which constrains frequency).

A second contribution is leakage (static) power: even when a transistor isn't switching, it leaks current. Leakage scales with transistor count, temperature (exponentially), and process technology. On modern processes, leakage is 20-40% of total power, especially when the chip is "idle" but still powered.

A third, smaller contribution is short-circuit power: during a transition, both the pull-up and pull-down networks briefly conduct simultaneously. Modern circuits minimize this with fast transitions.

02. Voltage and Frequency Scaling

Frequency and voltage are deeply linked. A higher frequency requires a higher voltage to maintain timing margins (the gate must switch faster, which means stronger current, which requires higher voltage). The relationship is roughly:

• Doubling frequency at constant voltage: 2× dynamic power, but timing fails.
• Doubling frequency with proportional voltage increase: ~8× power increase ($V^2 \cdot f \approx 4 \cdot 2 = 8$).
• Conversely, halving frequency with voltage reduction can cut power by 5-8×.

This is the foundation of DVFS (Dynamic Voltage and Frequency Scaling): the CPU adjusts its voltage and frequency to match workload needs.

A modern CPU has dozens of P-states (performance states) the OS or hardware can select. The states span a wide range: a Ryzen 9 might have idle states near 800 MHz at ~0.6V and turbo states near 5.7 GHz at ~1.5V. The same core, depending on state, has nearly 30× difference in power consumption.

Turbo

When a CPU operates below its full thermal envelope, it can boost frequency above the nominal "base clock." Intel calls this Turbo Boost; AMD calls it Precision Boost; ARM has similar mechanisms. The boost lasts until thermal headroom is exhausted, at which point the CPU drops back.

Turbo is opportunistic: a single-threaded workload can run very fast (because the rest of the chip is idle and contributes no heat); a multi-threaded workload across many cores must run at lower per-core frequency to stay in the thermal envelope.

Single-core turbo values often exceed all-core turbo by 500 MHz or more. A part advertised at 4.0 GHz base / 5.5 GHz boost might run all-core at 4.5-4.8 GHz under heavy load.

Adaptive Voltage / Body Biasing

Some modern processes support adaptive voltage: per-core voltage rails that can be tuned based on the specific silicon's needs (manufacturing variation means some cores work at lower voltage than others). Apple's M-series uses per-core power and frequency controls.

Body biasing (forward or reverse) adjusts the substrate voltage to tune transistor threshold voltages — useful for reducing leakage at idle or boosting performance at active. Used on some advanced processes; not universal.

03. Thermal Design

The thermal design power (TDP) is a marketing-driven number that approximates sustained power dissipation under heavy workload. Real peak power can be 1.5-2× TDP for short bursts, with the boost mechanism throttling back as heat accumulates.

Heat must be moved from the silicon to ambient air (or liquid). The path:

1. Junction: the silicon transistor where heat is generated.
2. Die: thermal spreading across the chip.
3. TIM (Thermal Interface Material): paste or solder between die and IHS (integrated heat spreader).
4. IHS: a metal lid spreading heat to the cooler.
5. TIM2: between IHS and cooler.
6. Cooler: heatsink, fan, liquid cooling — moves heat to ambient.

Each step has thermal resistance ($\Delta T / P$, in $^\circ C/W$). The total resistance times power gives temperature rise above ambient.

Modern desktop CPUs have effective junction-to-ambient resistance around $0.2-0.4 ^\circ C/W$ with a quality cooler. At 200W, that's 40-80°C above ambient. With a 25°C room, the CPU runs at 65-105°C — close to the thermal junction limit (often 100-105°C for x86 desktop parts).

Server CPUs typically allow lower junction temps (85-95°C) for reliability. Mobile CPUs often hit thermal limits faster due to limited cooling and accept lower sustained performance.

Throttling activates when the junction temperature exceeds a threshold. The CPU reduces frequency and voltage; if necessary, it can drop to its lowest P-state and even halt cores ("thermal monitor 2" on Intel; equivalents elsewhere). Sustained throttling indicates inadequate cooling.

04. Power Management

The OS and CPU cooperate on power management:

P-states (performance / active): different frequency/voltage combinations during active execution. The OS hints (via cpufreq governor on Linux); modern CPUs (Intel Speed Shift, AMD CPPC) take over in hardware for finer-grained control.

C-states (idle): when the CPU has nothing to do, it enters increasingly deep idle states. C0 = active. C1 = clock gated (HLT/WFI). Deeper states power-gate clocks, then power-gate the core itself, eventually cache and uncore. Each state has a wake latency (deeper = slower wake) and a power floor.

A modern x86 CPU in C6 (deep idle) might consume 1-2W. Coming out of C6 takes microseconds. The idle governor predicts how long the idle period will last and selects the deepest state that still offers useful power savings.

S-states (system): S0 = running; S3 = suspend to RAM (most hardware off, DRAM in self-refresh); S4 = hibernate to disk; S5 = off.

ACPI standardizes these notions across x86 and many ARM systems.

big.LITTLE / P-cores and E-cores

A complementary approach: heterogeneous cores. ARM big.LITTLE pairs high-performance cores (e.g., Cortex-X4) with efficient cores (e.g., Cortex-A520). Apple does the same (P-cores and E-cores). Intel introduced this on x86 with Alder Lake (P-cores Golden Cove, E-cores Gracemont).

The OS scheduler dispatches workloads to the right cluster: latency-sensitive foreground threads to big cores, background and parallel work to small cores. The small cores draw a fraction of the power, often providing 60-80% of the throughput at 25% of the power.

Heterogeneous scheduling is complex. Misplacing a foreground thread on a small core causes user-visible slowness; running too many threads on small cores wastes throughput. Modern schedulers (Linux's EAS — Energy-Aware Scheduling, Apple's task QoS classes) handle this with workload classification and core affinity hints.

05. Manufacturing Process

Semiconductor manufacturing is the bedrock under all this. The "process node" name (e.g., "5 nm", "3 nm") has become marketing more than physics — actual transistor dimensions don't match the names anymore — but it indicates the generation.

Key process generations:

• 28 nm planar (2011-2014): last planar (non-FinFET) leading-edge process for most foundries.
• 14/16 nm FinFET (2014-2017): first widespread FinFET — 3D fin channels for better leakage control.
• 7 nm (2018-): refined FinFET, EUV (extreme ultraviolet) lithography introduced.
• 5 nm (2020-): TSMC N5 / Samsung 5LPE; widespread EUV.
• 3 nm (2022-): TSMC N3, N3E variants; extreme density.
• 2 nm and below (2025+): GAA (gate-all-around) transistors replacing FinFET.

Each generation reduces transistor area, reduces switching capacitance per transistor, and (often) reduces operating voltage. But the gains are diminishing: from N7 to N5, density gain was ~1.8×; N5 to N3 about ~1.6×. Power-performance gains have similarly compressed.

EUV Lithography

The transition from 193 nm immersion to EUV (13.5 nm wavelength) lithography enabled sub-7 nm features without resorting to extensive multi-patterning. The single tool that produces EUV light is among the most complex machines ever built: ASML's TWINSCAN systems, with tin droplets vaporized by a high-power laser to emit EUV photons through a chamber of mirrors (refractive optics don't work at 13.5 nm).

EUV is now standard at leading-edge foundries (TSMC, Samsung, Intel). It hasn't reduced costs — wafer prices have risen — but it enables continued shrinking.

Variability and Binning

Manufacturing isn't perfectly uniform. Adjacent dies on the same wafer have slightly different transistor characteristics. Some dies hit higher frequencies at lower voltages (better silicon); others need more voltage to hit lower frequencies. Dies are tested and binned: the best go to top-tier products; lesser bins fill out the product stack.

The same physical chip might be sold as a top-tier flagship part or a mid-range product, depending on how it tested. Sometimes failed cache or cores are disabled, and the chip is sold as a lower-core-count or smaller-cache part.

06. The End of Dennard Scaling

Dennard scaling (named after Robert Dennard's 1974 paper) was the historical pattern: as transistors shrunk, voltage could shrink proportionally, keeping power density constant. Twice the transistors at half the size in the same chip area meant the same total power, but with double the gate count or double the frequency.

Dennard scaling broke around 2005-2006. Voltage stopped scaling because leakage at low threshold voltages became uncontrollable. Suddenly, more transistors meant more power per area — power density was rising. The cooling solutions of 2005 couldn't keep up if we kept ramping frequency.

The industry's response: stop scaling frequency. From 2006 onward, single-thread performance gains came from architecture (wider pipelines, better predictors, larger caches) rather than from clock speed. Multi-core became the new dimension of growth.

07. The "Power Wall"

The thermal limit of a chip package is roughly fixed: typical desktop CPUs target 100-250 W; server CPUs 250-400 W; mobile CPUs 15-45 W. Within this budget, the architects allocate transistors to whatever delivers the most performance per watt.

This shapes choices:

• Cores: more cores means more parallel throughput, but only if workloads are parallel. Excess core count idles much of the time.
• Frequency: pushing frequency past the sweet spot is exponentially expensive in power.
• Vector / accelerator: dedicated units (AVX-512, AMX, NPUs) are power-efficient for their target workloads but waste power if unused.
• Cache: large caches are area-expensive and leakage-heavy but reduce off-chip traffic (which is even more expensive in energy).
• Voltage scaling: the most powerful lever; modern CPUs scale voltage aggressively per workload.

The single most important per-watt metric for a modern CPU is energy per useful operation. The goal isn't maximum performance or minimum power — it's the right balance for the deployment.

08. Dark Silicon

A consequence of broken Dennard scaling: dark silicon. Even at full thermal budget, only a fraction of a chip's transistors can be active simultaneously. The rest are "dark" — powered down or unused.

This justifies specialized accelerators: a dedicated AI matrix engine that's idle during normal compute is fine, because we couldn't use those transistors for general compute anyway (the chip would overheat). Better to spend the transistors on specialization that wins big when used.

Apple's M-series epitomizes this: the chip has GPU, NPU (Neural Engine), media engines, and a Secure Enclave, all coexisting on one die. Each is power-gated when idle. The CPU cores share the budget with the rest, dynamically.

09. Power Delivery

Delivering hundreds of watts at 1V or below requires a sophisticated power-delivery network:

• VRM (Voltage Regulator Module) on the motherboard converts 12V (or 48V in datacenters) to ~1V at high current.
• On-package or on-die regulators further regulate per-domain voltages, allowing fine-grained voltage control.
• Decoupling capacitors smooth current spikes during workload transitions.

A modern desktop CPU can swing from 5W idle to 250W full load within milliseconds. The voltage regulator must respond fast enough to keep voltage stable; otherwise the CPU undervolts and crashes (or overshoots and damages itself).

Server and AI accelerators have pushed power delivery to extremes: NVIDIA's H100 and B200 GPUs can pull 700-1000W; future generations may push toward 2 kW. Datacenter power distribution is moving from 12V to 48V to 400V to reduce I^2R losses in distribution.

10. Reliability and Aging

Transistors degrade over time:

• NBTI (Negative-Bias Temperature Instability): PMOS threshold voltage drifts under negative gate bias, especially at high temperature.
• HCI (Hot Carrier Injection): hot electrons damage the gate oxide.
• Electromigration: high current density causes metal atoms to migrate, eventually breaking interconnects.
• TDDB (Time-Dependent Dielectric Breakdown): gate oxide eventually fails.

Manufacturers design for a typical 7-10 year service life at rated conditions. Operating at higher voltage/temperature significantly accelerates aging. Servers run cool to maximize service life; consumer CPUs run hotter.

Adaptive voltage scaling can compensate for aging: as the chip ages and threshold voltages drift, the supply voltage is increased slightly to maintain timing. This is a form of in-field calibration.

11. Variability-Aware Design

Modern designs explicitly account for variability. Worst-case design (assume the slowest possible silicon) wastes performance on average parts. Adaptive designs measure each chip's characteristics and tune accordingly:

• Resonant clock distribution for power efficiency.
• Adaptive clocking: clocks slow down briefly during voltage droops (rather than crashing).
• Per-core / per-cluster voltage rails: each core gets the voltage it needs, not a worst-case voltage.
• In-field telemetry: temperature, voltage, current monitoring used to dynamically tune.

12. Summary

The physical foundation of computing is shaped by CMOS physics: dynamic power scales with $V^2 \cdot f$, leakage with transistor count and temperature, and reliability with operating conditions. DVFS, deep sleep states, heterogeneous cores, and architectural specialization are all responses to power and thermal constraints. Manufacturing process advances continue but with diminishing per-node gains; the era of "free" performance from process scaling is over.

The "power wall" reshaped processor architecture from frequency scaling (1990s-early 2000s) to multi-core (mid-2000s-2010s) to specialized accelerators and big.LITTLE heterogeneity (mid-2010s onward). Dark silicon makes specialization economically attractive. Most architectural choices today are bounded by the W/mm² thermal envelope of the package.

The next chapter looks at reliability and validation: how chips are tested, how soft errors are detected and corrected, and how complex systems are designed to keep working despite failures.