Part IIIOut-of-Order Execution
Security, Side Channels, and Speculative Execution Hazards
July 31, 2026·46 min read·advanced
text What changed? The instruction set manual answers precisely. RAX holds 5, RBX holds the eight bytes at that address, the program counter advanced, nothing else. Dump every register and every byte of…
01.Part 1, why a performance feature became a security problem
1.1 Two kinds of state
Run two instructions.
mov rax, 5 // put the constant 5 into RAX
mov rbx, [rcx] // load 8 bytes from the address in RCX into RBX
```text
What changed? The instruction set manual answers precisely. RAX holds 5, RBX holds the eight bytes at that address, the program counter advanced, nothing else. Dump every register and every byte of memory, compare against a hand simulation, and the two agree exactly. For forty years that was treated as the complete answer.
It is not the complete answer to what physically changed. The load installed a TLB entry and evicted an old one, allocated a fill buffer, activated a DRAM row and left it open, wrote a 64-byte line into L1 by choosing a victim in that set and dropping it, and fed its stride to a prefetcher that may have started a stream. None of that appears in the manual.
<Figure src="/figures/hardware-interview-prep/iv-16-Security-Side-Channels-and-Speculation-fig01.svg" alt="The instruction set makes promises about the upper box only, so every structure in the lower box is state the implementation keeps without any architectural obligation to preserve or erase it." caption="The instruction set makes promises about the upper box only, so every structure in the lower box is state the implementation keeps without any architectural obligation to preserve or erase it." id="fig:16-Security-Side-Channels-and-Speculation-1" />
**Architectural state** is what the instruction set defines and guarantees. **Microarchitectural state** is every other piece of storage the implementation happens to keep. The line between them is contractual, not physical.
That the ISA promises nothing about the lower box is not an oversight, it is why microarchitecture is a discipline. A machine with a cache and a machine without one are both correct implementations of the same ISA, running the same binaries and giving the same answers, and one is a hundred times faster. That freedom allowed forty years of improvement without recompiling anything, and it was believed safe because nothing in the lower box was thought observable in a way that mattered. That belief was the mistake.
### 1.2 Squash restores the top box only
From [Out of Order Execution](/learn/hardware-interview-prep/out-of-order-execution), recovery from a wrong prediction restores the program counter, restores the rename map from a checkpoint, frees the physical registers wrong-path instructions allocated, clears their reorder buffer entries, and removes their stores from the store queue before those stores reach memory. Afterwards the architectural state is exactly what it would have been had the wrong path never been fetched. That guarantee holds and nobody has found a bug in it.
Recovery does not evict the lines the wrong path brought into the cache, remove the TLB entries it installed, roll back the prefetcher's stream table, or reverse the predictor updates the wrong path itself made. Put a number on it. Forty wrong-path instructions with 8 L1 misses among them pull in 8 lines of 64 bytes and evict 8 others, so 512 bytes of cache hold different data after the squash than they would have, permanently, in the sense that nothing will ever put it back.
### 1.3 The core insight, stated once
**Misspeculation recovery is defined over architectural state only. The microarchitectural traces of speculatively executed work are not undone. Anything that can measure microarchitectural state can therefore observe the results of work the program never architecturally performed.**
Two things about it are counterintuitive. It looks like a squash bug and is not, because the squash is correct by the definition of correctness everybody agreed to. And the attacker is not reading the discarded register value, which is genuinely gone. What survives is the shadow that value cast on the way past, in the form of which cache line got filled, and the attacker reads the shadow. That is why nobody noticed for two decades. The definition of correctness was not wrong. It was incomplete.
### 1.4 Every attack has exactly two halves
A **transmitter** causes a microarchitectural change whose identity depends on a secret. A **receiver** measures that change and recovers the secret. In every attack here the receiver is a cache timing measurement.
| Attack | Transmitter, the bug | Receiver |
|---|---|---|
| classic cache timing on RSA | the victim's own code branches on a secret bit | FLUSH+RELOAD on a shared library |
| Spectre v1 | mistrained direction predictor runs a secret-dependent load | FLUSH+RELOAD on a probe array |
| Spectre v2 | mistrained BTB steers the victim into a gadget | same |
| Meltdown | faulting load forwards data before the permission check | same |
| MDS | a stale fill buffer entry is forwarded to a transient path | same |
The right column never changes. The receiver is generic, reusable, and not itself a defect, while the left column differs every time, which is why new variants kept appearing for years. Once the receiver exists, every new transmitter is a new CVE. It also says where defence must happen. You cannot remove the receiver, because you cannot remove the cache and cannot forbid programs from measuring time, so defence has to attack the transmitter or make the channel too slow to be worth using. Part 2 builds the receiver because it is the simpler half.
---
## Part 2, the covert channel primitive
### 2.1 A clock can tell a hit from a miss
Take the latency ladder from [Cache Organization and Prefetching](/learn/hardware-interview-prep/cache-organization-and-prefetching) at 3 GHz, where one cycle is 0.333 nanoseconds.
```text
WHERE THE DATA WAS CYCLES NANOSECONDS RELATIVE
----------------------------------------------------------------
L1 data cache hit 4 1.3 1x
L2 hit 14 4.7 3.5x
last level cache hit 45 15.0 11x
DRAM, row already open 150 50.0 38x
DRAM, row miss, activate needed 200 66.7 50x
A 50x gap. Not a subtle statistical effect. It is the
difference between a one second answer and a fifty second one.
```text
That gap is the entire receiver. Time one memory access and you learn whether the line was in L1 or in DRAM, which is a bit of information about what the machine did recently.
On x86 the timer is `rdtsc`, fenced with `lfence` on both sides so nothing drifts across the measurement. On ARM it is `PMCCNTR_EL0`, which privileged software must enable, or the coarser `CNTVCT_EL0`. Where no architectural timer is reachable an attacker builds one by spinning a second thread that increments a shared counter, which ticks every few cycles and is plenty, and that is exactly why browsers responded in 2018 by coarsening `performance.now()` and disabling `SharedArrayBuffer`. One sample is noise, so measure 100 times and take the minimum, since noise only ever adds time. A threshold near 120 cycles then separates hits from misses.
### 2.2 FLUSH+RELOAD
The strongest of the three techniques. It needs shared physical memory, which sounds restrictive and is not, because shared libraries are backed by the same physical pages in every process that links them, memory-mapped files behave the same way, and hypervisors that deduplicate identical guest pages create the same sharing between virtual machines.
<Figure src="/figures/hardware-interview-prep/iv-16-Security-Side-Channels-and-Speculation-fig02.svg" alt="The attacker never watches the victim directly; it empties one shared line, hands the machine over, and then reads its own stopwatch to learn whether the victim put that line back." caption="The attacker never watches the victim directly; it empties one shared line, hands the machine over, and then reads its own stopwatch to learn whether the victim put that line back." id="fig:16-Security-Side-Channels-and-Speculation-2" />
`clflush` is unprivileged on x86, because flushing a line you are allowed to read is not a privilege violation by any sensible definition. ARM's `DC CIVAC` is the analogue.
Work a real target. Textbook modular exponentiation for RSA squares every iteration and additionally multiplies when the current exponent bit is 1, so the multiply routine executes if and only if that bit is 1. The attacker maps the same crypto library, so that routine's instruction bytes sit in shared physical pages. Flush the line containing its entry point, wait one loop iteration, reload and time. Fast means the bit was 1, and the private exponent falls out bit by bit. No CPU bug and no speculation was involved. The cache behaved as designed and the key leaked anyway, because the victim's own control flow depended on a secret and the cache recorded which way it went. This is why constant-time cryptography exists, and it predates Spectre by more than a decade.
### 2.3 Encoding a whole byte
One measurement leaks one bit. To leak a byte, use 256 lines and let the secret choose which. Allocate a **probe array** of 256 slots, each 4096 bytes, one megabyte total, and flush all of it. Then arrange, by whatever transmitter Parts 3 through 5 supply, for the machine to touch `probe[secret * 4096]`. Then time all 256 slots and find the fast one.
<Figure src="/figures/hardware-interview-prep/iv-16-Security-Side-Channels-and-Speculation-fig03.svg" alt="Every slot but one reloads at DRAM latency, so the index of the single fast slot is the secret byte itself." caption="Every slot but one reloads at DRAM latency, so the index of the single fast slot is the secret byte itself." id="fig:16-Security-Side-Channels-and-Speculation-3" />
Why 4096 bytes of spacing rather than the 64-byte line size? Prefetchers. An adjacent-line prefetcher would light up slots 64 and 66 as collateral, and the reload pass itself walks 256 consecutive lines, exactly the pattern a stream prefetcher loves, so it would run ahead of the attacker's own probes and destroy the measurement. Putting each slot on its own 4 KB page defeats both, because hardware prefetchers do not cross page boundaries. They cannot, since the physical pages behind two adjacent virtual pages need not be adjacent.
Now the throughput arithmetic, which lets you check published numbers rather than memorize them. A naive reload pass is 255 misses at roughly 200 cycles plus one hit, about 51,000 cycles, or 17 microseconds at 3 GHz, which is 59 kilobytes per second. The original Spectre paper reported around 10 kilobytes per second, below that ceiling because retriggering the transmitter and retraining the predictor costs more than the measurement does. The Meltdown paper reported several hundred kilobytes per second, above the ceiling because of an obvious optimization. Leak four bits at a time with a 16-slot probe array, so the reload pass is 15 misses plus a hit, roughly 3,000 cycles, giving 2 microseconds per byte and 500 kilobytes per second. That derivation lands on the published figure almost exactly.
### 2.4 PRIME+PROBE, with nothing shared
Remove shared memory and you need a different receiver. PRIME+PROBE uses the fact that a set-associative cache has limited ways per set, so the victim's accesses must evict something.
Take a 32 kilobyte, 8-way L1 data cache with 64-byte lines. That is $32768/64 = 512$ lines and $512/8 = 64$ sets. Bits [5:0] pick the byte in the line and bits [11:6] pick the set, so two addresses share a set when they differ by a multiple of $64 \times 64 = 4096$ bytes. That 4096 is the page size, which is not a coincidence, it is the condition that makes a virtually indexed physically tagged L1 work.
<Figure src="/figures/hardware-interview-prep/iv-16-Security-Side-Channels-and-Speculation-fig04.svg" alt="With no memory shared at all, the attacker learns that the victim touched set 37 simply because one of its own eight primed ways came back slow." caption="With no memory shared at all, the attacker learns that the victim touched set 37 simply because one of its own eight primed ways came back slow." id="fig:16-Security-Side-Channels-and-Speculation-4" />
You learn the set index, 6 address bits here. Coarser than FLUSH+RELOAD, which identifies an exact line, and noisier, because anything else on the machine also evicts, but it needs no shared memory, so it works across virtual machines against a victim that shares nothing with you but silicon. Against the physically indexed and slice-hashed last level cache it is harder, and public research solved that by searching iteratively for a minimal group of addresses that evict each other. **EVICT+TIME** is a third and weaker variant that times the victim end to end, evicts a candidate set, and times it again, giving one bit per victim execution but needing neither shared memory nor fine-grained timing.
### 2.5 The separation that matters
None of Part 2 is a bug. The RSA attack works on a perfectly correct CPU, and the fix there is in software, because the victim's source branched on a secret and a careful programmer stops doing that. In Spectre the victim's source contains no such branch, the machine invents one by mispredicting, and no amount of care in the source removes it. That is the difference between a software problem and a hardware problem.
---
## Part 3, Spectre
### 3.1 The gadget, which is correct code
```c
// array1 is a byte array of length array1_size
// array2 is the 1 MiB probe array from section 2.3
// x is supplied by the caller and may be anything
if (x < array1_size)
y = array2[array1[x] * 4096];
```text
Read it as a reviewer. The bounds check is present and correct, `array1[x]` is evaluated only when `x` is legal, and there is no overflow, no signedness bug, no off-by-one. A memory-safety checker passes it. The pattern is everywhere, especially at a system call entry where `x` arrives from user space and `array1_size` is a kernel-side bound.
### 3.2 Training, then widening the window
From [Front End and Branch Prediction](/learn/hardware-interview-prep/front-end-and-branch-prediction), the direction predictor is a table of saturating counters indexed by a function of the branch address and recent history. Call the function a few hundred times with legal values of `x` and every call takes the branch, which saturates a two-bit counter after three outcomes and fills the history-indexed entries of a TAGE-style predictor. The predictor now believes the branch is taken with high confidence, and it is not wrong. It has seen hundreds of taken outcomes and zero others.
Two preparations then turn a few cycles of speculation into a few hundred. Flush `array1_size` out of the cache, so the comparison cannot resolve until it returns from DRAM, about 200 cycles. Flush all 256 probe slots, so any slot found fast afterward must have been filled during this attempt.
How much speculation does 200 cycles buy? On a 4-wide machine that is 800 instruction slots, and in practice the limit is the reorder buffer, several hundred entries on a big core. Either number is enormous next to what the attack needs, which is one address computation, one load, one shift, and one more load. Roughly five micro-operations. The window is three orders of magnitude larger than required, which is why widening it is easy and narrowing it is not a useful defence.
### 3.3 The timeline
Set `array1_size = 16` and call with `x = 1000`. The address `array1 + 1000` lands outside the array where the attacker arranged for a secret to live. Say that byte is `0x41`, which is 65.
<Figure src="/figures/hardware-interview-prep/iv-16-Security-Side-Channels-and-Speculation-fig05.svg" alt="Architectural state stays empty from the first cycle to the last, yet the cache column changes permanently, which is the whole of Spectre v1 in one column pair." caption="Architectural state stays empty from the first cycle to the last, yet the cache column changes permanently, which is the whole of Spectre v1 in one column pair." id="fig:16-Security-Side-Channels-and-Speculation-5" />
The attacker then times all 256 slots. Slot 65 returns in 43 cycles and the rest in about 200. The secret byte was 65.
One detail is where people go wrong. It does not matter whether the fill for `probe[65]` completes before or after the squash at cycle 202. **A fill request already issued to the memory system is not cancelled by a pipeline squash.** It sits in a fill buffer, travels outward, and the returning line is allocated into L1 when it arrives.
### 3.4 Three reasons the footprint survives
An interviewer asking why the squash does not clean up the cache is checking whether you have one reason or several. The fill was already handed to the memory system and memory systems have no undo, and cancelling it would mean tracking which outstanding fills are speculative and unwinding the fill buffer allocation, which no shipping cache does. Even if the fill could be cancelled the eviction it caused already happened, since allocating in a full set means choosing a victim and dropping it. And the wrong path updated more than the cache, since it updated predictor state, may have trained a prefetch stream, and consumed fill buffers whose contents persist.
### 3.5 Variant 2, branch target injection
Variant 1 mistrains a direction predictor to run past a check. Variant 2 mistrains a target predictor to run somewhere else, and it is worse because the attacker chooses the destination. An indirect branch has a target unknown until the register is read, so the front end predicts it from the branch target buffer, a table indexed by a function of the branch's own address. Before the 2018 fixes that index used a subset of address bits and the entry carried no tag for privilege level or address space.
<Figure src="/figures/hardware-interview-prep/iv-16-Security-Side-Channels-and-Speculation-fig06.svg" alt="Two branches at completely different privilege levels land on the same untagged BTB entry, so whatever the attacker taught that entry is what the victim's front end believes." caption="Two branches at completely different privilege levels land on the same untagged BTB entry, so whatever the attacker taught that entry is what the victim's front end believes." id="fig:16-Security-Side-Channels-and-Speculation-6" />
The attacker never executes a single instruction inside the victim. It only changes what the victim's own predictor believes about the victim's own code, and the victim does all the work itself, with its own privileges, on its own data. The gadget need not be a function anyone wrote for the purpose, only a byte sequence that loads from a register-derived address and uses the result to index a second load, and gadget search in a multi-megabyte kernel is the same search problem as classic return-oriented programming, which reliably succeeds.
### 3.6 Why Spectre is genuinely hard
It attacks correct code, so there is no bug in the victim to fix, no compiler warning can fire, and rewriting in a memory-safe language does not help, because the machine's willingness to run the body under a false condition is below the language. The attack surface is the whole binary, since any load-then-dependent-load pair is a candidate gadget. And you cannot simply stop speculating.
Work that last one. Take a base CPI of 0.5, branches at 20 percent of instructions, and a 15-cycle mispredict penalty. At 98 percent accuracy the added CPI is
$$0.20 \times 0.02 \times 15 = 0.06$$
for a total of 0.56, about 1.79 instructions per cycle. With no prediction the front end stalls at every branch until the condition resolves, call it the same 15 cycles, so the added CPI is
$$0.20 \times 1.00 \times 15 = 3.0$$
for a total of 3.5, about 0.29 instructions per cycle. That is $3.5/0.56 = 6.25$ times slower. A six-fold slowdown is not a mitigation, it is a different product, which is why every real defence is surgical rather than a switch.
---
## Part 4, Meltdown and the fault-deferral class
### 4.1 What precise exceptions force the hardware to do
From [CPU Foundations Pipeline and Hazards](/learn/hardware-interview-prep/cpu-foundations-pipeline-and-hazards), an exception is **precise** when three things hold as the handler starts. Every instruction before the faulting one has completed and is architecturally visible, no instruction at or after it has had any architectural effect, and the saved program counter points at the faulting instruction so the handler can fix the problem and re-execute it. Demand paging depends on that.
Now put the requirement into an out-of-order machine. Instructions execute out of program order, so the load at position 40 may execute long before the load at position 20, and if the load at 40 raised its fault the instant it detected one, the handler would run with instructions 20 through 39 unfinished. So the machine does the only thing it can. A faulting instruction records the fault in its reorder buffer entry and keeps going, and the fault is raised when that entry reaches the head of the ROB. Deferral to retirement is not a shortcut, it is the mechanism that makes precision possible.
### 4.2 The window deferral creates
<Figure src="/figures/hardware-interview-prep/iv-16-Security-Side-Channels-and-Speculation-fig07.svg" alt="The data leaves the array at cycle 3 and the permission answer arrives at cycle 4, and everything Meltdown does happens inside that one-cycle inversion." caption="The data leaves the array at cycle 3 and the permission answer arrives at cycle 4, and everything Meltdown does happens inside that one-cycle inversion." id="fig:16-Security-Side-Channels-and-Speculation-7" />
That is the whole bug. A handful of cycles between the data array producing a value and the permission logic saying no, during which dependents consumed the value. In an in-order machine those cycles exist too, but nothing downstream runs far enough to build a transmitter.
### 4.3 The attack
```c
// running as an ordinary unprivileged user process
// step 1, survive the fault. Three public ways:
// (a) install a SIGSEGV handler and longjmp back
// (b) wrap it in a hardware transaction so the abort resumes
// (c) put the faulting load on a mispredicted path, never retires
char v = *(char *) kernel_address; // faults at RETIRE, not now
y = probe[v * 4096]; // transmitter, runs transiently
// step 2, after recovering, time all 256 probe slots.
```text
The transmitter on line two is identical to Spectre v1's. What changed is how much work the attacker does to get it to run. Spectre v1 needs a suitable gadget in the victim, needs to find it, and needs to mistrain a predictor to reach it. Meltdown needs a faulting load, and the machine runs the dependents during the deferral window without being asked. No gadget, no predictor training, and no dependence on the victim's code at all, because the victim is the kernel's page mapping rather than the kernel's instructions.
### 4.4 This one was a bug, and the fix is cheap
Saying which of these are inherent and which are implementation mistakes decides whether the fix is free or expensive. Meltdown is the clearest mistake in the family, and there are two cheap correct implementations. Do not forward load data to dependents until the permission result resolves, which is affordable because the permission bits come out of the same TLB lookup that produced the physical address, so the information is already present at roughly the same moment. The cost is a tighter timing path on the L1 hit loop, which is real, bounded, and a physical design problem rather than an architectural one. Or forward a constant instead of the data on a permission failure, so dependents compute on zeros and are squashed anyway, which costs one multiplexer controlled by a bit that already exists.
The strongest evidence that this was a choice is that contemporaneous designs from other vendors were not affected. AMD's public position in 2018 was that its designs did not let a load forward data to dependents before the permission check completed, and ARM published a whitepaper listing which of its cores were affected, most of which were not. Machines built in the same years, at the same performance, with the same out-of-order depth, did the check in the right order. Contrast Spectre v1, where correct behaviour means not executing the body until the condition resolves, costing the 6.25x from 3.6. That is a genuine tradeoff. Meltdown was not one.
---
## Part 5, the wider family
### 5.1 Microarchitectural Data Sampling
The cache is not the only place data sits. A load that misses allocates a **fill buffer** entry to hold the incoming line, stores sit in the **store buffer** until they retire and drain, and loads occupy **load ports** with staging latches. All three hold real data, all three are shared by whatever runs on the core, and none is tagged with an owner the way a cache line is tagged with an address. On affected designs a load that faulted, or that required a microcode assist, could receive stale data from one of these buffers belonging to a different context and forward it to dependents that ran transiently and transmitted it.
The character of the leak differs from Meltdown and the name says so. This is **sampling**, not addressing. The attacker does not choose an address, it gets whatever is in the buffer at that instant, a stream of essentially random bytes from other contexts, so the work is statistical, running the loop millions of times and filtering for recognizable structure such as a key format. Simultaneous multithreading makes it far better for the attacker, because from [Simultaneous Multithreading (SMT)](/learn/computer-architecture/smt) the sibling thread shares those exact buffers.
| Public name | Buffer sampled |
|---|---|
| RIDL, also MFBDS | line fill buffers |
| Fallout, also MSBDS | store buffer |
| MLPDS | load ports |
| MDSUM | uncacheable memory through the same buffers |
The mitigation was microcode plus operating system cooperation. `VERW`, an existing instruction, was redefined by microcode to flush these buffers, and operating systems execute it on every privilege transition. Where the threat model includes untrusted co-tenants, SMT is disabled outright, costing whatever SMT was buying, commonly 15 to 30 percent on throughput workloads.
### 5.2 Foreshadow and the terminal fault
A page table entry has a present bit, and when it is clear the entry is architecturally meaningless and its remaining bits are software's to use. On affected designs a load hitting a non-present entry still used the physical address field to index L1D speculatively while the fault was deferred, so any data resident in L1 at that physical address could be read transiently. Severity came down to who controls the entry. A guest operating system controls its own page tables, so a malicious guest could construct a non-present entry containing any host physical address and read whatever was in L1 there, and the same mechanism broke enclave isolation. Two mitigations shipped, flushing L1D on every virtual machine entry, and **PTE inversion**, where the operating system inverts the physical address bits when clearing the present bit so the speculative address points outside installed memory.
### 5.3 Speculative store bypass
This one comes straight out of [Load Store and Memory Ordering](/learn/hardware-interview-prep/load-store-and-memory-ordering) and is the variant most worth understanding for the X role. A load must not bypass an older store to the same address, but checking that requires knowing the addresses of all older stores and some have not computed theirs yet. Waiting is slow and usually pointless, so the machine predicts. A **memory dependence predictor** learns per load whether that load has historically conflicted, and on a no-conflict prediction the load issues immediately and reads from cache.
Work the failure. A store to address X sits in the store buffer with its address unresolved, a load from X issues, the predictor says no conflict, and the load reads the cache and gets the value of X *before* the store, which is stale. Dependents compute on it. Later the store's address resolves, the conflict is detected, and the load and everything downstream are squashed and replayed, so architecturally the answer is correct. Transiently, dependents computed on the old contents of that location, and if those contents are a secret and the dependents form a transmitter, that is Spectre variant 4. The software pattern that made it practical is worth naming. A sandbox, typically a JavaScript engine, enforces a bound by writing a masked safe value into a slot and then reading the slot back, and speculatively the read bypasses the write and returns the unmasked value that was there before, so the sandbox's own enforcement is undone by the machine without the sandbox doing anything wrong.
The mitigation is a control bit, speculative store bypass disable, that turns the prediction off so loads wait for older store addresses, at a public cost from a few percent up to around 8 percent. Is it a design bug? Honestly no, or much less so than Meltdown, because the predictor does exactly what it was built to do and its mispredictions are recovered correctly. The problem is the general one, that transient results were observable, and the fix is not "predict better" but "do not let transient work leave traces", which nobody has solved cheaply.
### 5.4 Load value injection, the arrow reversed
Everything so far has the attacker reading the victim's data through a transient path. LVI runs the other way. The attacker poisons a microarchitectural buffer with a chosen value, then arranges for a **victim** load to fault or require an assist, and that load transiently picks up the poisoned value instead of the real data. The victim's own subsequent instructions then execute transiently on attacker-chosen data, so if they dereference it the victim leaks its own memory, and if it lands in an indirect branch target the victim transiently executes at an attacker-chosen address with the victim's privileges. Buffer flushing does not help, because the injection happens inside the victim's execution and the flush happens at the boundary crossing, which is the wrong side of the event.
The published mitigation was compiler-inserted `lfence` after essentially every load plus hardening of indirect branches and returns, and the original paper reported overheads on fully hardened enclave code from roughly two-fold to nineteen-fold. That number is the point of including this variant. Some attacks here have no known general mitigation anyone can afford, and the industry's answer has been to bound the threat model rather than pay it.
### 5.5 Which are bugs and which are inherent
| Variant | Verdict | Reasoning |
|---|---|---|
| Meltdown | **bug** | a permission check existed and was applied after the data was forwarded. Contemporaneous designs got the order right. |
| Foreshadow, L1TF | **bug** | same shape. A present-bit check existed and the physical address was used before it resolved. |
| MDS | **bug** | buffers held data across a boundary with no ownership tagging and no clearing at the transition. |
| Spectre v2 | **mostly bug** | prediction structures were untagged and shared across privilege and address space. Tagging is a normal design decision. |
| Spectre v4, SSB | **borderline** | the predictor is correct and its recovery is correct. What leaks is transient observability. |
| Spectre v1 | **inherent** | no check was violated. The machine ran instructions it later discarded, which is what speculation is. |
| LVI | **inherent, and worse** | requires trusting nothing about the values loads return. |
The pattern is one sentence. **Where a check already existed and was merely applied too late, it was a bug and it got fixed in silicon quickly and cheaply. Where no check was violated and the machine simply ran work it later discarded, it is inherent, and eight years later it is still being mitigated case by case.**
---
## Part 6, defences and what each one costs
### 6.1 Hardware
**Do not forward faulting-load data.** The Meltdown fix from 4.4. Cost is a multiplexer and a tightened path on the L1 hit loop, and it shipped broadly in 2018 and 2019 parts with vendors publicly stating no measurable performance impact.
**Tag predictor entries by context.** Make a BTB entry usable only by the privilege level, address space, or thread that created it. ARM exposes architectural feature bits, CSV2 and CSV3, by which an implementation tells software it provides these guarantees, so software can skip mitigations rather than applying them blindly, and Intel's enhanced IBRS and AMD's automatic IBRS are the x86 equivalents. Cost is capacity, not latency. A 4096-entry BTB shared between two cooperating contexts gives either of them all 4096, and partitioning gives each 2048, so mispredicts go up. Modest, and much cheaper than flushing.
**Flush or invalidate at boundary crossings.** The indirect branch predictor barrier, branch history clearing, and the `VERW` buffer flush from 5.1. Cost is paid per transition and lands exactly where it hurts, since a system call that cost 100 cycles of entry overhead now costs several hundred plus a cold predictor for the next few hundred instructions.
**Partition the cache.** Cost is capacity, directly. Work it for a memory-bound workload where 2 percent of memory instructions miss the last level cache. Halving effective capacity raises the miss rate by roughly $\sqrt{2}$, to about 2.83 percent, and with 30 percent of instructions being memory operations and a 200-cycle penalty the added CPI is
$$0.30 \times 0.0083 \times 200 = 0.50$$
Against a base CPI of 0.6 that nearly doubles execution time, which is why static partitioning appears in specific high-assurance products and not in general purpose cores. The research alternative is a **randomized-index cache**, where the set index is a keyed function of the address so an attacker cannot construct an eviction set arithmetically. CEASER and ScatterCache are the known proposals, and as far as public documentation shows no mainstream shipping core uses one, which is an inference from absence rather than a positive fact.
**Guarantee data-independent timing.** ARMv8.4 added the DIT bit in PSTATE and Intel documents a data operand independent timing mode, under which an enumerated subset of instructions is architecturally guaranteed to take the same time and leave the same microarchitectural footprint regardless of operand values. That matters because without an architectural guarantee a constant-time crypto library relies on undocumented behaviour a future core is free to change. Cost is nothing when off, and when on it disables data-dependent optimizations so the affected instructions run at worst-case latency. The connection back to [Arithmetic Hardware](/learn/hardware-interview-prep/arithmetic-hardware) is direct. An early-terminating multiplier that finishes in two cycles for a small operand and five otherwise is a timing channel by construction, and a divider whose iteration count depends on operand magnitude is the same. Ordinary PPA optimizations become vulnerabilities the moment the operand is a key.
**Make speculation invisible.** The research direction. Hold speculatively fetched lines in a separate structure and merge them into the cache only at retirement, which is InvisiSpec and SafeSpec, or track which values are speculative and secret-derived and delay any operation that would make them observable, which is speculative taint tracking. Published overheads range from a few percent to tens of percent, and no mainstream shipping core implements one according to public documentation.
### 6.2 Software and firmware
**Serializing barriers.** On x86 `LFENCE` was documented after 2018 as a speculation barrier, and ARM has `CSDB` and, from ARMv8.5, `SB`. Put one between a bounds check and the use and the machine will not run the use speculatively, at the cost of a full window's worth of instruction level parallelism, which is acceptable on one hot system call entry and ruinous applied everywhere.
**Index masking.** Replace the control dependence with a data dependence.
```c
// vulnerable: the bound is enforced by a BRANCH,
// and branches can be mispredicted
if (x < array1_size)
y = array2[array1[x] * 4096];
// hardened: the bound is enforced by ARITHMETIC,
// and arithmetic cannot be mispredicted
if (x < array1_size) {
x = array_index_nospec(x, array1_size); // builds a mask, no branch
y = array2[array1[x] * 4096];
}
```text
`array_index_nospec` computes a mask that is all ones when the index is in range and all zeros when it is not, using comparison arithmetic rather than a branch, and ANDs the index with it. On the speculative path the index becomes 0 rather than 1000, so the load reads `array1[0]`, which is inside the array and carries no secret. Cost is one AND. The general principle, converting a control dependence into a data dependence, is the same transformation as if-conversion from [Front End and Branch Prediction](/learn/hardware-interview-prep/front-end-and-branch-prediction) applied for a different reason.
**Retpoline.** A software replacement for indirect branches that steers speculation into a dead end.
```text
; instead of: jmp *%rax
call set_up_target
capture_speculation:
pause ; hint, low power spin
lfence ; and stop speculating
jmp capture_speculation ; speculation LOOPS HERE forever
set_up_target:
mov %rax, (%rsp) ; overwrite the pushed return address
ret ; ARCHITECTURALLY goes to the real target
```text
The `call` pushes a return address onto the stack and also onto the return address stack predictor, which is what predicts `ret`, so the front end predicts `capture_speculation` and speculation runs into the spin loop and does nothing observable, while the `mov` replaced the real return address so architecturally the `ret` goes where it should. Speculation goes one place and architecture goes another, on purpose. Cost was reported from roughly 1.5 to 10 percent depending on indirect-branch density. The caveat matters. Retbleed in 2022 showed that on some cores a `ret` falls back to BTB prediction when the return stack underflows, which defeats retpoline entirely, so a software mitigation built on undocumented hardware behaviour has a shelf life.
**Kernel page table isolation.** From [TLBs and Address Translation](/learn/computer-architecture/tlbs), the kernel is normally mapped into every process's address space with supervisor-only permission, so a system call does not change the page table root, and that mapping is what Meltdown read. KPTI unmaps almost all of the kernel from the user page table. Cost is a page table root write on every entry and exit, and without process context identifiers that write flushes the whole TLB. Call the combined switch and refill 300 cycles. A workload doing 100,000 system calls per second on a 3 GHz core pays $100{,}000 \times 300 = 3 \times 10^{7}$ cycles out of $3 \times 10^{9}$, which is 1 percent, while a database doing 1,000,000 pays 10 percent. Public 2018 reports spanned that range, under 1 percent for compute-bound work up to roughly 30 percent for the most syscall-bound cases, and process context identifiers cut it substantially by tagging TLB entries so the root write does not flush.
### 6.3 Why the industry cared this much
A cloud operator running 100,000 servers that takes an average 5 percent loss has effectively lost 5,000 servers' worth of capacity, which at a few thousand dollars of capital each is tens of millions, plus power and cooling for machines that now do less work per watt. That arithmetic explains the industry's behaviour better than any security argument. It is why operators pushed vendors hard for in-silicon fixes, why a hardware fix costing half a percent is worth a very large amount of engineering compared to a software fix costing five, and why mitigations are exposed as per-process controls rather than being on globally.
---
## Part 7, the design lesson for a microarchitect
### 7.1 The principle
**Any resource shared across a security boundary, whose observable behaviour depends on data, is a channel.**
All three clauses are load bearing. **Shared** means two principals use the same physical structure, not two copies of the same design but the same silicon. **Across a security boundary** means the two are supposed to be isolated, so two threads of one process share everything and that is fine because they were never isolated, while two processes, two privilege levels, two virtual machines, or an enclave and its host are boundaries. **Behaviour depends on data** means some property a principal can measure, almost always timing, varies with values the other handles. Remove any one clause and the channel disappears, which is why the defences in Part 6 sort so neatly. Partitioning removes sharing, tagging removes the crossing, and data-independent timing removes the dependence.
### 7.2 The map
Every structure below is one somebody designed for performance, and every one is a channel.
<Figure src="/figures/hardware-interview-prep/iv-16-Security-Side-Channels-and-Speculation-fig08.svg" alt="Every structure listed here was designed for performance and every one of them straddles the boundary the two principals are supposed to be separated by, which is what makes each a channel." caption="Every structure listed here was designed for performance and every one of them straddles the boundary the two principals are supposed to be separated by, which is what makes each a channel." id="fig:16-Security-Side-Channels-and-Speculation-8" />
The mechanisms differ enough to be worth naming. Caches leak by occupancy, which is all of Part 2. TLBs leak the same way at page granularity, so you learn address bits [47:12] instead of [47:6], which is coarser and still enough to tell which of several code paths ran. The direction predictor leaks branch outcomes, so a victim branch whose direction depends on a secret bit leaks that bit with no cache involvement at all. Execution ports leak the instruction mix through contention, which is what PortSmash demonstrated between SMT siblings. Variable-latency functional units leak operand magnitude, and DRAM row buffers leak the row index, because an access to an already-open row skips the activate.
Prefetchers deserve their own paragraph because the recent case is the interesting one. A stride prefetcher tells you the stride, which is an address-pattern leak like the others. A **data-memory-dependent prefetcher** inspects loaded values, decides whether a value looks like a pointer, and dereferences it, so its behaviour depends on the *contents* of memory rather than the addresses accessed, which is a strictly more powerful channel. The public GoFetch research in 2024 demonstrated this against constant-time cryptographic code on Apple M-series parts, and the point it made is important. Correctly written constant-time code was still leaking, because the prefetcher introduced a data dependence the programmer had no way to see.
The last row closes the loop back to [DVFS Droop and Thermal](/learn/hardware-interview-prep/dvfs-droop-and-thermal). A DVFS controller that moves frequency in response to activity is by definition a shared resource whose behaviour depends on what code is running, and the public Hertzbleed work showed data-dependent power draw modulating frequency in a remotely measurable way. Code that is genuinely constant-time in cycles is not constant-time in wall clock when frequency moves. That result is unsettling in the right way, because the channel is not a storage structure at all.
### 7.3 What this changes about the job
Security joined power, performance, and area as a design-time constraint. Not a review at the end, a question asked when the structure is proposed. **What does this share, across which boundary, and what does an observer learn?** In a design review that expands into a checklist worth carrying.
- Which principals use this structure? Threads, processes, privilege levels, guests, enclaves.
- Is any entry tagged or indexed with something identifying the principal, or is it purely address- or history-indexed?
- Does allocation, replacement, or timing depend on values a principal handles rather than only on addresses?
- What happens to its contents at a context switch, a privilege transition, a virtual machine exit? Nothing, flush, or partition?
- Can a principal drive it to a known state and then observe its state again? That prime-and-probe shape is the tell.
- Is there an architectural control to disable or partition it, documented well enough that software knows when to use it?
Two honest caveats. Adding security to the triad does not soften the other three, because every mitigation in Part 6 is paid in performance, area, or power. And the goal is not zero channels, which is not achievable in a shared machine, since sharing is what makes the machine efficient and a machine with no sharing is several machines. The goal is to make the bandwidth low enough and the setup cost high enough that the attack is not worth mounting for the value of the data, and to give software an architectural control where the threat model demands more.
---
## Part 8, Apple's public security posture
Everything here comes from Apple's published Platform Security guide, its public support documents, ARM architecture documentation, and peer-reviewed papers. Where something is inference rather than a documented statement, it is flagged.
### 8.1 The Secure Enclave
Apple's published documentation describes the Secure Enclave as a dedicated secure subsystem integrated into the SoC and isolated from the main application processor, with its own boot ROM, its own operating system, a memory protection engine that encrypts and authenticates its dedicated region of DRAM, a hardware true random number generator, a dedicated AES engine, and a separate secure storage component used for anti-replay counters.
Read that against Part 7. The architecture does not mitigate the shared-resource problem, it **eliminates** it by not sharing. A separate processor with its own caches, its own predictors, and its own instruction stream has no microarchitectural structure in common with the application processor, so there is nothing for an application-processor attacker to prime, flush, or time. That is the strongest possible answer to Part 7's question and it is affordable only for a small amount of very high value work. The one thing genuinely shared is physical DRAM, and the published design handles that by encrypting and authenticating the enclave's region rather than isolating it.
### 8.2 Pointer authentication
Pointer authentication is an ARMv8.3-A architectural feature. Apple shipped it in the A12 in 2018, early relative to the rest of the ARM ecosystem, and it is a documented part of the arm64e ABI. A 64-bit pointer does not need 64 bits, because implementations use 48 or fewer address bits, so the upper bits are free. PAC computes a message authentication code over the pointer value together with a 64-bit context value, using a per-process, per-boot key held in system registers user code cannot read, and stores the truncated MAC in those free bits.
Work the sizes. With 47 address bits plus a sign bit there are 16 free bits, so a `PACIA` instruction writes a 16-bit signature there. A later `AUTIA` recomputes it and either strips it to yield a clean usable pointer or, on mismatch, produces a pointer guaranteed to fault when dereferenced. An attacker who overwrites a saved return address with an arbitrary value has a $2^{-16}$ chance of the signature happening to be right, about 1 in 65,536. That defeats return-oriented programming as normally practised, because ROP works by overwriting saved return addresses and function pointers with addresses of existing code fragments.
Be honest about the limits, because an interviewer will push. Sixteen bits is not a cryptographic margin, and an attacker with an oracle that reveals whether a guess authenticated can brute-force in about 32,768 attempts on average. The public PACMAN research from MIT in 2022 demonstrated exactly that on an M1, and the oracle it used was a speculative execution channel, because a speculatively executed authentication does not raise an architectural fault but does produce a timing difference. That is the cleanest illustration in this note of why the topic matters, since a speculative side channel was used to attack a hardware security mechanism rather than to read data. PAC also protects only pointers, not data, and substituting one validly signed pointer for another signed with the same context remains possible, which is why the context discriminator matters as much as the signature does.
### 8.3 Memory tagging
ARMv8.5-A introduced the Memory Tagging Extension. Every 16-byte granule carries a 4-bit tag and every pointer carries a 4-bit tag in its upper bits, and on each access the hardware compares them and faults on mismatch. An allocator assigns a fresh tag at allocation and a different one at free, catching use-after-free, and different tags to adjacent allocations, catching linear overflows. Four bits is 16 values, so a blind guess matches one time in 16, which is 6.25 percent. That sounds weak and is not, because an exploit chain typically needs many successful accesses in sequence and in synchronous mode the first mismatch faults immediately. Ten required accesses at 6.25 percent each is about one chance in $10^{12}$.
Apple publicly announced Memory Integrity Enforcement in 2025, described as built on an enhanced form of memory tagging and as always-on for the kernel and key userland processes on its newest silicon. The public description is high level and the specific differences from baseline MTE are not fully documented publicly as far as public sources show, so treat any claim about the internals as speculation and say so rather than guessing.
### 8.4 Kernel integrity, and what Apple said about Spectre and Meltdown
Apple's published security documentation describes Kernel Integrity Protection, in which the kernel's code and read-only data pages are locked after boot by the memory controller so that even code running with kernel privileges cannot modify them. Related mechanisms described publicly include System Coprocessor Integrity Protection, the Page Protection Layer, and more recently the Secure Page Table Monitor. The pattern is the same in each case and it is a hardware pattern. Put the enforcement somewhere the attacker's privilege level cannot reach, which in practice means a different hardware unit rather than a more careful software layer, because a memory controller that refuses writes to a physical range does not care what privilege level issued the write.
In January 2018 Apple published a support document stating that all Mac and iOS devices were affected, that Meltdown mitigations had shipped in iOS 11.2, macOS 10.13.2 and tvOS 11.2, and that Safari mitigations for Spectre were coming. Those Safari mitigations reduced timer resolution, added jitter to `performance.now()`, and disabled `SharedArrayBuffer`. Note which half of 1.4 they attack. They degrade the **receiver**, not the transmitter, which buys difficulty rather than safety and is the right emergency measure while hardware fixes are years away.
Public academic work on Apple silicon has continued. Augury in 2022 examined the M1's data-memory-dependent prefetcher, PACMAN in 2022 attacked pointer authentication, and GoFetch in 2024 showed the data-memory-dependent prefetcher undermining constant-time cryptographic implementations, with Apple's public guidance pointing developers at the data-independent timing control on hardware that supports it. Cite these as published research, which is what they are, and do not imply knowledge of unpublished internals.
---
## Part 10, check yourself
Answer out loud, in full sentences, as if an interviewer asked. If you cannot, reread the section named.
1. Draw the two boxes of machine state, say which one the ISA makes promises about, and explain why the freedom to build the second box however you like is the reason microarchitecture exists. (1.1)
2. List exactly what a mispredict squash restores and exactly what it does not, then state in one sentence the insight that generates every attack here. (1.2, 1.3)
3. Split any one attack into a transmitter and a receiver, and explain why the receiver is nearly always the same while the transmitter differs. What does that say about where defence has to happen? (1.4)
4. You have an L1 hit at 4 cycles and a DRAM access at 200. Explain how that gap becomes a one-bit channel, including how you handle measurement noise. (2.1)
5. Walk FLUSH+RELOAD in three steps, then use it to extract an RSA private exponent bit by bit. Does that attack require any CPU bug at all? (2.2)
6. Why is the probe array 256 slots of 4096 bytes rather than 64? Then derive the leak rate for an eight-bit probe array and a four-bit one and check the four-bit number against the published Meltdown figure. (2.3)
7. When can you not use FLUSH+RELOAD, and what do you use instead? Work the set geometry for a 32 KB 8-way L1 with 64-byte lines and say what address bits you learn. (2.4)
8. Walk the Spectre v1 timeline from the mistrained branch to the surviving footprint, then say why the squash does not remove it. Three reasons, not one. (3.3, 3.4)
9. Explain branch target injection without using the word Spectre. Who executes the gadget, whose privileges does it run with, and how many instructions does the attacker execute inside the victim? (3.5)
10. Why can you not fix Spectre v1 by not speculating? Work the CPI arithmetic with a 0.5 base CPI, 20 percent branches, and a 15-cycle penalty. (3.6)
11. Explain why an out-of-order machine is forced to defer faults to retirement, then show where the Meltdown window sits inside that. (4.1, 4.2)
12. Argue that Meltdown was an implementation bug and not an inherent cost of out-of-order execution. Give the two cheap correct implementations and the strongest evidence for your claim. (4.4)
13. Explain speculative store bypass starting from why a memory dependence predictor exists at all, describe the sandbox pattern that made it exploitable, and say whether it is a bug or inherent. (5.3)
14. Sort Meltdown, MDS, Spectre v1, and Spectre v4 into design bugs and inherent problems, and state the rule that decides which is which. (5.5)
15. Name four hardware and four software mitigations with the cost of each in the units it is actually paid in, including the cache partitioning arithmetic. (6.1, 6.2)
16. State the general principle about shared resources in one sentence, unpack all three clauses, then name six structures you would build as a microarchitect and what each leaks. (7.1, 7.2)
17. Explain pointer authentication including the bit arithmetic, then explain its brute-force limit and how public research used a speculative channel as the oracle. (8.2)
---
## Part 11, related notes
- [Out of Order Execution](/learn/hardware-interview-prep/out-of-order-execution) for the reorder buffer, the rename checkpoint, and precisely what recovery restores, which is the foundation of Part 1
- [Front End and Branch Prediction](/learn/hardware-interview-prep/front-end-and-branch-prediction) for the direction predictor, the BTB, and the return address stack, mistrained in Part 3 and steered by the retpoline in 6.2
- [Load Store and Memory Ordering](/learn/hardware-interview-prep/load-store-and-memory-ordering) for the store queue and the memory dependence predictor that 5.3 attacks
- [Cache Organization and Prefetching](/learn/hardware-interview-prep/cache-organization-and-prefetching) for sets, ways, latencies, and prefetcher behaviour, the entire measurement apparatus in Part 2
- [CPU Foundations Pipeline and Hazards](/learn/hardware-interview-prep/cpu-foundations-pipeline-and-hazards) for precise exceptions, which is why the Meltdown window exists at all
- [SRAM Arrays and ECC](/learn/hardware-interview-prep/sram-arrays-and-ecc) for fill buffers and array read timing, where the MDS data in 5.1 lives
- [Virtual Memory and Memory Ordering](/learn/hardware-interview-prep/virtual-memory-and-memory-ordering) for page tables and permission bits, which Meltdown and Foreshadow abuse
- [Arithmetic Hardware](/learn/hardware-interview-prep/arithmetic-hardware) for early-terminating multipliers and variable-latency dividers, timing channels created by ordinary PPA optimization
- [DVFS Droop and Thermal](/learn/hardware-interview-prep/dvfs-droop-and-thermal) for the frequency and voltage loop that 7.2's last row turns into a channel
- [Verification Methodology](/learn/hardware-interview-prep/verification-methodology) for where formal fits, the strongest connection between this topic and your own background
- *Apple Context and Behavioral* for how to discuss Part 8 without overclaiming knowledge of internals
- [TLBs and Address Translation](/learn/computer-architecture/tlbs) for the vault's treatment of KPTI and its measured TLB costBook mode
Was this helpful?