Part V·Advanced ILP and Out-of-Order Execution·Chapter 59 of 66

Part VAdvanced ILP and Out-of-Order Execution

Simultaneous Multithreading (SMT)

August 3, 2026·32 min read·advanced

The preceding chapters of Part V built an out-of-order execution engine that exploits the instruction-level parallelism inside a single thread. Chapter 50 introduced renaming and dynamic scheduling. Chapter 51 factored renaming into a distinct pipeline stage with a physical register file. Chapter 52 added precise commit. Chapter 53 analyzed the issue scheduler. Chapter 54 layered the memory ordering machinery. Chapter 55 and Chapter 56 sized the speculative horizon. Chapter 57 pushed the fetch and decode throughput up to match the rename width. The result is a core that can have a hundred or more instructions in flight, dozens of physical registers renamed, and an issue width of four to six operations every cycle.

Most workloads cannot keep that backend busy. A four-wide rename slot can be filled four times per cycle in principle. A spec runs on a modern desktop core typically averages between one and two instructions retired per cycle, even after every ILP technique studied so far. The gap between peak rename width and actual retire rate is the symptom that motivates this chapter. Issue slots sit idle because the in-flight window of one thread runs out of independent work. The fixes inside a single thread, such as wider issue, deeper reorder buffers, and better prediction, hit diminishing returns. SMT proposes a different fix. If one thread cannot keep the backend full, feed it instructions from another thread.

The result is a core that runs two or more software threads at the same time, sharing the execution units, the issue queues, the load-store queue, and most of the front end. Each thread keeps its own program counter, its own architectural register state, and its own retirement order. The hardware tags every in-flight instruction with a thread identifier, so the renaming map, the reorder buffer entries, and the load-store queue entries know which thread they belong to. The chapter walks through how this is done, what design choices the architect faces, where SMT helps and where it hurts, and how IBM POWER, Intel Hyper-Threading, AMD Zen, and the Apple M-series cores have answered those questions differently.

01.From One Thread to Many

The motivation for SMT is concrete. The table below shows backend utilization data of the kind reported by Tullsen and colleagues in their original SMT paper [1]. A four-wide superscalar core fed by a single thread fills about one and a half of its four issue slots per cycle on integer codes, and slightly more on floating-point codes. The other two and a half slots stand empty because the thread has no independent instruction ready to issue. The empty slots have a name. Tullsen distinguished between horizontal waste, in which a cycle issues fewer than the maximum number of instructions, and vertical waste, in which a cycle issues zero instructions. SMT addresses horizontal waste directly, by filling unused issue slots with instructions from another thread, and it addresses vertical waste partially, by giving the scheduler more candidate threads to draw from when one thread stalls completely.

Table 1. Backend utilization in a four-wide superscalar core

Workload type	Average IPC	Issue slot use	Wasted slots
Integer single-thread	1.4	35%	65%
Floating-point single-thread	1.8	45%	55%
Mixed two-thread SMT	2.6	65%	35%

Source: representative values reproduced from the canonical SMT measurement studies cited in Hennessy and Patterson 2019 [1]. Exact numbers vary by benchmark and microarchitecture.

The route from a single-threaded core to an SMT core is not a small patch. Several pipeline structures need redesign. The architectural register file has to be duplicated, one image per thread. The front-end fetch logic has to be able to switch between thread program counters. The branch predictor tables either need to be tagged with the thread identifier or partitioned so that one thread’s history does not pollute the other’s prediction. The reorder buffer, the issue queue, the load-store queue, and the physical register file all become shared structures whose entries belong to whichever thread allocated them. Retirement is now per-thread, since each thread has its own commit order. Exceptions and interrupts have to be steered to the right thread. The TLB needs an address-space identifier or a thread tag to keep one thread’s translations from masquerading as another’s.

The first commercial SMT core was the Alpha 21464, which Compaq canceled before silicon. The first shipping SMT core was Intel’s Pentium 4 with Hyper-Threading in 2002. IBM POWER5 followed in 2004 with SMT2. POWER7 in 2010 widened to SMT4. POWER8 in 2014 widened again to SMT8. AMD’s Zen family adopted SMT2 in 2017. Apple’s M-series cores have so far chosen not to implement SMT, on the argument that the area is better spent on additional independent cores. Sun’s Niagara family, sometimes described as SMT, is closer to fine-grained barrel multithreading, which is studied in the next section as a separate point in the design space.

02.Three Multithreading Styles

SMT is the most aggressive of three multithreading styles that have all shipped in commercial silicon. The other two are coarse-grained multithreading and fine-grained multithreading. The three differ in how often the hardware switches between threads and in what the backend looks like at any one instant.

Definition: Three multithreading styles

Coarse-grained multithreading switches the active thread on a long-latency stall, typically a last-level cache miss or a TLB miss. While the active thread runs, the backend looks like a normal single-threaded core. When the stall hits, the pipeline drains, the active thread context is saved, and a different thread context is loaded in. Switch cost is tens of cycles.

Fine-grained multithreading interleaves threads at the cycle granularity. Each cycle’s fetch and issue are dedicated to one thread, but a different thread is chosen every cycle. The classic example is the Sun Niagara T1, which round-robins between four threads per core. The backend at any one cycle is single-threaded.

Simultaneous multithreading interleaves threads inside one cycle. Each issue cycle can launch operations from multiple threads into the execution units. The backend at any one cycle is genuinely multithreaded.

The table below compares the three on the dimensions the architect cares about. Coarse-grained multithreading is the cheapest to implement because the backend never sees two threads at once. Fine-grained multithreading hides short-latency stalls (a two-cycle bubble inside one thread is covered by another thread’s issue slot) but does not improve issue-slot utilization in single-thread-heavy phases. SMT covers both kinds of waste at the cost of substantial backend redesign.

Table 2. Multithreading style tradeoffs

Property	Coarse-grained	Fine-grained	SMT
Switch granularity	Stall event	Cycle	None (concurrent)
Backend at one cycle	1 thread	1 thread	N threads
Hides long-latency stalls	Yes	Yes	Yes
Hides short pipe bubbles	No	Yes	Yes
Fills horizontal waste	No	No	Yes
Backend redesign effort	Low	Medium	High

The chapter from this point forward focuses on SMT. Coarse-grained and fine-grained multithreading appear in embedded and specialized parts and remain pedagogically useful, but the design pressure on modern out-of-order cores is fully on SMT.

03.Hardware Thread State

Each hardware thread on an SMT core needs a private copy of every piece of architectural state the ISA defines. For a 64-bit RISC-V core implementing the base integer and floating-point extensions, this is the 32 integer registers, the 32 floating-point registers, the program counter, the floating-point control and status register fcsr, and the supervisor mode CSRs (sstatus, sepc, stvec, satp). For an x86-64 core, the per-thread state is much larger: the 16 general-purpose registers, the 16 SSE/AVX vector registers, the 8 x87 stack registers, MXCSR, the segment registers, the model-specific registers visible to user code, the FS and GS base registers, the debug registers, and the various extended-state save areas.

The clean way to draw this is to give each thread its own architectural register file and let the rename machinery sit downstream of them, sharing a single physical register file between the threads. The rename map is per-thread because each thread’s logical-to-physical translation is independent. The physical pool is shared, with each entry tagged by the thread that owns it.

Table 3. Per-thread state on a 64-bit RISC-V SMT core

State element	Width per thread
Integer architectural registers	32 $\times$ 64 = 2048 bits
Floating-point architectural registers	32 $\times$ 64 = 2048 bits
Program counter `pc`	64 bits
`fcsr` (FP CSR)	32 bits
Supervisor CSRs (`sstatus`, `sepc`, $\dots$ )	$\approx$ 512 bits
Branch predictor history bits	64 to 256 bits (shared or split)

The branch-history state lives in a gray area. On some SMT designs the history register is replicated per thread. On others it is shared and tagged with the thread identifier. The choice affects predictor accuracy when threads have very different control-flow signatures. Per-thread history avoids interference. Shared history with tagging recovers a little area at the cost of a small number of mispredictions. Chapter 56 covered the predictor structures in depth.

Each thread also needs its own retirement order. The reorder buffer is shared, but the head pointer and the commit logic operate per-thread. Thread 0’s retirement decisions cannot depend on thread 1’s reorder buffer head, since the two threads are independent streams. The retirement logic walks the reorder buffer entries tagged with the corresponding thread identifier and commits them in program order for that thread, while leaving entries from the other thread untouched.

04.Resource Partitioning

The central design question in SMT is how to divide each shared microarchitectural structure between the active threads. Three canonical policies have shipped, and most cores mix them. The shared structures of interest are the instruction queue, the reorder buffer, the load-store queue, the physical register file, the store buffer, the branch order buffer, and the rename map free list. Each one is a fixed-size table with a fixed entry budget per cycle.

Static partitioning

The simplest policy is static partitioning. A 192-entry reorder buffer on a two-thread SMT core is split exactly in half. Thread 0 gets entries 0 to 95. Thread 1 gets entries 96 to 191. The rename stage looks up the current thread, finds its half of the ROB, and allocates within it. If thread 1 only needs 30 entries, the other 66 entries in its half are unused even when thread 0 has work for them.

Intel’s original Hyper-Threading on the Pentium 4 used static partitioning for the reorder buffer, the store buffer, the load buffer, and several other structures. The policy made the design verifiable: the worst-case behavior of one thread cannot affect the other thread’s allocation, so the two threads can be reasoned about almost independently.

The downside is throughput. If one thread is in a high-IPC phase and the other is stalled on a cache miss for a hundred cycles, the stalled thread’s reorder-buffer slots sit empty while the active thread would happily use them. Static partitioning underutilizes the shared structure exactly when the workload is most asymmetric.

Fully shared use

The opposite policy is to leave every shared structure fully shared. Either thread can allocate any free entry. The rename stage simply takes the next free entry from a single shared free list, tags it with the thread identifier, and proceeds. If one thread is much busier than the other, it grabs the lion’s share of the entries.

The danger is starvation. A single thread that is making no forward progress but holds a hundred reorder-buffer entries on speculative loads waiting on DRAM will starve a second thread of any ROB slot to allocate. The blocked thread cannot fetch, decode, rename, or issue. The two-thread core ends up with worse throughput than a single-thread core would have had.

Dynamic partitioning with quotas

The shipping compromise on modern cores is dynamic partitioning with quotas. Each thread gets a minimum guaranteed share of the structure and a maximum cap. Between those two limits, the thread allocates from a shared pool. The rename stage checks each thread’s current occupancy and either allocates, blocks, or steers to a backup queue based on whether the thread is under its guaranteed minimum, between minimum and cap, or at its cap.

The table below summarizes the three policies and where they tend to land in shipping cores.

Table 4. Partition policies and typical use

Policy	Maximum throughput	Starvation risk
Static	Lowest	None (proof by construction)
Fully shared	Highest in theory	High in practice
Dynamic + quota	Near maximum	Bounded by minimum quota

The quota knob is exposed differently across vendors. POWER8 lets the operating system set per-priority shares through software-visible SMT control bits. Intel Hyper-Threading and AMD SMT do not expose the quota directly to software but document the general policy in the optimization guides [2][3].

Worked example: ROB allocation

To make the three policies concrete, walk through a 192-entry reorder buffer on a two-thread SMT core. Thread A is running a high-IPC integer kernel. Thread B is stalled on a last-level cache miss for 200 cycles. Over those 200 cycles, thread A wants to keep the reorder buffer as full as possible.

Under static partitioning, thread A gets 96 ROB entries. The other 96 sit empty for thread B’s stalled load. Effective ROB depth for A is 96.

Under fully shared use, thread A can take all 192 entries as long as thread B’s reorder-buffer occupancy is zero. But thread B has in-flight loads waiting on DRAM, so its occupancy is not zero. Say B holds 64 entries. Thread A gets 128. The reorder buffer is full.

Under dynamic partitioning with a 16-entry minimum quota for B, thread A gets up to $192 - 16 = 176$ entries, since the minimum quota reserves 16 for B. If B currently holds 8 entries, B is under-quota, so B gets allocated next on its next decoded instruction. If B holds 64 entries, well above the minimum, A and B compete from the same free list. The quota only kicks in to prevent starvation, not to slice the structure evenly.

The dynamic policy gives A the bulk of the ROB during B’s stall, without ever letting A grow to the point where B is locked out of making forward progress when its load eventually returns.

05.The SMT Scheduler

The fetch stage is where the multithreading scheduler lives. Each cycle the fetch unit picks one thread (or sometimes two) to fetch from. The selection policy materially affects the throughput of the core. The three policies that have been studied at length are round-robin, prefer-empty, and ICOUNT.

Round-robin

The simplest scheduler alternates threads cycle by cycle. On a two-thread core, even cycles fetch from thread 0 and odd cycles fetch from thread 1. The implementation is one bit of state. The weakness is that a thread stalled on an instruction-cache miss still gets its turn, which wastes fetch bandwidth.

Prefer-empty

A small refinement is to prefer the thread whose decoded-instruction buffer is least full. The intuition is that the front end should work harder for the thread that has fewer instructions waiting in its decode buffer. This avoids fetching for a thread whose decode queue is already saturated.

ICOUNT

The most influential scheduling proposal in the SMT literature is ICOUNT, named for "instruction count", from Tullsen, Eggers, and Emer’s 1996 follow-up paper [1]. ICOUNT prefers the thread with the fewest instructions in the decode, rename, and issue queues combined. The intuition extends prefer-empty from the decode buffer to the whole front end. A thread already behind in the front end will only clog the shared back end if it is allowed to fetch more, so the scheduler preferentially fetches for the thread that is moving fastest through the back end.

ICOUNT measured roughly twice the throughput of round-robin on the original SMT benchmarks. Most modern SMT designs use a policy that resembles ICOUNT with additional heuristics for branch misprediction recovery and instruction-cache miss avoidance.

A separate question is whether to fetch from two threads in the same cycle. The 2.8-2.8 ICOUNT scheme proposed by Tullsen fetches from up to two threads per cycle, with up to 8 instructions split between them. In practice this requires duplicate fetch ports, which is expensive in the instruction cache. Most shipping cores fetch from one thread per cycle but pick that thread aggressively.

06.Pipeline View of an SMT Core

The pipeline of an SMT core differs from a single-thread out-of-order core only at well-defined stages. Figure 1 illustrates where the thread identifier flows through the stages.

Figure 1. SMT pipeline. The fetch stage maintains one program counter per thread and selects one each cycle via the scheduler. Decode attaches the thread identifier to each microoperation. Rename uses a per-thread map but allocates from a shared physical register file. Issue and execute see a mixed stream tagged by thread. Commit operates per-thread, draining the reorder buffer in each thread’s program order independently.

Fetch carries one program counter per thread. The scheduler in the preceding section picks the thread of the cycle. The fetched cache line goes into a thread-tagged decode queue.

Decode attaches the thread identifier to each microoperation it emits. From this point onward, every in-flight instruction carries its thread tag through the pipeline.

Rename consults the renaming map of the current thread. The physical register pool is shared. When rename allocates a physical register, it tags the rename-map entry and the ROB entry with the thread identifier.

Issue draws ready microoperations from the shared issue queue. The operand readiness check is identical to the single-thread case because operand readiness depends only on the physical register state, which is shared across threads. The issue queue’s wakeup logic does not need to know about threads.

Execute is fully shared. The ALU lanes, the load-store units, the floating-point pipes, and the vector lanes all execute microoperations from whichever thread reached them first. The thread identifier rides along with each microoperation so that the result can be steered to the correct thread’s reorder buffer entry at writeback and so that a pipeline flush can be scoped to one thread. Wakeup itself still matches on the physical register tag alone.

Commit operates per thread. The retirement logic walks each thread’s reorder buffer in that thread’s program order and retires the head when it is complete. A single core may retire instructions from both threads in the same cycle, up to the shared retire width.

07.Fairness Versus Throughput

The architect of an SMT core faces a tension between two objectives. The first is to maximize aggregate throughput, defined as the total instructions retired per cycle summed across all hardware threads. The second is to ensure that each thread receives a fair share of the core’s resources, so that no thread is starved and the latency of any individual thread does not degrade beyond a tolerated bound.

The two objectives often conflict. The throughput-maximizing policy on a workload pair where one thread is high-IPC and one thread is memory-bound is to give the high-IPC thread all the issue width and reorder buffer it wants. But that policy starves the memory-bound thread. The fairness-maximizing policy is to give each thread half of every shared structure regardless of demand, which is exactly the static partition policy that loses throughput when one thread is stalled.

The metric that captures the tradeoff is harmonic mean speedup. For two threads with single-thread IPCs $I_1$ and $I_2$ and SMT-mode IPCs $I_1'$ and $I_2'$ , the harmonic mean speedup is

$\mathrm{HMS} = \frac{2}{\dfrac{I_1}{I_1'} + \dfrac{I_2}{I_2'}}.$

Maximizing harmonic mean speedup is roughly equivalent to maximizing throughput while keeping per-thread slowdown bounded. A throughput-only metric would let one thread run at $0.9$ of its single-thread IPC while the other runs at $0.05$ , and report a respectable aggregate. The harmonic mean penalizes that case heavily because the second fraction dominates.

A second consideration is real-time and priority work. An operating system may want to elevate one hardware thread over another. The ISA exposes a priority hint on some platforms. POWER8’s or 1,1,1 no-op-with-priority instruction lowers the issuing thread’s SMT priority. POWER8’s or 6,6,6 raises it. The hardware fetch scheduler weights its decisions by these software-set priorities. Intel and AMD do not expose a comparable mechanism in user mode, although the kernel can disable hyperthreads on a per-core basis.

08.SMT-2, SMT-4, SMT-8

The number of hardware threads per core is a primary design choice. Three points in the space have shipped commercially. SMT-2 is the choice of Intel x86-64 server lines through Sapphire Rapids and the mainline Xeon E-series, AMD Zen, and many embedded cores. SMT-4 is the choice of IBM POWER7 (2010). SMT-8 is the choice of IBM POWER8 (2014), POWER9 (2017), and POWER10 (2021). The baseline case, SMT-1, means no SMT at all, and it is where Apple’s M-series cores, ARM’s Cortex-X line, most RISC-V application cores, and Fujitsu’s A64FX (2018) sit.

Why SMT-2 dominates client and most server x86

SMT-2 is the default for two reasons. First, the marginal throughput gain from going beyond two threads is small on most desktop and client workloads, because the bottleneck shifts from issue-slot utilization to memory bandwidth. Two threads typically recover 70-80% of the available horizontal waste. A third thread recovers another 5-10%. Beyond that, the gains diminish.

Second, SMT-2 has a manageable area cost. The duplicated architectural register file is one extra copy. The thread tags through the pipeline are one extra bit per entry in every shared structure. Going to SMT-4 doubles the architectural register file again, widens the thread tag to two bits, and forces a redesign of the static-partition boundaries on every shared structure that uses them.

Table 5. SMT degree on representative shipped cores

Core / Family	SMT degree	Reference
Intel Pentium 4 HT (2002)	2	[2]
Intel Sandy Bridge (2011)	2	[2]
Intel Sapphire Rapids (2023)	2	[2]
AMD Zen 4 (2022)	2	[3]
ARM Cortex-X3 (2022)	1	[4]
Apple M3 (2023)	1	Public disclosures
IBM POWER7 (2010)	4	[1]
IBM POWER8 (2014)	8	[1]
IBM POWER10 (2021)	8	[1]
SiFive Bullet (RISC-V)	1	Public disclosures

Public sources only. SMT degree per the cited manuals and HotChips disclosures.

Why POWER goes to SMT-8

IBM POWER targets transaction-processing and database workloads that have very high latent thread-level parallelism and tolerate moderate per-thread slowdown well. A database server frequently runs hundreds or thousands of concurrent connections, each largely independent. SMT-8 turns one physical POWER core into eight visible processors for the operating system. The aggregate throughput on DB and OLTP benchmarks is substantially higher than what eight single-threaded cores would have delivered at the same area budget, because the eight virtual processors share the very large POWER caches and execution units. The cost is that the per-thread throughput on a CPU-bound single thread is roughly halved compared with single-threaded mode.

The POWER ISA recognizes this and provides a prtm (priority to me) and prsm (priority shared) mechanism, plus a mode register that lets software ask the hardware to run the core in single-thread mode, two-thread mode, four-thread mode, or eight-thread mode. A latency-sensitive workload can lower the SMT degree at runtime. A throughput workload can raise it.

Why Apple chose SMT-1

Apple’s M-series cores ship without SMT. The publicly disclosed reasoning is that the area required for the duplicated architectural state, the wider tag bits through the shared structures, and the fairness logic is better spent on an additional independent core. Apple’s Firestorm and Avalanche cores have very wide rename (8-wide) and very large reorder buffers (about 630 entries on Firestorm), which already extracts much of the available ILP from a single thread. Going from 8 cores to 12 cores buys more aggregate throughput on Apple’s target workloads than going from SMT-1 to SMT-2 on 8 cores would have. The exact tradeoff is platform-specific and not a universal answer.

An x86-64 SMT core has an additional sharing question. Decode is followed by a microcode ROM and a microoperation cache. Both are expensive structures. The microoperation cache (the "uop cache" in Intel terminology) on Sandy Bridge stores 1.5K decoded microoperations. Sharing it between two SMT threads is more intricate than sharing the integer register file.

The uop cache is content-addressed by physical instruction address. Two threads executing different code never collide. Two threads executing the same code (for example, two threads of the same process running the same function) can share the same uop-cache entries with no penalty, since the entries are read-only. The cache is statically partitioned on Sandy Bridge (each thread gets half), shared with thread tags on Haswell and later. Intel’s optimization manual gives the per-microarchitecture detail [2].

The microcode ROM is read-only and shared without partitioning, since it is a fixed lookup table.

10.Memory Ordering and SMT

Memory ordering on an SMT core is one place where the design choices become subtle. The load-store queue is studied in Chapter 54. On an SMT core, the load-store queue becomes shared, and a fresh problem appears: the memory model has to be enforced not just within a thread, but between threads on the same core.

Two threads on the same SMT core share the L1 data cache. A store from thread A writes the L1 cache. A subsequent load from thread B on the same core sees the new value with no inter-core coherence traffic. This is faster than the inter-core path because the cache line never leaves L1. It is also a tighter coupling between the two threads than the inter-core coherence protocol would provide. The memory model has to define when one thread’s store becomes visible to the other thread.

The x86-64 memory model treats SMT siblings on the same core as indistinguishable from external observers for the purpose of total store order. A store from thread A is observable to thread B at the same point in the global order as it would be to any other external agent. The implementation makes thread A’s store visible to thread B only when it drains from A’s store buffer into the L1 data cache, which is the same point at which it becomes visible to any other agent. Store-to-load forwarding out of the store buffer stays private to the storing thread.

The RISC-V memory model (RVWMO) is weaker than x86-64 TSO, and SMT siblings observe each other’s stores under the same ordering rules as separate harts on the same chip. The principle is the same: the implementation has to enforce the architectural memory model regardless of the physical sharing.

11.When SMT Helps and When It Hurts

The empirical record on SMT is mixed because the benefit depends strongly on the workload. The cases where SMT helps are clear. The cases where SMT hurts are equally clear.

SMT helps

A pair of threads where one is memory-bound and the other is compute-bound is the textbook case. The memory-bound thread spends most cycles waiting on a cache miss. Its issue slots are empty. The compute-bound thread fills them. Total throughput rises to nearly the sum of the two threads’ single-thread IPCs, since they barely contend for the same execution units.

Two database threads running the same query plan on different partitions share instruction-cache footprint and branch-predictor history, which improves predictor accuracy for both. The threads also tolerate each other’s stalls well because their working sets are small.

Web-server workloads with hundreds of independent connections benefit from SMT because the per-request work is short and the miss rate per request is high. The aggregate throughput on SPEC web benchmarks improves by 25-40% with SMT-2.

SMT hurts

Two threads that both have high single-thread IPC and large working sets are the worst case. Each thread’s compute demand exceeds the issue width of a single thread already. Putting both on one core forces them to share execution units they cannot share, doubles the L1 cache pressure, and cuts per-thread cache capacity in half. Aggregate throughput typically rises by less than 10%, and per-thread performance falls by 30% or more.

Highly tuned SIMD kernels (matrix multiply, FFT, dense linear algebra) often see no benefit from SMT and sometimes a small regression. The kernels already saturate the issue width and the floating-point pipelines. SMT cannot find idle slots to fill, and the shared L1 cache becomes the bottleneck.

Some HPC sites disable SMT in the BIOS for exactly this reason. The decision is workload-specific and should be measured rather than assumed.

12.Security Considerations

SMT introduces inter-thread microarchitectural side channels because the two threads share so much state. The disclosures of Spectre, Meltdown, Foreshadow, MDS, and RIDL between 2018 and 2020 made these channels public. The vulnerabilities exploited shared branch predictors, shared L1 data caches, shared line-fill buffers, and shared store buffers between SMT siblings.

The defenses fall into three classes. Software-level defenses arrange the OS scheduler to avoid co-locating threads from different security domains on the same physical core. Microcode-level defenses partition some of the shared structures when the two threads belong to different security domains. Architectural defenses introduce ISA-level support for clearing shared microarchitectural state at well-defined boundaries. The detailed treatment lives in Part IX, in Chapter 107 (Side-Channel Attacks) and Chapter 108 (Transient Execution Attacks). For the current chapter the point is that the very sharing that gives SMT its throughput benefit also opens new attack surfaces.

13.Worked Examples

Example: Static-partition ROB sizing

A core has a 256-entry shared reorder buffer and supports SMT-2. The architect chooses static partitioning to simplify the implementation. (a) How many ROB entries does each thread receive? (b) If thread A’s average ROB occupancy in single-thread mode is 180 entries and thread B’s is 60 entries, which thread is more constrained by the static partition?

Solution. (a) Each thread receives $256 / 2 = 128$ entries. (b) Thread A is constrained because its single-thread average occupancy of 180 entries exceeds its partition of 128. Thread B’s average of 60 fits within its partition with 68 entries to spare. Static partitioning costs thread A a meaningful reduction in in-flight window. A dynamic partition with a 32-entry minimum quota for B would let A use up to 224 entries, recovering most of the lost capacity.

Example: Harmonic mean speedup

Two threads run on an SMT-2 core. In single-thread mode, thread 1 achieves 1.6 IPC and thread 2 achieves 2.0 IPC. In SMT mode they achieve 1.2 IPC and 1.4 IPC respectively. Compute the harmonic mean speedup and compare it to the arithmetic mean speedup.

Solution. $\mathrm{HMS} = \frac{2}{\dfrac{1.6}{1.2} + \dfrac{2.0}{1.4}} = \frac{2}{1.333 + 1.429} = \frac{2}{2.762} = 0.724.$ The harmonic mean speedup is $0.724$ , which means each thread loses about 28% of its single-thread performance on average. The arithmetic mean speedup would be $(1.2/1.6 + 1.4/2.0)/2 = (0.75 + 0.70)/2 = 0.725$ , which is similar in this balanced case. The two metrics diverge when one thread loses far more than the other.

14.Exercises

Exercises

Resource list. List five microarchitectural structures that must be either duplicated, tagged, or partitioned to add SMT-2 support to a single-threaded out-of-order core. For each structure, state which of the three treatments (duplicate, tag, partition) is the appropriate choice and why.
Static versus dynamic partition. A two-thread SMT core has a 96-entry issue queue. Workload pair W1 has one thread averaging 50 entries occupancy and the other averaging 20 entries. Workload pair W2 has one thread averaging 80 entries and the other averaging 10 entries. Compute the issue-queue utilization under static partitioning and under fully shared use for both pairs. Comment on which policy each pair prefers.
ICOUNT pathology. Construct a workload pair where ICOUNT performs worse than round-robin. Describe the pair and explain why ICOUNT misbehaves.
Per-thread state on x86-64. List the per-thread architectural state on a modern x86-64 SMT core implementing AVX-512. Compute the total per-thread storage in bits.
Fairness metric. Three threads on an SMT-4 core (one of the threads is idle) report single-thread IPCs of 2.0, 1.5, and 1.0, and SMT-mode IPCs of 1.6, 1.0, and 0.6 respectively. Compute the harmonic mean speedup across the three active threads. Compute the arithmetic mean. Which metric better captures the cost of the worst-served thread?
SMT degree tradeoff. A core can be built as either SMT-2 with 4-wide rename or SMT-1 with 8-wide rename. Both options have the same total area. Identify two workloads where the SMT-2 option wins and two workloads where the SMT-1 option wins. Justify each.
Memory ordering. Thread A on an SMT-2 x86-64 core executes MOV [X], 1 followed by MOV EAX, [Y]. Thread B on the same core executes MOV [Y], 1 followed by MOV EBX, [X]. Both X and Y start at 0. Can the result EAX = 0 and EBX = 0 occur? Explain in terms of the x86-64 memory model and the SMT load-store queue sharing rules.
POWER priority hints. The POWER ISA’s priority no-op instructions raise or lower the issuing thread’s SMT priority. Describe a programming pattern in which a critical section’s start would lower its own priority and the end would raise it back. What is the use case?
Apple’s SMT-1 choice. Apple’s M-series cores ship as SMT-1. Identify two architectural features of those cores that compensate for the absence of SMT. For each feature, explain how it recovers throughput that SMT would otherwise have contributed.
Workload measurement. A benchmark suite runs on an SMT-2 server and reports a 1.4 $\times$ aggregate throughput improvement over SMT-off. Compute the per-thread slowdown assuming the two threads share the workload equally. Was SMT a win on this benchmark?

References

[1]Hennessy, John L. and Patterson, David A. (2019). “Computer Architecture: A Quantitative Approach.” Morgan Kaufmann.
[2](2024). “Intel.”
[3](2024). “AMD64.”
[4](2024). “ARM.”

Book mode