Case Study --- ARM Cortex-X3 and Neoverse N2

The preceding chapter studied BOOM, an open-source RISC-V out-of-order core whose full RTL is public. BOOM’s value as a case study is that every design decision is visible in the Chisel source. The cost is that BOOM, as an academic effort, has not received the multi-decade investment in branch prediction tuning, prefetcher co-design, and cache hierarchy parameter tuning that the largest commercial cores have. To see the upper end of out-of-order execution, the case study has to turn to industry parts.

This chapter studies two ARM cores published with substantial public detail: Cortex-X3, ARM’s flagship mobile big core for the 2022 Total Compute platform, and Neoverse N2, ARM’s flagship server core announced in 2021. Both cores implement the ARMv9-A architecture, both are documented in public Technical Reference Manuals, both have been presented at HotChips with annotated pipeline diagrams, and both ship in commercial silicon at scale. Together they show how ARM’s microarchitecture team has executed the abstract OoO model of Part V at two different points in the performance-power-area design space.

The sources for this chapter are strictly public: the ARM Architecture Reference Manual [1], the per-core Technical Reference Manuals (Cortex-X3 TRM, Neoverse N2 TRM), HotChips disclosures from 2021-2023, ARM’s developer blog posts, and peer-reviewed measurement studies that cite those documents. No proprietary microarchitecture details from any vendor are introduced.

01.Two Points in the Design Space

The mobile and the server cores share an ISA but optimize for different objectives. The Cortex-X3 targets a single thread’s peak performance within a strict thermal envelope, the kind a smartphone has to live in. The Neoverse N2 targets sustained throughput across many threads in a server, where per-thread peak matters less than per-rack throughput per watt.

The table below gives a one-page summary of the two cores side by side. The remaining sections of the chapter unpack each row.

The Cortex-X3 is wider in nearly every dimension: 6-wide rename versus 5-wide on N2, twice the reorder buffer depth, and one more load pipe. The widening costs power and area, both of which the mobile core spends generously on burst single-thread performance during the few seconds a user is waiting for an app to launch or a page to render. The N2 is narrower because its design budget is better spent on more cores per cluster (the N2 cluster scales to 128 cores per chip) and on lower power per core to sustain that many cores within a thermal envelope.

Neither core supports simultaneous multithreading (SMT-1 across the whole ARM product line). The reasoning, discussed in Chapter 59, is that ARM’s design philosophy spends the SMT area budget on more independent cores instead. The N2 cluster at 128 cores per chip versus an x86-64 server part at 64 SMT-2 cores per chip presents two different views of how to extract throughput from a fixed silicon area.

02.Cortex-X3: The Mobile Flagship

The Cortex-X3 is the heir to a long lineage. Cortex-A57 (2013), Cortex-A72 (2015), Cortex-A75 (2017), Cortex-A76 (2018), Cortex-A77 (2019), Cortex-X1 (2020), Cortex-X2 (2021), and Cortex-X3 (2022) form the unbroken sequence of ARM’s high-performance big cores. Each generation added rename width, reorder buffer capacity, or pipeline parallelism over the previous.

Pipeline overview

Figure 1 sketches the X3 pipeline at the stage granularity disclosed in the TRM.

The 13-stage depth on the integer path is comparable to other modern wide-issue cores. The deeper pipeline lets X3 hit higher clock frequencies. The misprediction penalty is on the order of 13 to 15 cycles, which is the cost of the deeper pipeline traded for the higher frequency.

Front end

The Cortex-X3’s front end fetches up to 8 instructions per cycle from a 64 KiB, 4-way set-associative L1 instruction cache. The predicted-taken path uses a multi-level branch target buffer arrangement: a small fast L1 BTB for hot branches, a larger but slower L2 BTB for cold branches, and an L3 BTB for the largest working set. The direction predictor is a multi-level TAGE variant covered in Chapter 56.

Two decode stages crack ARM macro-operations into uops. AArch64 is closer to a uniform 32-bit RISC encoding than x86-64’s variable-length CISC, so most instructions decode 1-to-1. The exceptions are SIMD instructions that decompose into 2-4 uops (e.g., a 128-bit SIMD load-pair instruction is two uops), and a handful of load-store-multiple compound instructions that pre-date AArch64 and are retained for compatibility.

Rename and dispatch

The X3 renames 6 uops per cycle. The physical register file is split into integer and SIMD/FP files, with separate rename maps per domain. Dispatch routes the renamed uops into the matching issue queue at 8 per cycle (note the dispatch width exceeds rename width because some macro-uops dispatch multiple times).

The reorder buffer is 320 entries, one of the largest in any shipping core in 2022. The size sets the speculative horizon: roughly 320 instructions can be in flight at once. This is enough to hide a typical last-level cache miss, which costs 80 to 150 cycles on an X3 SoC, provided the workload’s IPC is low enough that 320 in-flight instructions span that many cycles. At a sustained 4 IPC the window is only 80 cycles wide, which covers the low end of that range.

Issue and execute

The X3 has six integer ALU pipes, of which two also resolve branches. The dispatch stage distributes uops to per-class issue queues, each of which holds 20 to 30 entries depending on the class. The wakeup-and-select logic per queue follows the standard pattern from Chapter 53.

Memory dispatch routes loads to three load pipes and stores to two store pipes. The L1 data cache is 64 KiB, 4-way set-associative, with 64-byte lines. The cache supports two simultaneous reads and one write per cycle (or, in some configurations, three reads if no write is outstanding). Store-to-load forwarding is implemented on a byte-mask basis as described in Chapter 54.

The FP/SIMD subsystem has four pipes, three of which can execute the multiply-add. The vector width is 128-bit Advanced SIMD. The X3 does not implement SVE2 at the standard 256-bit width that some other ARMv9 implementations do. (The Cortex-X4 added wider SVE2 support.)

Commit

The X3 commits up to 8 uops per cycle, matching the dispatch width. Commit walks the reorder buffer in program order and finalizes the rename map updates. The commit-width is wider than the rename-width because multi-uop instructions retire as a group when all their uops have completed.

03.Neoverse N2: The Server Workhorse

Neoverse N2 is the second generation in ARM’s Neoverse server line. The first generation, Neoverse N1 (2019), shipped in AWS Graviton 2 and in Ampere Altra and Altra Max. Neoverse N2 (2021) shipped in Alibaba’s Yitian 710 and Microsoft’s Azure Cobalt 100. Successor cores Neoverse V2 and Neoverse N3 followed in 2023 and 2024.

Pipeline differences from Cortex-X3

The N2 pipeline is structurally similar to Cortex-X3 but narrower and shallower in several dimensions. Decode and rename are 5-wide versus 6 on X3. The reorder buffer is 160 entries versus 320 on X3. There are four integer pipes versus six on X3. Two load pipes versus three on X3.

The N2 sacrifices peak single-thread performance to fit more cores per cluster within a given thermal envelope. The table below gives ARM’s published per-core area estimates from HotChips disclosures.

The N2 has roughly half the area and roughly two-thirds the per-thread performance of an X3. The cluster perspective inverts that: a 128-core N2 chip delivers roughly ten times the aggregate throughput of an 8-core X3 cluster, despite the lower per-core performance.

SVE2 implementation

The Neoverse N2 implements SVE2 at a 256-bit vector length (twice the 128-bit Advanced SIMD width). The four FP/SIMD pipes each execute up to 256 bits per cycle, for an aggregate 1024-bit-per- cycle throughput on dense FP work. This is on par with x86-64 AVX-512 in flops per cycle, with the difference that SVE2 specifies the vector length as implementation-defined, so different cores can implement different widths without changing the binary.

The N2’s SVE2 implementation is one of the showcase features of the design. Server workloads that benefit from wide vectors (scientific simulation, AI inference, dense linear algebra) exploit it directly. The Cortex-X3’s narrower Advanced SIMD path is less suitable for these workloads but covers the more typical mobile use cases.

Cluster integration

The N2 integrates through the Coherent Mesh Network (CMN-700), a mesh-on-chip interconnect that scales to 128 cores per chip with distributed L3 cache slices. Each L3 slice is 1 MiB to 4 MiB, distributed across the mesh nodes. The total L3 in a 128-core N2 SoC can exceed 128 MiB.

The mesh interconnect is similar in spirit to Intel’s mesh on Skylake-SP and successor server parts, but with different bandwidth and latency tradeoffs. The CMN-700 supports up to 12 memory controllers per chip, which gives the N2 SoC the memory bandwidth needed to feed 128 cores.

04.Branch Prediction in ARM’s Big Cores

ARM has been particularly aggressive about branch prediction investment. The Cortex-X3’s branch predictor includes the following structures, as documented in the TRM and various HotChips disclosures.

Direction predictor

The X3 direction predictor is a TAGE variant with several tables of geometric history length and additional statistical correctors. Total predictor storage exceeds 50 KiB on X3, roughly double the size on the Cortex-A77. The accuracy on typical workloads is reported as above 97% misprediction-free for SPECint and JavaScript-style workloads.

Branch target buffer

The BTB on X3 is multi-level: a small first-level BTB at the fetch stage to enable zero-cycle redirects on hot branches, a larger second-level BTB to handle cold branches with a one-cycle redirect penalty, and a third-level BTB to cover the largest working sets at a higher redirect penalty. The total BTB capacity is reported in the TRM as 8K to 12K entries across the levels.

Return address stack

The RAS in X3 is 16 to 32 entries depending on the configuration. The RAS handles function call and return targets, which are notoriously hard for direction predictors to get right. The RAS gives a target prediction for return instructions that is correct unless the call/return stack overflows.

Comparison with Neoverse N2

The N2’s predictor is similar in algorithm but smaller in storage, on the order of 30 KiB. The smaller storage saves area on a per-core basis. The N2’s target workloads (server, cloud) have smaller per-thread branch working sets than the mobile workloads the X3 targets, so the smaller predictor is well-matched.

05.The Memory Pipeline

ARM’s load-store unit deserves a separate section because the implementation departs from the textbook in interesting ways.

Load pipes and bandwidth

The X3 has three load pipes. Each pipe can deliver one register-width load per cycle (16 bytes for Advanced SIMD, 8 bytes for scalar). The L1 data cache supports three reads per cycle when no write is outstanding, or two reads plus one write per cycle.

The three-pipe configuration matters for memory-intensive workloads. A loop that issues three loads and two ALU operations per iteration can sustain 5 IPC on X3 because all five operations execute in the same cycle. A two-load-pipe core such as N2 needs a second cycle for the third load, so the same loop sustains only 2.5 IPC. The load-pipe count becomes the discriminator once a loop carries more than two loads per iteration. Below that threshold the load pipes are not what separates the two cores, since N2 also has four integer pipes and can pair two loads with an ALU operation in the same cycle.

Store buffer and forwarding

The store buffer in X3 holds 96 entries. Stores enter the buffer at dispatch and stay until the L1 cache accepts them. Loads check the store buffer for older stores to the same address, forwarding data if the overlap is complete and stalling if partial.

The ARM memory model (release acquire) is weaker than x86-64 TSO, which allows the load-store unit more flexibility in reordering than an x86 implementation would have. The X3 exploits this flexibility through aggressive memory-disambiguation prediction: the LSU speculates that load and store addresses do not alias and proceeds with the load before the store address is known. If the speculation is wrong, the load is replayed.

Prefetcher

The X3 has multiple hardware prefetchers operating at L1 and L2. The L1 stride prefetcher detects regular access patterns and prefetches ahead. The L2 region prefetcher prefetches contiguous cache lines on detected sequential access. The L2 spatial-locality prefetcher uses access-history tracking similar to the SMS scheme covered in Chapter 40.

The N2’s prefetchers are similar in algorithm but tuned for sustained server workloads rather than burst mobile workloads.

06.Power Management

Both cores ship with ARM’s standard power-management infrastructure. Per-core voltage and frequency control is supported. Both cores participate in DVFS at the cluster level, with per-core overrides where the cluster allows it.

The X3 is paired with smaller "medium" and "little" cores in the big.LITTLE configuration. The scheduler (typically Energy Aware Scheduling in Linux) migrates threads between the big and little cores based on the workload phase. A latency-sensitive burst runs on the X3. A background thread runs on a Cortex-A510 little core. The result is much better energy efficiency than a uniform-core design would have delivered.

The N2 in server configurations runs all cores at similar operating points. Per-core DVFS still applies, but the dispersion between cores is smaller. A 128-core N2 cluster typically runs at 2.5 GHz to 3.0 GHz sustained, with brief turbo bursts to 3.4 GHz on lightly-loaded cores.

07.Comparison with the Textbook Model

Both Cortex-X3 and Neoverse N2 conform closely to the abstract OoO model of Part V. The same structures appear: a multi-stage front end, rename, dispatch, distributed issue queues, multiple execute pipes, a load-store queue with store-to-load forwarding, and an in-order commit stage. The differences are mostly in scale, in tuning, and in two or three places where the implementation diverges from the textbook for a specific engineering reason.

The table below summarizes the comparison.

The two departures worth highlighting are the split rename (versus the merged PRF that BOOM uses) and the absence of SMT (versus the SMT-2 that x86-64 cores ship).

The split rename gives smaller per-domain physical register files, which simplifies the read-port count and the rename allocation logic. The cost is a larger total register count for the same number of in-flight live destinations, since integer and FP cannot share physical entries.

The SMT-1 choice is consistent across the whole ARM product line. The reasoning is that the area spent on SMT (duplicated architectural state, wider tags, fairness logic) is better spent on more independent cores. The Neoverse N2 SoC scaling to 128 cores per chip illustrates the payoff.

08.What the Public Sources Show

This chapter has cited only public sources: the ARM Architecture Reference Manual [1], the per-core Technical Reference Manuals, ARM’s HotChips presentations from 2021 to 2023, and ARM’s developer blog posts. The TRMs are particularly valuable because ARM, unlike Intel and AMD, publishes per-core microarchitecture documents that explicitly state the pipeline width, reorder buffer depth, issue queue size, branch predictor configuration, cache geometry, and many other parameters that this chapter relied on.

A reader who wants to extend the case study can do so directly from those documents. The Cortex-X4 TRM, the Cortex-X925 TRM (2024 flagship), and the Neoverse V2 / N3 TRMs follow the same format and document the same parameters for successor cores. The HotChips archive at hotchips.org carries the slide decks for every recent ARM big-core announcement.

The contrast with x86-64 documentation is worth a brief note. Intel publishes the Optimization Reference Manual [2] and the Software Developer Manual, which document architecturally visible behavior and provide tuning guidance. They do not publish a per-core microarchitecture reference at the depth of an ARM TRM. AMD publishes the Software Optimization Guide [3] at a similar level. The microarchitecture parameters of Intel and AMD cores are inferred from HotChips disclosures, ISCA / Micro papers, and reverse-engineered measurement studies. The level of verifiable detail is lower than for ARM big cores.

09.Worked Examples

10.Exercises

References

1. [1](2024). “ARM.”
2. [2](2024). “Intel.”
3. [3](2024). “AMD64.”