Before You Start

Error Correction Coding is planned at 187 chapters. Fifteen Parts hold 182 of them, two chapters of front matter come before Part I, and three reference chapters come after Part XV. This chapter is a map of the whole book, a guide to the shape every Part and every chapter follows, and a primer on the notation later chapters use without stopping to explain. Every number it states is reproduced by chf1_ecc_before_you_start.py in the book's companion repository, ecc-book-lab, which prints each value beside the one stated here and stops at the first disagreement. Its two charts are drawn by chf1_charts.py. Both scripts use only Python's standard library.

The subject has a clear birth date. In 1948 Claude Shannon showed that a channel which corrupts some of what passes through it can still deliver information with as few errors as anyone demands, provided the information is sent more slowly than a limit set by the channel's noise [1]. His argument proved that good codes exist without building one. In 1950 Richard Hamming published codes a machine could use, codes that find a single wrong bit in a block and put it right [2]. The distance between knowing that good codes exist and having circuits that run them is the territory this book covers.

After a tour of the Parts and the shape of each chapter, the longest stretch is a notation primer, from sums and logarithms to XOR, sets and proofs, each taught from an example small enough to check by hand. The companion repository and a suggested route close the chapter.

01.What the Book Covers and for Whom

An error correction code adds carefully chosen redundancy to data, so that damage done while the data is stored or sent can be noticed and, with enough redundancy, repaired. The book follows that idea from a single parity bit to the research frontier, through block codes and the finite fields under them, the cyclic redundancy checks that guard network packets, the BCH and Reed–Solomon codes inside storage, and the soft-decision, convolutional, turbo, low-density parity-check and polar codes that radios and solid-state drives run on.

Each major family of codes is treated three ways. The mathematics comes first, with every claim proved, except a few deep results whose proofs are sketched and pointed to a full one. The decoding algorithm comes next, worked by hand on a small case. The hardware comes last, for the codes that have it, as SystemVerilog supplied finished from the companion repository, tested with cocotb and synthesized with Yosys.

The reader the book has in mind knows high-school algebra and binary numbers and nothing else about the subject. The depth ramps across the Parts rather than inside a chapter, from undergraduate level in Parts I to V, through graduate level in Parts VI to XII, to research level in Parts XIII to XV. Three textbooks make good companions. Lin and Costello treat the classical codes and their decoders in depth [3], MacKay places codes beside probabilistic inference and is strong on iterative decoding [4], and Cover and Thomas give the information theory behind Part I with full rigor [5].

02.The Fifteen Parts

Figure 1 shows how the 187 chapters are spread across the book. The Parts range from 7 chapters, in Part XV, to 19, in Parts II and XIII. The brackets mark the front matter, the three levels and the back matter, and Table 1 gives the totals.

Part I, Errors, Information and Channels, explains why codes are needed and measures what they buy. Part II, Linear Block Codes, builds arithmetic modulo a prime and linear algebra over it, then the Hamming codes and codes that correct one error and detect two. Part III constructs every finite field and the circuits that compute in them. Part IV covers cyclic codes, the shift registers that encode them and the cyclic redundancy check. Part V develops the BCH and Reed–Solomon decoders and the architectures that run them at line rate.

Part VI uses the channel's confidence in each bit and builds strong codes from weaker ones. Parts VII and VIII cover convolutional codes with the Viterbi decoder, then turbo codes, whose two decoders trade beliefs until they agree. Part IX covers low-density parity-check codes, the decoders inside Wi-Fi, 5G and solid-state drives, and Part X covers polar codes. Part XI meets real faults in caches, DRAM, flash, disks and serial links, and Part XII designs codes together with the signals that carry them.

The research Parts follow. Part XIII covers the structure theory of codes and decoding beyond half the minimum distance. Part XIV covers cryptography over the same finite fields, public-key schemes built on codes, and codes as defenses against physical attack. Part XV covers quantum error correction, DNA storage and other places the subject is growing. The three reference chapters list every symbol, give tables such as small finite fields, and annotate further reading by Part.

Figure 2 shows how the Parts lean on one another. The first five form a spine, since Part II's linear algebra builds Part III's finite fields, Part IV's cyclic codes are polynomials over those fields, and Part V's decoders solve equations in them. From there the book branches, one branch leaving Part V through soft decisions to convolutional and turbo codes, another leaving Part IV's cyclic codes for low-density parity-check and polar codes. Every Part still assumes all the Parts before it, so the arrows say which Part to reopen first when a later chapter sends the reader back.

03.How a Part Is Laid Out

Most Parts follow the shape in Figure 3. Teaching chapters come first and build the ideas. One or more hardware architecture chapters follow, turning the mathematics into circuits. The last three chapters apply what came before, with a case study that finds the Part's codes in a real system, a project that builds a working component, usually an encoder or decoder, and a lab that measures something about it. Part IV returns to teaching after its CRC circuits, Part IX places its decoder architecture chapter late, and Part XIII, on the structure theory of codes, has no hardware chapter and builds its project and lab in Python.

Part I shows the pattern with one variation. Its first nine chapters teach, from Why Codes Exist to Error-Control Strategies and Performance, and its two circuits, a majority voter drawn in the first chapter and a parity checker built in its lab, are small enough to need no separate hardware chapter. The Part closes with Case Study: Codes in the Wild, then Project: A Channel Simulator, then Lab: A First Hardware Checker. Part II closes at greater length, with a chapter on verifying error-correcting code (ECC) hardware, a case study of the codes inside processor caches, a project that corrects any single error and detects any two in a 72-bit codeword protecting 64 data bits, and a lab that measures how often a decoder corrects a word into the wrong one.

A project's code is already in the repository, finished and tested, and the chapter is a guided reading of it. It walks through the Python reference model that defines the right answer, explains the SystemVerilog, where the project has hardware, at the level of the architecture, shows how the testbench checks the circuit against the model, and ends with exercises that change the design. No solutions are held back. A lab runs that code and measures something, such as an error rate, an area or a speed.

04.How a Chapter Is Laid Out

Every chapter follows the order in Figure 4. The opening paragraphs link the companion scripts, a box of key terms defines the chapter's words, and the teaching sections follow. A short section headed Looking Ahead points to the next chapter, then come worked examples, exercises with answers, and the references.

Formal statements sit in callout boxes so they can be found again, and Table 2 lists the kinds. Each box opens with a band that names its kind in small capitals, followed by the box's own name where it has one, such as Theorem over "The triangle inequality". The color groups the kinds by purpose: cyan for words, violet for mathematics, green for practice and orange for warnings. The label in the band, not the color, is what says which kind a box is, so a reader who cannot tell the colors apart loses nothing.

A proof follows the theorem or lemma it proves in ordinary paragraphs. It begins with the word Proof in italics and ends with a small black square, $\blacksquare$, which marks where the argument stops. Key displayed equations carry a number in parentheses at the right, so that later text can point back to them.

05.Reading the Mathematics

The notation below appears in nearly every later chapter. Each piece starts from a number small enough to check by hand, then gives the general rule.

Sums and Products

The whole numbers, in this book, are the integers $\ldots, -2, -1, 0, 1, 2, \ldots$, the numbers with no fractional part, and the rules below usually need only those from 0 up. Adding the whole numbers from 1 to 5 gives $1 + 2 + 3 + 4 + 5 = 15$. Mathematics writes such a sum with a capital sigma, $\Sigma$. The expression $\sum_{i=1}^{5} i$ reads "the sum of $i$ for $i$ from 1 to 5". The index $i$ takes each whole-number value from the one below the sigma to the one above, and the expression to its right is evaluated for each value and added. Multiplication uses a capital pi, so $\prod_{i=1}^{4} i = 1 \times 2 \times 3 \times 4 = 24$. In general, for numbers $x_1, x_2, \ldots, x_n$,

A sum with no terms is 0 and a product with no factors is 1, because adding 0 and multiplying by 1 change nothing. Products appear whenever several independent things must all go right. If each of the 8 bits in a byte survives a channel with probability 0.99, independently of the others, all eight survive with probability $\prod_{i=1}^{8} 0.99 = 0.99^{8} \approx 0.9227$, which leaves about $0.0773$ for the chance that at least one bit is wrong.

Factorials and Binomial Coefficients

The letters a, b and c can be lined up in six orders, abc, acb, bac, bca, cab and cba. The first position takes any of 3 letters, the second any of the 2 left, and the last the 1 remaining, so the count is $3 \times 2 \times 1 = 6$.

So $3! = 6$ and $5! = 120$. Binomial coefficients answer the counting question that runs through the whole book, which is how many strings of $n$ bits contain exactly $k$ ones. For $n = 4$ and $k = 2$ the strings are 1100, 1010, 1001, 0110, 0101 and 0011, six in all. Without listing them, the first 1 can go in any of 4 positions and the second in any of the 3 left, giving $4 \times 3 = 12$, but each string has been counted twice, once for each order its two positions could have been picked, so there are $12 / 2 = 6$. The same reasoning works for any $n$ and $k$. Picking $k$ positions one after another can be done in $n(n-1)\cdots(n-k+1)$ ordered ways, which equals $n!/(n-k)!$ because the factors from $n - k$ down to 1 cancel. Each set of $k$ positions is reached in $k!$ of those orders, one for each order of its members. Dividing the ordered choices by $k!$ gives, in general,

On the example, $4! / (2!\,2!) = 24 / 4 = 6$. For a byte, $\binom{8}{2} = (8 \times 7)/2 = 28$ strings have exactly two ones.

Proof. Count the $n$-bit strings with exactly $k$ ones by their first bit. If it is 1, the other $n - 1$ bits hold the remaining $k - 1$ ones, in $\binom{n-1}{k-1}$ ways. If it is 0, the other $n - 1$ bits hold all $k$ ones, in $\binom{n-1}{k}$ ways. Every string falls into exactly one case, so the counts add to $\binom{n}{k}$. $\blacksquare$

As a check, $\binom{7}{3} = 35$ and $\binom{6}{2} + \binom{6}{3} = 15 + 20 = 35$. Pascal's rule builds every binomial coefficient by addition alone. Written as a triangle, row $n$ lists $\binom{n}{0}, \binom{n}{1}, \ldots, \binom{n}{n}$, every row starts and ends with 1, since there is one way to pick none and one way to pick all, and each inside entry is the sum of the two entries just above it, which is Pascal's triangle. The first rows are 1, then 1 1, then 1 2 1, then 1 3 3 1. Figure 5 plots row 7, the counts $\binom{7}{k}$ for $k$ from 0 to 7. The row reads the same from either end and adds up to $128 = 2^7$, the number of 7-bit strings. Both facts are proved later in this chapter.

Exponents and Base-2 Logarithms

Doubling 1 ten times gives $2^{10} = 1024$. An exponent counts repeated multiplication, so $2^3 \times 2^4 = 2^7 = 128$ and $(2^3)^2 = 2^6 = 64$. The two rules behind those, $2^a \times 2^b = 2^{a+b}$ and $(2^a)^b = 2^{ab}$, fix what zero and negative exponents must mean. Since $2^a \times 2^0$ must equal $2^a$, $2^0 = 1$, and since $2^1 \times 2^{-1}$ must equal $2^0$, $2^{-1} = 0.5$. Likewise $2^{-3} = 0.125$. Exponents that are not whole numbers follow the same two rules: $2^{1/2} \times 2^{1/2} = 2^1$, so $2^{1/2} = \sqrt{2} \approx 1.414$, and in general $2^y$ is defined for every real $y$, is always positive and grows as $y$ grows.

A byte of 8 bits takes $2^8 = 256$ values. The reverse question, how many bits a given number of values needs, is answered by the logarithm.

So $\log_2 1024 = 10$, and because exponents add when powers of 2 multiply, logarithms turn products into sums, as in $\log_2 (8 \times 32) = 3 + 5 = 8 = \log_2 256$. For positive $x$ and $y$ and any number $k$,

Logarithms of numbers that are not powers of 2 are not whole. A calculator gives $\log_2 10 \approx 3.322$, and the second rule gives $\log_2 1000 = 3 \log_2 10 \approx 9.966$, just under 10, since 1000 lies just below 1024. The base-2 logarithm will measure information in Part I and count the stages in a tree of two-input gates.

Error rates are written as powers of ten. A bit error rate of $10^{-12}$ means one bit in $10^{12}$ is wrong on average, so a link carrying $10^{10}$ bits per second sees $10^{10} \times 10^{-12} = 10^{-2}$ errors per second, one every 100 seconds.

Floor and Ceiling

So $\lfloor 3.7 \rfloor = 3$ and $\lceil 3.2 \rceil = 4$. Below zero the rule still means rounding down and up, so $\lfloor -3.7 \rfloor = -4$, not the $-3$ that cutting off the fraction would give, and $\lceil -3.7 \rceil = -3$. A whole number is its own floor and ceiling, $\lfloor 4 \rfloor = \lceil 4 \rceil = 4$. The ceiling counts containers. A message of 100 bits fills $100 / 8 = 12.5$ bytes and so needs $\lceil 12.5 \rceil = 13$. Naming 8 things takes $\lceil \log_2 8 \rceil = 3$ bits and naming 9 takes $\lceil \log_2 9 \rceil = 4$. The floor counts whole things that fit, so $\lfloor 5/2 \rfloor = 2$.

Mod-2 Arithmetic and XOR

Dividing 7 by 2 leaves remainder 1, and so does dividing 13 by 2, written $7 \bmod 2 = 1$ and $13 \bmod 2 = 1$. Arithmetic modulo 2 keeps only that remainder, always 0 or 1, so it is arithmetic on single bits. The one surprising case is $1 + 1 = 2$, and $2 \bmod 2 = 0$, so modulo 2 one plus one is zero.

Read down the input pairs 00, 01, 10 and 11, the XOR truth table is 0, 1, 1, 0, and so is the table of addition modulo 2, entry for entry. XOR and addition modulo 2 are one operation under two names, as Figure 6 shows. Multiplication modulo 2 gives 1 only when both bits are 1, which is the AND operation. The OR of two bits is 1 when at least one of them is 1, so it differs from XOR only on the input 11, where OR gives 1 and XOR gives 0. A gate is a small circuit that computes one of these operations on its input bits, and hardware diagrams draw each gate in a standard shape of its own, as the figure does for XOR.

On strings, $1011 \oplus 0110 = 1101$, and XORing with 0110 again gives $1101 \oplus 0110 = 1011$, the starting string. That return trip is no accident.

Proof. Each law involves at most three bits, each 0 or 1, so every case can be checked, which is a proof by cases. The first law has four cases. When $a = b$ both sides are the same expression, and when $a \ne b$ both sides are 1. The second has eight cases, one per choice of $a$, $b$ and $c$, and in each of the rows 000 through 111 both sides equal 1 exactly when an odd number of the three bits are 1. The third has two cases, $0 \oplus 0 = 0$ and $1 \oplus 0 = 1$, and the fourth has two, $0 \oplus 0 = 0$ and $1 \oplus 1 = 0$. On strings each law holds in every position, so it holds for the whole strings. $\blacksquare$

These laws give the most useful fact in the book. Suppose a string $m$ is sent and the channel flips the bits marked by an error pattern $e$, a string with a 1 wherever a bit was flipped. Flipping a bit is XORing it with 1, so what arrives is $r = m \oplus e$, and XORing with $e$ again gives

The XOR of all the bits of a string is its parity. The parity of 1011 is $1 \oplus 0 \oplus 1 \oplus 1 = 1$, which says the string holds an odd number of ones. That reading holds for every string. Work along it from the left with a running XOR that starts at 0. A 0 bit leaves the running value alone and a 1 bit flips it, so by the end it has flipped once for each one in the string and stands at 1 exactly when the number of ones is odd. Why Codes Exist builds its first error detector from that one bit.

Sets and Set-Builder Notation

A set is a collection of distinct objects written between braces. The set of all 3-bit strings is

$$\{0,1\}^3 = \{000, 001, 010, 011, 100, 101, 110, 111\},$$

which has 8 members. In general $\{0,1\}^n$ is the set of all $n$-bit strings, and since each position is independently 0 or 1, it has $2^n$ members. The size of a set $S$ is written $|S|$, so $|\{0,1\}^n| = 2^n$. The symbol $\in$ means "is a member of", as in $101 \in \{0,1\}^3$, and $\emptyset$ is the empty set. For sets $A$ and $B$, the union $A \cup B$ holds everything in either, the intersection $A \cap B$ everything in both, and $A \subseteq B$ says every member of $A$ is in $B$.

Set-builder notation describes a set by a rule. The 4-bit strings with exactly two ones form

$$\{\, x \in \{0,1\}^4 \mid x \text{ has exactly two ones} \,\},$$

read "the set of $x$ in $\{0,1\}^4$ such that $x$ has exactly two ones", with the vertical bar meaning "such that". It holds the six strings listed earlier. A code, in the chapters to come, is exactly this kind of object, a set of allowed strings chosen from $\{0,1\}^n$ by a rule.

Probability Notation

The probability of an event $A$ is written $P(A)$, a number between 0 and 1, and the probability that $A$ does not happen is $1 - P(A)$. When several events are independent, meaning that knowing whether some of them happened says nothing about the others, the probability that all of them happen is the product of their probabilities. The byte calculation above used both rules. Probability for Coding builds the rest from the beginning.

What a Proof Is

A proof is an argument that a claim holds in every case it covers. Checking a hundred examples is evidence, not proof, because the hundred-and-first might fail. The book uses three kinds of proof again and again. A proof by cases splits the possibilities into groups that together cover everything and settles each group, as the proof of the XOR laws did.

Proof. Every whole number is even or odd. If $n$ is even, the product has an even factor $n$. If $n$ is odd, then $n + 1$ is even and is an even factor. The two cases cover every $n$. $\blacksquare$

For instance, $7 \times 8 = 56$. A proof by induction handles every whole number from a starting value on, all at once. It proves the claim for the first value, the base case, then proves that whenever the claim holds for one value it holds for the next, the inductive step. Like a row of dominoes, the base case tips the first, and the inductive step makes each one tip the next.

Proof. For the base case $k = 1$, the sum is $2^0 = 1$ and the right side is $2^1 - 1 = 1$. For the inductive step, suppose the claim holds for some $k$. Adding the next term, $2^k$, to both sides gives $\sum_{i=0}^{k} 2^i = 2^k - 1 + 2^k = 2^{k+1} - 1$, the claim for $k + 1$. $\blacksquare$

With $k = 8$, $1 + 2 + 4 + \cdots + 128 = 255 = 2^8 - 1$. In binary this is plain, since the eight powers of 2 from 1 to 128 are the place values of a byte, and adding all of them sets every bit, giving 11111111. A proof by counting two ways shows two expressions are equal because each counts the same collection.

Proof. Count $\{0,1\}^n$ in two ways. Directly, it has $2^n$ members. Sorted by how many ones they contain, the strings with exactly $k$ ones number $\binom{n}{k}$, and every string lands in exactly one group, so the total is the sum of the group sizes. Both counts describe the same set, so they are equal. $\blacksquare$

For $n = 3$ the groups are $\{000\}$, $\{001, 010, 100\}$, $\{011, 101, 110\}$ and $\{111\}$, of sizes 1, 3, 3 and 1, which add to $8 = 2^3$. The symmetry of each row has a counting proof too, left as an exercise.

06.Reading the Figures

Every figure follows one style. Strokes and labels take the color of the surrounding text, so a figure follows the page between dark and light themes. One accent, green, marks what the caption is about. When a figure must tell states apart, cyan marks what is active, orange what is waiting, red what is faulted and violet what is speculative, and each state also carries a label or shape, so color is never the only signal. A chart's caption ends with a line beginning Data or Source that says where its values come from. In the map of the companion repository, Figure 8, a dashed gray outline marks a folder that is planned but not yet present. Every figure carries a description that a screen reader reads aloud.

Circuits draw the basic gates in their ANSI/IEEE Std 91-1984 distinctive shapes, a flip-flop with a clock wedge, and a multiplexer as the common trapezoid, and Figure 7 is the key. AND is a D shape, OR a curved shield, and XOR an OR with a second curve behind its inputs. A small circle, a bubble, on an output inverts it, turning AND into NAND, OR into NOR, XOR into XNOR and a plain triangle into NOT. A flip-flop, which stores one bit and updates it on each clock edge, is a rectangle with a wedge on its clock input. A multiplexer, which passes one of several inputs to its output, is a trapezoid with its select line on a slanted edge. Wires enter a gate without arrowheads.

Older coding texts often draw a mod-2 adder as a circled plus, but since addition modulo 2 is XOR, this book draws the XOR gate, and it draws a delay cell, which holds a bit for one clock cycle, as the D flip-flop it is. Rectangles are kept for blocks with no gate symbol, such as a memory or a module a later figure opens up.

07.What Is Assumed and What Is Not

The opening Parts assume high-school algebra and binary numbers. High-school algebra means rearranging an equation, using exponents and reading a graph. Binary numbers means knowing that each bit stands for a power of 2, so that $1101_2 = 8 + 4 + 0 + 1 = 13$, where the subscript 2 marks a number written in binary.

No abstract algebra is assumed, so groups, rings and finite fields are built from worked examples in Parts II and III before anything relies on them. No coding theory is assumed, which is the point of the book. No programming fluency is assumed either, since the Python scripts are short and meant to be read, and the next chapter includes a primer on reading them. Where a later chapter needs a tool from calculus, such as the number $e$, it introduces the tool where it first appears.

08.The Companion Repository

Every number, chart and hardware listing in the book comes from ecc-book-lab, laid out as Figure 8 shows. It is public and released under the MIT license.

The python folder holds one script per chapter, beside the scripts that draw the charts, and each stops with an error at the first number that disagrees with the text. They use only the standard library, so Python 3.10 or newer is all they need. The hdl folder holds the SystemVerilog designs, one folder per circuit. The tb folder holds their testbenches, written in Python with cocotb, which drive each circuit and compare its outputs with a model written in Python. The synth folder holds scripts that run Yosys, an open-source synthesis tool, to turn a design into gates and count them. Folders for formal proofs and for measured data are planned for later Parts.

The first script reproduces this chapter and ends with "All chapter f1 numbers reproduced." The second ends with "All chapter 1 numbers reproduced." Script numbers count from Why Codes Exist, whose script is ch01, and the two front-matter chapters have scripts named chf1 and chf2, so the number in a script's name and its output differs from the chapter number shown at the top of each page. The hardware tools are needed only for projects and labs. They are Verilator, which simulates the SystemVerilog [6] and must be version 5.036 or newer for cocotb to drive it [7], cocotb 2.1, which runs the Python testbenches against the simulation [8], and Yosys [9].

09.Learning the Hardware Languages

This book does not teach SystemVerilog or cocotb. Every hardware listing comes finished from the repository, and the text explains each circuit at the level of its architecture, meaning which signals enter, what is stored, what is computed and what leaves on each clock cycle, so every hardware chapter can be followed without writing SystemVerilog.

The languages themselves belong to other books on this site. RTL Design and Verification, in preparation, will teach SystemVerilog design and its verification with cocotb, UVM and formal tools. Until it appears, the Computer Architecture book covers SystemVerilog and the open-source toolflow, though not cocotb, in three chapters. Hardware Description Languages introduces Verilog and SystemVerilog beside VHDL and Chisel, Lab: The Open-Source HDL Workflow runs a design through Icarus Verilog, Verilator and Yosys, and Lab: Verification and Cycle-Accurate Simulation tests a processor against published test suites and reads its waveforms in Surfer, the open-source waveform viewer this book uses too [10].

10.A Suggested Route Through the Book

The book is written to be read in order, which is the route that never leaves a gap, and the levels give natural stopping points. The front matter and Parts I to V run from a single parity bit to the Reed–Solomon decoders inside storage, Parts VI to XII carry on to graduate level, and Parts XIII to XV go on to research. The reference chapters serve at any point. When a chapter leans on an idea that has faded, Figure 2 says which Part to reopen first. Worked examples repay pencil and paper before their answers are read.

11.Looking Ahead

The next chapter, on setting up the lab bench, installs the tools on macOS, on Linux and on Windows through the Windows Subsystem for Linux, clones the companion repository, runs its first test, and gives a short primer on reading Python. Part I then opens with Why Codes Exist, which starts from a single flipped bit. The number 100, stored in a byte as 01100100, becomes 228 if its highest bit flips, and that flip is an XOR with 10000000. Why Codes Exist works out how likely such flips are.

12.Worked Examples

13.Exercises

References

1. [1]Shannon, Claude E. (1948). “A Mathematical Theory of Communication.” Bell System Technical Journal, 27(3), pp. 379–423. The paper concludes in the October issue, volume 27, number 4, pages 623 to 656. doi:10.1002/j.1538-7305.1948.tb01338.x
2. [2]Hamming, Richard W. (1950). “Error Detecting and Error Correcting Codes.” Bell System Technical Journal, 29(2), pp. 147–160. doi:10.1002/j.1538-7305.1950.tb00463.x
3. [3]Lin, Shu and Costello, Jr., Daniel J. (2004). “Error Control Coding: Fundamentals and Applications.” 2nd edition. Pearson Prentice Hall.
4. [4]MacKay, David J. C. (2003). “Information Theory, Inference, and Learning Algorithms.” Cambridge University Press. The author's site offers the whole book free to read on screen. https://www.inference.org.uk/itila/book.html
5. [5]Cover, Thomas M. and Thomas, Joy A. (2006). “Elements of Information Theory.” 2nd edition. Wiley-Interscience. doi:10.1002/047174882X
6. [6]Verilator Project “Installation.” Verilator 5.054 documentation. Read 2026-11-14. https://verilator.org/guide/latest/install.html
7. [7]cocotb Project “Simulator Support.” cocotb 2.1 documentation. Read 2026-11-14. https://docs.cocotb.org/en/stable/simulator_support.html
8. [8]cocotb Project “Installation.” cocotb 2.1 documentation. Read 2026-11-14. https://docs.cocotb.org/en/stable/install.html
9. [9]YosysHQ “Installation.” Yosys documentation. Read 2026-11-14. https://yosyshq.readthedocs.io/projects/yosys/en/latest/getting_started/installation.html
10. [10]Surfer Project “Introduction.” The Surfer book. Read 2026-11-14. https://docs.surfer-project.org/book/