In some of the earlier articles on this blog, we talked about the nature of real numbers and the meanings of infinity. The theory outlined in these posts is interesting but also hopelessly abstract. It’s as if we’re inventing make-believe worlds that have no discernible connection to reality.

In today’s post, we’ll examine a cool counterexample: an outcome of an numerical experiment that can be backed up with fairly simple proofs, but that makes sense only if you take a step back to consider the construction of real numbers and rationals.

We start by picking a real number r. Your job is to approximate it as closely as possible using a rational fraction a / b with a reasonably small denominator. The approximation can’t be the same as r. Is this task easier if r itself is rational or irrational?

Make a guess and let’s dive in. For simplicity, we’ll stick to a positive r and positive fraction denominators throughout the article.

Defining “good”

For a chosen denominator b, it’s pretty easy to find the value of a that gets us the closest to the target r. We must consider two cases: the largest “low-side” fraction that’s still less than r; and the smallest “high-side” fraction greater than r. If there’s a rational fraction that matches r exactly, that solution is prohibited by the rules of the game; we need to pick one of the nearby values instead.

If you wanted to find an exact match, you could try a_ideal = r · b; this makes the a / b fraction equal to r · b / b = r. That said, r · b might not be an integer (or even a rational number), so even without the added constraint, the approach is usually a bust.

If we round the value up (⌈r · b⌉), we get a number that is equal or greater than a_ideal; if it’s greater, the difference between the two values will be less than 1. In other words, we can write the following inequality:

This is saying that the rounded-up number may be equal to the ideal solution needed to match r exactly, or it might overshoot the target, but always by less than the minimum possible increment of the numerator in the a / b fraction.

The result is almost what we need, but once more, the rules prohibit approximations that are exactly equal to r. The workaround is to subtract 1 from all sides of the inequality:

The effectively tells us that the middle term — ⌈r · b⌉ - 1 — is always less than the value needed to match r, but the difference is never greater than a single tick of the numerator. We’re as close as we can be; the value of a for the optimal low-side approximation (a / b < r) is:

We can follow the same thought process to find the high-side estimate (a / b > r); this time, we round the product down and then add 1:

Finally, the error (ε) associated with any a / b can be easily calculated as:

We have previously established that if we pick a_low or a_high, the error can’t exceed one tick of the numerator, which works out to ± 1/b. As a practical example, if we’re trying to approximate r = 2 using b = 5, the best inexact solutions are 9/5 = 1.8 on the low side and 11/5 = 2.2 on the high side; they both have an error of 1/b = 0.2.

Next, we’ll try to examine if the error can be less. If we find any approximations that are better than the worst-case scenario — i.e., that satisfy ε < 1/b — we’re gonna call them 1-good.

As an inspection aid, we can also define a normalized score s calculated by multiplying ε by b:

The equation keeps the maximum error at 1 regardless of the denominator we’ve chosen. By that metric, a 1-good approximation is associated with s < 1.

The rational test case

Now that we have the mechanics spelled out, let’s take r = 1/4 and analyze the optimal solutions for some initial values of b:

We find that many approximations are 1-good (ε < 1/b, s < 1) and outperform their peers; for example, 1/5 = 0.2 is better than 2/6 ≈ 0.333. Nevertheless, the results are underwhelming: the values diverge from the 1/b baseline by small factors that are stuck on repeat. If we plot a larger sample, we get:

In this log-scale plot, I also included a diagonal line that represents error values decreasing with the square of the denominator (1/b²). Approximations for which the error dips below this line would be markedly better than the ones that merely dip below 1/b. We can label these 1/b² solutions as 2-good.

In the plot, we see two trivial approximations below the 2-good line, but for a rational r, we can prove that the effect can’t last. We start by rewriting r as a fraction of two integers: r = p / q. The 2-goodness criteria is ε < 1/b². We’ve previously defined ε = | r - a/b | and per the rules of the game, valid solutions must have ε greater than zero. Putting it all together, we can write this inequality that spells out the requirements for 2-good approximations of rationals:

To tidy up, we can bring the middle part to a common denominator:

The denominator is positive, so there’s no harm in taking it out of the absolute-value section and multiplying all sides of the inequality by it:

All the variables here are integers. If b ≥ q, the fraction on the right is necessarily ≤ 1. That creates an issue because it implies the following:

Again, the middle section comprises of integers, so it can’t possibly net fractions. In effect, the equality says: 0 < integer < 1; there’s no integer that satisfies this criterion, so the assumption of b ≥ q leads to a contradiction. If any inexact 2-good approximations of rational numbers exist, they can only exist for b < q.

That’s where we look next. If r = p / q is given at the start, then q doesn’t change and the b < q condition restricts the solutions to integer a / b fractions with smaller denominators. There’s a limited number of these, so if there are any rule-compliant 2-good approximations for a rational, their number is capped and they must be clustered near the beginning of the plot.

This is about as much as we can squeeze out of that stone. The bottom line is that rational numbers are difficult to approximate using other rationals; in fact, the simpler the number, the fewer good approximations we get.

Approximating irrationals

If rationals tend to be relatively resistant to approximations, it might be tempting to assume that the situation with irrational numbers is going to be worse. But, to cut to the chase, here’s the plot of approximations for r = π:

Note that the plot keeps dipping below the 2-good line over and over again. And there are some really nice approximations in there! The first arrow points to 22 / 7 ≈ 3.143 (s ≈ 0.009). The second arrow points to an even better one: 355 / 113 ≈ 3.141593 (s ≈ 0.00003). What’s up with that?

Before we investigate, let’s confirm that π is not special. Surely enough, these patterns crop up for other irrationals too. And it’s not just famous transcendental constants; here’s √42:

To understand what’s going on, we need to build a lengthier proof, although we’ll still stay within the realm of middle-school math.

First, we make a simple observation: given some number r, we can always split it into an integer part v and a fractional part x that satisfies 0 ≤ x < 1. In other words, we’re saying that x always lies in the interval [0, 1).

We can also apply this logic to any multiple of r. In particular, for each integer k between 0 and some arbitrary upper bound K greater than zero, we can calculate k · r and then split the result to obtain a sequence of integer parts (v₀ to v_K) and fractional parts (x₀ to x_K). A simple illustration for π is:

The resulting sequence of fractional parts has K + 1 elements because we started counting at zero; again, each of these elements falls somewhere in the interval [0, 1).

Next, we divide the [0, 1) interval into K sub-intervals (“buckets”) of equal size:

We have K buckets and K + 1 fractional parts tossed into them; no matter how we slice and dice it, at least two elements will necessarily end up in the same bucket. This is the pigeonhole principle.

Again, the reasoning implies that there’s at least one pair of indices, g < h, such that x_g and x_h both ended up in the same bucket. We don’t know anything about the underlying values, except that by the virtue of where they landed, they must be spaced less than 1 / K apart:

The existence of a pair of elements with spacing of less than 1 / K is the crux of the proof. The rest is just a bit of manipulation to relate these elements to a rational approximation of the starting number r.

We show this in a couple of steps. First, as a consequence of how we constructed these fractional parts, we can rewrite any x_k as the difference between the k-th multiple of r and the associated integer part v_k. After making these substitutions for indices h and g, we get:

Next, we rearrange the terms and split out the expressions that appear to form two independent integers: (v_h - v_g) and (h - g). We label these a and b:

We don’t need to dwell on the possible values of a; it’s enough that the number can exist. We’re also not concerned about the exact value of b, although we ought to note that it’s always positive (because we specified g < h) and that it’s necessarily less than or equal to K (because it represents the difference of indices in a list with K + 1 elements).

We’ll lean on these properties soon, but for now, we make these substitutions to obtain:

Remember the flag for later and divide both sides by b to obtain:

Huh — the left-hand portion of the expression is the same as the formula for the error ε of approximating r using a rational fraction a / b. In other words, the inequality seems to be saying that, as a consequence of the pigeonhole principle, we can pick any r and any K > 0, and there’s always some integer a / b that approximates r with an error of less than 1 / (K · b).

We haven’t picked any specific K, but we know that b is always less or equal than K; again, this is because we defined b as a substitution for h - g, the delta between two indices in a sequence of K + 1 elements. Therefore, the expression in the denominator — K · b — involves multiplying b by a value that’s equal or greater than b. In effect, we have proof that for any real r, some a / b that satisfies ε < 1/b².

This proof is known as Dirichlet’s approximation theorem. At first blush, it only guarantees a single 2-good approximation for every real. Worse yet, the solution guaranteed by the proof might not comply with the rules of our game, because nothing stops it from producing exact approximations (ε = 0). So, what did we achieve?

Well, that’s where we come back to an intermediate equation marked with ⚑:

In the earlier course of the proof, we divided the left-hand side of this inequality by b to arrive at the formula identical to ε. Equivalently, we can say that the current form is equal to ε · b. We already have a name for this parameter: s. Further, looking at the formula, we can assert that for any irrational r, the value must be greater than zero because if we multiply r by a positive integer (b) and then subtract another integer (a), we always end up with some fractional part.

To take the next step — and we’re close to the finish line! — note that the proof doesn’t put any constraint on the upper value of K. If we choose some definite K₁, the proof establishes the existence of a single 2-good pair, which we can label a₁ and b₁. If we choose K₂, it proves the existence of a pair we’ll call a₂ and b₂; that pair may or may not produce a functionally different approximation of r. Maybe there’s just a single solution that repeats for every K?

Let’s assume that’s the case; that would mean that s₁ = s₂ regardless of our choice of K₂. However, in the irrational case, we’ve established that s₁ is necessarily a positive number. We can make the 1 / K₂ fraction arbitrarily small by increasing K₂, so no matter how small s₁ is, the right-hand fraction can be made smaller to flip the inequality. This produces a contradiction. It must be that for a given irrational r and some chosen K₂ > K₁, the equation will produce a new, distinct a₂ / b₂ such that s₂ < s₁.

This new approximation is vulnerable to the same fate as the a₁ / b₁ solution it replaced; that’s to say, we can keep incrementing K to conjure as many distinct 2-good pairs as we want. The proof doesn’t guarantee that it’s going to happen on any specific cadence, but it says that it will if we try long enough.

As a postscript, we ought to ask if the same reasoning applies to rationals; if it does, that would contradict our earlier argument that rational numbers can only have a handful of 2-good solutions. To show that there is no contradiction, note that in the rational case, s can conceivably reach zero (i.e., a / b can be an exact approximation). Next, rewrite r as p / q in the left-hand portion of the earlier inequality:

The numerator of the fraction on the right is an integer (because so are a, b, p, and q). The denominator is also an integer that stays constant for a given r. It follows that when approximating rationals, there is a fixed, minimum decrement for s: 1/q. We might start from a non-zero s₁, but if we keep ramping up K, the system must reach the degenerate s_n = 0 case after producing a finite number of inexact approximations. From that point on, s < 1 / K is satisfied for any K and the generation of distinct 2-good pairs ceases.

In other words, you get an infinite supply of surprisingly accurate solutions for irrational numbers, but at limited (often very limited) number of decent results for rationals.

But… why?

Right. The proofs are interesting but don’t offer an intuitive explanation of why these patterns emerge. This is where we go back to my opening remark: it’s easier to grasp the outcome if you look at how rational numbers and reals are “made”.

From the construction of rationals, we know that the spacing between them is arbitrarily close, but at any “magnification level” — for any chosen denominator b — the values divide the continuum into uniform intervals. Uniform spacing also implies maximal spacing: even though there is no upper or lower bound to the values of a / b, they are as far apart as they can be. Any new value inserted onto the number line will necessarily sit “closer” to an existing rational.

The gaps between rationals is where we find irrational numbers. This comes with a lot of weird baggage explored in the previous article, but it also means that for any given irrational r, we have an inexhaustible supply of unexpectedly accurate rational approximations in the vicinity.

Although the puzzle we started with might seem silly, the study of these structures — known as Diophantine approximations — is taken seriously and gets complicated fast. For example, it’s possible to construct so-called Liouville numbers that have an infinite irrationality exponent (endless n-good approximations for any n), but it’s a lot harder to prove that there’s any commonly-encountered number with an irrationality exponent greater than two. In the same vein, algebraic irrationals (e.g., √2) all have an irrationality measure of two, but the proof of this is fiendishly difficult and netted its discoverer the Fields Medal back in 1958.

You might also enjoy my other articles about math, including:

How many dimensions is this?

lcamtuf

·

September 3, 2025

Read full story

Folks, we have the best π

lcamtuf

·

September 15, 2025

Read full story

You gotta think outside the hypercube

lcamtuf

·

Jan 26

Read full story

I write well-researched, original articles about geek culture, electronic circuit design, algorithms, and more. If you like the content, please subscribe.

A while ago, I posted an article about the 19th and early 20th century quest to derive mathematics from the principles of formal logic. We kicked off with Peano arithmetic, which built natural numbers from two ad-hoc constructs: an element representing zero and an abstract “successor” function S(…).

Later, we leaned on set theory to encode the underlying structure of these symbols. This netted us a hierarchy of set-theoretic natural numbers known as ordinals. It also led to an interesting insight: if we allowed the existence of infinite sets, then the set of all natural numbers (ℕ) itself had the structure of an ordinal. In the article, we labeled this infinite number ω and demonstrated that it could be manipulated using the same arithmetic rules as finite numbers, but that it sometimes behaved in wacky ways. For example, we established that ω + 1 ≠ 1 + ω.

We also touched on various methods of reasoning about the magnitude of ordinals and showed that these approaches diverge from each other in the realm of infinities. In particular, we talked about Georg Cantor’s notion of cardinality, which put many distinct infinite ordinals in the same size class, but indicated that there’s a fundamental difference in scale between the set of natural numbers and the set of reals (ℝ).

If you haven’t read the article but are intrigued, I strongly encourage you to give it a go (link). I think it’s an excellent and accessible introduction; if you need an endorsement, the brain trust over at HN called it “watered down” and “slop”.

If you’re up to speed, there might be one thing that’s bugging you: we carefully defined natural numbers from first principles, but then pulled reals out of a hat. This is a gap worth addressing, because as it turns out, real numbers are profoundly weird.

Natural numbers (ℕ)

As a reminder, in the earlier article, we constructed a succession of natural numbers by conjuring an object representing zero and then successively applying function S(…) to it:

The “:=” operator means “is defined as”. Elsewhere, you might see this written as ≝, ≜, or a regular =.

In Peano arithmetic, the label “0” and the successor function S(…) have no deeper meaning: they are just “things” with a couple of common-sense properties spelled out. All the notation indicates is that every subsequent label has some fixed relationship to the one that came before. In the set-theoretic approach we switched to later, we defined these concepts with more precision, but this detail doesn’t matter now.

The important point is that in both models, the successor relationship allowed us to define the behavior of the “+” operator using a pair of simple substitution rules:

Although the rules may seem cryptic, they effectively just codify that adding zero is a no-op and that a + (b + 1) is the same as as (a + b) + 1.

This ruleset lets anyone solve problems such as 2 + 2 without any assumptions about the fundamental meaning of “2” or “+”. To illustrate, from the definition of Peano numbers, we note that “2” is the same as S(1), so 2 + 2 can be restated as 2 + S(1). Switching to that form allows us to apply rule #2, in turn rewriting 2 + S(1) as S(2 + 1):

After that, we can apply the same steps again to expand the nested 2 + 1 sum:

We now have a doubly-nested sum involving zero, so we can apply rule #1, getting rid of the sum (2 + 0 = 2) and “unwinding” the successor functions to arrive at the result:

Again, if you’re interested in a more detailed walkthrough, including C code that explains the process in programmer-friendly terms, check out the article linked earlier on.

What we haven’t covered in that article is that we can use a similar approach to recursively define multiplication for natural numbers:

In effect, without having to explicitly define subtraction, we’re saying that a · b can be rewritten as a · (b-1) + a, and that this expansion should be continued until we get to a · 0. At that point, the multiplication part works out to zero, so we just unwind the stack and gather all the “+ a” terms. This will come in handy in a while.

Integers (ℤ)

A major hurdle on our path toward a complete system of arithmetic is that natural numbers can’t represent negative values. This means that if we attempt to define subtraction, many results will not have an in-system representation, throwing a wrench in the works.

To extend ℕ to negative numbers, we could futz around with a way to encode the minus sign and then special-case it in the arithmetic rulesets. That said, a better-behaved approach is to define integers as a separate hierarchy of numbers, each integer i consisting of an ordered pair of naturals: i = (a, b). The first element of the pair represents the positive component while the second represents the negative part. That is, integer +5 can be encoded as (5, 0) while integer -5 becomes (0, 5).

You might be wondering if we just pulled the concept of an “ordered pair” out of thin air. Yes and no: it’s new here, but in set theory, these pairs are mapped to normal sets, except we design the mapping so that (a, b) differs from (b, a). This can be done in a number of ways, but a common approach devised by Kazimierz Kuratowski is:

In essence, the pair is represented by a two-element set, but the second element also embeds a copy of the first, so the result is different depending on the order of the elements in the pair. In any case, the encoding is not important to what we’re about to do.

To define the addition of pair-based integers, we can simply add the “positive” and “negative” halves separately. Since the underlying elements are natural numbers, we already know how to add them, and we can write:

In the same vein, because each integer effectively expresses the difference between two underlying numbers, the result of multiplying two integers (a, b) and (c, d) will follow the school-taught pattern of (a - b) · (c - d) = ac - ad - bc + bd. We split out the positive and negative parts of the solution and write:

These rules work, sort of. For example, adding integer +5 and -5 nets us:

The result (5, 5) seems to be saying “zero”, but is not what we’d choose as the canonical representation of that value; we would have preferred (0, 0). More generally, we’d say that any two pairs (a, b) and (c, d) represent the same integer if a - b = c - d. Yet, our system doesn’t take this property into account.

Because we still haven’t defined subtraction, we must first shuffle the terms around to express the “sameness” criterion in terms of addition:

With this pair equivalence relationship defined, we assign integer labels such as “+5” not to a specific pair, but to an entire equivalence class: a collection of ordered pairs that satisfy the same criteria. In this model, our new hierarchy of numbers looks the following way:

We’re mostly done with integers, but before we wrap up, let’s ponder if the set of integers is “larger” than the set of natural numbers. By some metrics, you could argue it is. That said, as hinted in the earlier article, every integer can be mapped to a natural number without the risk of running out of naturals:

Because there’s no rule that stops us from taking this route, so we say that the sets have the same cardinality.

Rationals (ℚ)

Rational numbers are values that can be expressed as a ratio of two integers: a / b. In the previous section, we defined integers using an ordered pair that effectively encoded subtraction: a - b. So, here’s the cool part: nothing stops us from taking two integers and fashioning them into a new type of a pair that encodes division. Each of these integer pairs forms a new hierarchy of rational numbers: q = (a, b).

In this model, we consider rationals (a, b) and (c, d) to be equivalent if the underlying integers satisfy the criterion a / b = c / d. We don’t have division defined yet, but we know how to multiply integers, so we can restate the equivalence rule as:

This nets us the following taxonomy:

The multiplication rule for two pairs representing rational numbers is just a trivial restatement of a/b · c/d = ac/(bd):

The addition of rationals is equally straightforward and follows the normal a/b + c/d = (ad + cb) / (bd) pattern:

What’s the “size” of ℚ? Well, again, depends on how we look at it, but we can show that the cardinality of is not greater than ℕ. One visual approach is to construct a two-dimensional array of fractions in the form of x/y:

It should be evident that because x and y coordinates separately go through every possible natural number, the array contains all positive rational fractions. Some of the tiles are redundant (e.g., 2 is the same as 4/2), but this is not important for the proof.

With the rationals laid out, we can traverse this grid in a way that lets us assign every tile to an integer without leaving any gaps and without ever running out of members of ℕ. The start of one such traversal pattern is indicated by arrows in the figure. We begin in the top left corner, move one tile to the right, take a a sharp turn to and start moving diagonally until we hit the vertical edge, move one tile down, and then follow a diagonal pattern back. Rinse, repeat. By analogy to what we’ve done for integers, the result doesn’t change if we toss negative rationals into the mix.

Computable numbers

Not every number can be expressed as an integer fraction. The two examples of irrational numbers that every reader should be familiar with are √2, which can be expressed in polynomial terms, and π, which cannot.

Although these numbers can’t be represented as simple fractions, they can be explained in terms of an algorithm you need to follow to approximate them to an arbitrary degree. For example, the sum of the following terms starting at n = 0 will slowly but surely converge to π as the count of summed elements grows:

Within the bounds of the precision of floating-point numbers, you can observe the behavior by running the following C code (demo link):

#include <stdio.h>
#include <stdint.h>

int main() {
  double sum = 0;
  for (uint64_t n = 0, pos = 0; n <= 65536; n++) {
    sum += 8.0 / ((4*n + 1) * (4*n + 3));
    if (n >> pos) { printf("[%5ld] %.05f\n", n, sum); pos++; }
  }
}

At first blush, it would appear that any well-specified irrational number of our choice can be expressed as an approximation algorithm. This leads to a concept that should appeal to any geek: computable numbers. It’s the set of all numbers that can be approximated to an arbitrary precision in finite time by a theoretical model of a computer known as a Turing machine. In effect, the number is the algorithm.

Interestingly, the cardinality of the set of computable numbers is still the same as the cardinality of ℕ. An intuitive explanation is that there are only as many computable numbers as there are Turing machines that could generate them. The ruleset of every Turing machine can be encoded as a finite natural number — you could just write it down and then convert the spec to ASCII values — so we’re still in the realm of countable infinities.

Reals (ℝ)

Of course, we don’t teach about computable numbers in school. Instead, the most common “upgrade” from ℚ are reals: an idealized continuum on which, to put it in a hand-wavy way, every number exists whether we know how to algorithmically approximate it or not.

Now, my phrasing here is severely deficient. It’s not a free-for-all: 🥔 (a potato) is not a real number, and to avoid a variety of complications, neither is √-1. The set ℝ extends ℚ, but it does so only in the immediate vicinity of rationals. Pick any real and I can find a rational fraction that’s arbitrarily close.

To describe the underlying structure of real numbers, we can turn to Dedekind cuts. Informally, the idea is that we can unambiguously identify each real number by associating it with the set of all rationals that come before it. To describe real number x, we could take the set of rational numbers and partition it into an ordered pair of sets (A, B), such that set A contains every rational q < x and set B contains every q ≥ x.

This description may seem circular: to build the representation of a real number, we need to know where to make the cut, which seem to require some prior knowledge about how x relates to ℚ. That said, the point isn’t that the method lets you find the exact spot for π; it’s that the universe of real numbers is built by taking every possible Dedekind cut of ℚ. The numbers are the cuts, and π is somewhere out there, even if we can’t easily pinpoint its location.

To be fair, for some irrational numbers, we can describe the partition in pretty intuitive terms. For example, to describe the cut associated with ∛5, we can say that set A consists of every rational q such that q³ < 5 and set B contains every q such that q³ ≥ 5. It’s a pretty usable specification, but again: the existence of a real number doesn’t depend on us being able to say where it lies.

Once we have numbers expressed in terms of Dedekind cuts, we can define arithmetic operations on reals, too. For example, to add real (A, B) to (C, D), we construct a new number (E, F) such that for every possible rational a selected from A and every rational c selected from C, the sum a + c is placed in E. In the same vein, for every b in B and d in D, the sum b + d goes into F.

I’ll spare you the abstract set notation, but as a practical example, if (A, B) represents 2 and (C, D) represents 3, we know that every a selected from A will be less than 2 and every c chosen from C will be less than 3. Therefore, every a + c value placed in E will be always less than 5. Similarly, every b + d that goes into F will be greater or equal than 5. The resulting pair (E, F) is therefore the same as the cut representing the number 5.

We now have a continuum that contains numbers that are allowed to exist regardless of whether they can be described by an effective, finite procedure. As an unexpected consequence, the cardinality of ℝ is higher than ℕ.

We explored this property in the earlier article, but to briefly recap the argument, imagine an arbitrary, infinite mapping that assigns every integer to a real:

For every real number in the mapping, I underlined a successive decimal position. Equipped with this, we can imagine a new real that could be built by looking at each of the underlined digits and then placing a different digit in the corresponding position of the newly-constructed value.

By construction, our new real necessarily differs by at least one digit from every existing entry in the mapping. This means we still have a value — or really, an infinite supply of values — left over after assigning every natural number to a real. It would appear that there is a fundamentally higher “number” of reals than naturals — an uncountable infinity.

So what?

Well… from the earlier discussion, recall that the cardinality of computable numbers was the same as the cardinality of ℕ. The cardinality of ℝ — the “magnitude” of the set of reals — is fundamentally greater than that. In other words, we could assert that most reals are uncomputable.

But what would be an example of an uncomputable number? That’s a good question. Most obviously, we could be talking about numbers that encode the solution to the halting problem. It would lead to a paradox to have a computer program that allows us to decide, in the general case, whether some other computer program halts. So, if a procedure to approximate a particular real requires solving the halting problem, we can’t have that.

If you’re interested in a more thorough exploration of the idea, check out my earlier article on busy beavers and the limits of algorithmic knowledge. But to get to the point, there are those who believe that the universe is functionally a computer — that is, that its rules are deterministic and can be simulated by a Turing machine. If so, that would imply that uncomputable numbers can’t be zeroed in on by any physical process, including human thought. They would be truly out of reach… and again, this would apply to almost every member of ℝ.

Cue the Twilight Zone theme music — and see you in a bit.

Further reading in the series:

Gödel's beavers, or the limits of knowledge

lcamtuf

·

June 30, 2025

Read full story

I write well-researched, original articles about geek culture, electronic circuit design, algorithms, and more. If you like the content, please subscribe.

If you follow information security discussions on the internet, you might have heard that blurring an image is not a good way of redacting its contents. This is supposedly because blurring algorithms are reversible.

But then, it’s not wrong to scratch your head. Blurring amounts to averaging the underlying pixel values. If you average two numbers, there’s no way of knowing if you’ve started with 1 + 5 or 3 + 3. In both cases, the arithmetic mean is the same and the original information appears to be lost. So, is the advice wrong?

Well, yes and no! There are ways to achieve non-reversible blurring using deterministic algorithms. That said, in some cases, blur filters can preserve far more information than would appear to the naked eye — and do so in a pretty unexpected way. In today’s article, we’ll build a rudimentary blur algorithm and then pick it apart.

One-dimensional moving average

If blurring is the same as averaging, then the simplest algorithm we can choose is a single-axis moving mean. In photo editing software, this filter is commonly called motion blur and is fairly similar to the artifacts produced by rapid object movement or camera shake.

To implement the effect, we take a fixed-size window and replace each pixel value with the arithmetic mean of n pixels in its neighborhood. For n = 5, the process is shown below:

Note that for the first two cells, we don’t have enough pixels in the input buffer. We can use fixed padding, “borrow” some available pixels from outside the selection area, or simply average fewer values near the boundary. Either way, the analysis doesn’t change much: the presence of this boundary, along with the discrete stepover of the averaging window, leaks information about the underlying image.

To illustrate, let’s assume that we’ve completed the blurring process and no longer have the original pixel values. To reconstruct the data, we start at the left boundary (x = 0). Recall that we calculated the first blurred pixel like by averaging the following pixels in the original image:

Next, let’s have a look at the blurred pixel at x = 1. Its value is the average of:

We can easily turn these averages into sums by multiplying both sides by the number of averaged elements (5):

Note that the underlined terms repeat in both expressions; this means that if we subtract the expressions from each other, we end up with just:

The value of img(-2) is known to us: it’s one of the fixed padding pixels used by the algorithm. Let’s shorten it to c. We also know the values of blur(0) and blur(1): these are the blurred pixels that can be found in the output image. This means that we can rearrange the equation to recover the original input pixel corresponding to img(3):

We can also apply the same reasoning to the next pixel:

At this point, we seemingly hit a wall with our five-pixel average, but the knowledge of img(3) allows us to repeat the same analysis for the blur(5) / blur(6) pair a bit further down the line:

This nets us another original pixel value, img(8). From the earlier step, we also know the value of img(4), so we can find img(9) in a similar way. This process can continue to successively reconstruct additional pixels, although we end up with some gaps. For example, following the calculations outlined above, we still don’t know the value of img(0) or img(1).

These gaps can be resolved with a second pass that moves in the opposite direction in the image buffer. That said, instead of going down that path, we can also make the math a bit more tidy with a harmless, good-faith tweak to the averaging algorithm in a way that doesn’t change the nature of the effect.

Right-aligned moving average

The modification that will make our life easier is to shift the averaging window so that one of its ends is aligned with where the computed value will be stored:

In this model, the first output value is an average of four fixed padding pixels (c) and one original image pixel; it follows that in the n = 5 scenario, the underlying pixel value can be computed as:

If we know img(0), we now have all but one of the values that make up blur(1), so we can find img(1):

The process can be continued iteratively, reconstructing the entire image — this time, without any discontinuities and without the need for a second pass.

In the illustration below, the left panel shows a detail of The Birth of Venus by Sandro Botticelli; the right panel is the same image ran through the right-aligned moving average blur algorithm with a 151-pixel averaging window that moves only in the x direction:

Now, let’s take the blurry image and attempt the reconstruction method outlined above — computer, ENHANCE!

This is rather impressive. The image is noisier than before as a consequence of 8-bit quantization of the averaged values in the intermediate blurred image. Nevertheless, even with a large averaging window, fine detail — including individual strands of hair — could be recovered and is easy to discern.

Into the second dimension

One obvious problem with our blur algorithm is that it averages pixel values only in the x axis; as mentioned earlier, this roughly approximates motion blur or camera shake.

The approach we’ve developed can be trivially turned into a 2D filter by using a two-dimensional averaging window; if we use a square region, this is called box blur. That said, a more expedient and functionally similar hack is to apply the existing 1D filter in the x axis and then follow with a complementary pass in the y axis. To undo the blur, we’d then perform two recovery passes in the inverse order.

Unfortunately, whichever route we take, we’ll discover that the combined amount of averaging per pixel causes the underlying values to be quantized so severely that the reconstructed image is overwhelmed by noise unless the blur window is relatively small. For the 1D + 1D method, we get:

That said, if we wanted to develop an adversarial box blur filter, we could fix the problem by weighting the original pixel a bit more heavily in the calculated mean. For the x-then-y variant, if the averaging window has a size W and the current-pixel bias factor is B, we can write the following formula:

This filter still does what it’s supposed to do; here’s the output of an x-then-y blur for W = 200 and B = 30:

Surely, there’s no coming back from tha— COMPUTER, ENHANCE!

Remarkably, the information “hidden” in the blurred images survives being saved in a lossy image format. The top row shows images reconstituted from an intermediate image saved as a JPEG at 95%, 85%, and 75% quality settings:

The bottom row shows less reasonable quality settings of 50% and below; at that point, the reconstructed image begins to resemble abstract art.

Do I need to worry?

Maybe? As noted earlier, even in the case of simple box blur, the recovery is less successful for large, two-dimensional blur windows, unless we tip the scales in our favor by tinkering with the algorithm.

The likelihood of success will be also reduced if you use a more sophisticated blur filter. Virtually all practical implementations of blur algorithms work in a similar way, so they leak some information due to presence of blur region boundaries and the discrete stepover of a finite averaging window. That said, they use different weighting for pixel values, which complicates the math and can make recovery borderline impossible.

On the flip side, for constrained data such as text, pixel-level reconstruction may be unnecessary to begin with. Instead, regions of the blurred image can be successively compared to the blurred representations of every possible symbol. In this scenario, it doesn’t help that the transformation is technically irreversible. It suffices that there’s a finite alphabet of shapes and that they produce distinct output images.

👉 For more articles about math, visit this page. In particular, you might enjoy:

See it with your lying ears

lcamtuf

·

Jan 10

Read full story

You gotta think outside the hypercube

lcamtuf

·

Jan 26

Read full story

How many dimensions is this?

lcamtuf

·

September 3, 2025

Read full story

Gödel's beavers, or the limits of knowledge

lcamtuf

·

June 30, 2025

Read full story

I write well-researched, original articles about geek culture, electronic circuit design, algorithms, and more. If you like the content, please subscribe.

If you’re a nerd, you probably have encountered visualizations of a tesseract: a four-dimensional equivalent of a cube. Heck, various representations of the shape have made it into blockbuster sci-fi films, music videos, and more.

What might be harder to grasp is what these images mean or how they’re generated. You can find a handful of tesseract rendering demos on GitHub, but they all take different approaches, produce different results, and don’t really explain what’s going on.

In this article, we’ll take a look at the hypercube from first principles — and then, figure out how to map this beast to a computer screen.

It’s hip to be square

To build a mathematical model of the hypercube, let’s start with a square. If we get it right, our approach will generalize to three dimensions and produce a cube; if it does, it ought to extend to the hyperspace too.

More specifically, we’ll try to model the edges of a square — i.e., the line segments that connect the vertices in the four corners. For our purposes, a see-through wireframe model will work better than a solid.

For a 2D square with dimensions a×a, the horizontal edges can be described as a collection of points that satisfy two criteria:

In essence, we’re saying that we want to include points for which the y coordinate is equal to either -a (the lower edge) or +a (the upper edge); and where the x coordinate spans anywhere between -a and +a:

To construct the remaining vertical edges, we can just flip the criteria around, constraining x to one of two values and allowing y to span a range. This nets us the following combined formula:

The method is easy to generalize to a cube. We start by constructing a rectangle in the x-y plane using the earlier approach, except we add a third modulo constraint so that we end up with two images at the -a and +a offsets in the z axis:

What’s still missing are four edges oriented in the z direction that connect the corresponding corners of the two squares. We can add this with a third rule:

Note that each of these rules produces four line segments because there 2² possible combinations for the coordinates constrained by the equality relationship. For example, for horizontal lines, we can have the following pairs of y and z values: (-a, -a), (-a, +a), (+a, -a), and (+a, +a).

From here, the extension to the fourth dimension should be clear. I’m going to sensibly label the dimension 🌀; with this done, we just add a fourth constraint to each of the existing 3D rules and then add connecting segments in the fourth dimension:

This time around, each rule nets us 2³ = 8 line segments, so the tesseract has 4·8 = 32 edges. These edges connect 16 vertices.

Defining rotations

Most visualization of the tesseract spin the figure around. This allows the shape to be examined from different angles and makes for some mind-bending visuals. But what does it mean to rotate an object in 4D?

In a two-dimensional space, there’s only one type of rotation; it transposes coordinates in the XY plane. The following demonstrates the effect of rotating a point originally placed on the x axis around the center of the coordinate system:

From basic trigonometry, the new x coordinate of the rotated point is the adjacent of a right triangle with an angle α and a hypotenuse of x_orig. The new y is the opposite of that same triangle. If we want to start with a non-zero y coordinate for the point, we need add a small tweak:

In three dimensions, we have a lot more freedom. We can obviously spin things around in the XY plane (around the z axis), XZ (around y), or in YZ (around x). It is also possible to dream up more complex rotations that touch all three coordinates at once, but they don’t add much value. They can be deconstructed into a sequence of planar rotations in XY, XZ, and YZ.

Given this observation, in four dimensions, we should probably still stick to the primitive of planar rotations that modify just two axes at a time. The only difference is that we get additional X🌀, Y🌀, and Z🌀 planes to use.

For ease of viewing and for consistency with 3D models, we’ll focus on spinning things in the XZ plane — a “turntable” animation:

That said, we’ll also pay a brief visit the Z🌀 rotation plane. It plays by similar rules:

Projecting 4D to 2D

Our next challenge is figuring out how to project four-dimensional coordinates onto a two-dimensional drawing surface, such as a computer screen.

In standard geometries, Cartesian axes are orthogonal to each other — that is, there is a 90° rotation that can take you between any two dimensions. On a two-dimensional surface, we can only pull this off with two axes; that said, there are several imperfect ways to make do.

Cavalier projection

If we were to ask a random person to come up with a 3D-to-2D projection on the spot, they would probably suggest drawing the z axis as a diagonal line on a 2D plane, as shown below:

To convert the model-space z value to screen coordinates, we can use trigonometry to project the component onto the real x and y axes:

Alas, if you attempt this projection with a regular three-dimensional cube, you will immediately notice that it looks off:

The viewer-facing edges in the XY plane are exactly the same length as the z edge; nevertheless, it’s hard to shake the impression that the dimensions are off and the cube is stretched.

Cabinet projection

To address this issue, we need to shorten the projected z-axis edges, crudely approximating the length contraction that we expect in real life. To do this, we divide the z component by an ad hoc scaling factor, typically 2:

The cabinet projection is ubiquitous in informal sketches and technical drawings, and it does look good at first blush. That said, consider the following video of a cube that is rotated in the XZ plane:

Note that the shape looks OK at first, but then gets weirdly squished near the rotation angle of 70°; this is because the projection gives us incorrect visual cues that the shape is facing us — the back edges are tucked squarely behind while in reality, the shape is still at an angle in relation to the viewer.

The root of the problem is that the axes are not oriented in a way that would be possible in real life. If we constructed a model of 3D axes out of sticks, the only way for the z axis to appear at a 45° angle — or indeed, to be visible at all — is if at least one of the other axes is not parallel to the camera plane:

Isometric projection

This brings us to the isometric projection — a physically-plausible arrangement that places the model axes 60° apart:

The math for this projection is still simple. The screen x coordinate is dictated by model x multiplied by cos(30°) — that’s the angle between the model x axis and the real one. The value is also influenced in the same way but with an opposite sign by the model z axis, so we get:

Meanwhile, on the y side, we need to account for the projected sine component of x and z:

Both cosine expressions can be further divided by √2 if the goal is to match the model- and screen-lengths of a horizontal line drawn in the model XY plane and then rotated by 45° around the y axis. That said, it’s seldom a necessity.

The following video shows a cube rotated in the XZ plane in an isometric projection:

This looks great and it seems natural to extend the scheme to four dimensions simply by cramming another axis, giving us a progression of x, y, z, and 🌀 axes spaced 45° apart:

Yet, some readers might notice that with this modification, we’re back to the earlier “cavalier” scenario: our x, y, and z axes are now separated by an impossible angle of 45°. In other words, the projection should give us something, but it will distort some 3D shapes in undesirable ways.

Still, let’s keep going. In the new model, we calculate screen x as:

The projected model y axis is orthogonal to to screen x, so it doesn’t appear in this formula. As for the y coordinate, we need:

As before, since the projected model 🌀 axis is orthogonal to screen y, it doesn’t appear in the second equation.

Let’s put this projection to real use. Here’s the video of a tesseract rotating in the XZ plane:

It looks pretty, but it isn’t particularly informative: the projection makes the object change shape in ways that seem difficult to parse. The shape appears to be intersecting itself, but it’s hard to pinpoint what’s what.

Rectilinear one-point perspective

A simpler but surprisingly powerful projection method is to keep model x and y in the same plane as the screen, but divide the values of these coordinates in proportion to the distance in z. This produces a familiar vanishing-point perspective:

A fairly natural extension of this technique to the fourth dimension is to divide the x and y coordinates twice: first by a z-dependent factor and then by a 🌀-dependent one. This nets probably the most recognizable visualization of a tesseract:

If you want food for thought, consider the real-world appearance of a wireframe 3D cube when its shadow from a nearby overhead light is cast onto a 2D surface:

This both helps make some sense of the nested-cube visualization of the tesseract, and signals that our algorithm is directionally correct. That said, the approach we’ve taken is also a bit of a cop-out: by commingling model z and 🌀, we make these dimensions indistinguishable.

Fisheye perspective

At first blush, the tesseract visualization might look just like two nested 3D cubes connected in the corners. To reduce edge overlaps and better hint at the underlying shenanigans, we can switch to a curvilinear “fisheye” perspective, reminiscent of what you can see through a peephole or other low-quality, wide-angle lens. In this approach, point coordinates are reduced based on their Euclidean distance from a single reference point representing the camera. For a regular cube, we get:

But of course, we’re here to look at the tesseract:

The shading and the drawing order of the points is decided by the Euclidean distance to the viewing plane; this allows us to spot that the edges of the smaller, “inner” cube appear to pass behind the edges of the larger one:

Still, as noted earlier, the disappointing part of the mapping is that it commingles two dimensions; can we distinguish them better without ending up with a visual disaster?

Mixed isometric + vanishing point

Sort of?… Instead of trying to come up with a single projection for all four axes, we could always use a conventional isometric view for x, y, and z, and then use the vanishing-point approach to represent 🌀.

The result is a remarkably stable and easy-to-parse view of the tesseract when rotated in the YZ plane:

This also brings us to a somewhat less-correct rendering of the hypercube spinning in the Z🌀 plane that can be found on Wikipedia and in some YouTube videos. If we change screen depth calculations to only account for the z coordinate (i.e., completely ignore model 🌀), we obtain the following:

If you squint your eyes, this appears to show the tesseract passing through itself back-to-front as it rotates in the fourth dimension. I altered the proportions of the projection to make the effect easier to see.

👉 For more articles about math, visit this page.

I write well-researched, original articles about geek culture, electronic circuit design, algorithms, and more. If you like the content, please subscribe.

When writing a technical blog, the first 90% of every article is a lot easier than the final 10%. Sometimes, the challenge is collecting your own thoughts; I remember walking through the forest and talking to myself about the articles about Gödel’s beavers or infinity. Other times, the difficulty is the implementation of an idea. I sometimes spend days in the workshop or writing code to get, say, the throwaway image of a square-wave spectrogram at the end of a whimsical post.

That said, by far the most consistent challenge is art. Illustrations are important, easy to half-ass, and fiendishly difficult to get right. I’m fortunate enough that photography has been my lifelong hobby, so I have little difficulty capturing good photos of the physical items I want to talk about:

Similarly, because I’ve been interested in CAD and CAM for nearly two decades, I know how to draw shapes in 3D and know enough about rendering tech to make the result look good:

Alas, both approaches have their limits. Photography just doesn’t work for conceptual diagrams; 3D could, but it’s slow and makes little sense for two-dimensional diagrams, such as circuit schematics of most function plots.

Over the past three years, this forced me to step outside my comfort zone and develop a new toolkit for simple, technical visualizations. If you’re a long-time subscriber, you might have seen the changing art style of the posts. What you probably don’t know is that I often revise older articles to try out new visualizations and hone in my skills. So, let’s talk shop!

Circuit schematics

Electronic circuits are a common theme of my posts; the lifeblood of this trade are circuit schematics. I’m old enough to remember the beautiful look of hand-drawn schematics in the era before the advent of electronic design automation (EDA) software:

Unfortunately, the industry no longer takes pride in this craft; the output from modern schematic capture tools, such as KiCad, is uniformly hideous:

I used this style for some of the electronics-related articles I published in the 2010s, but for this Substack, I wanted to do better. This meant ditching EDA for general-purpose drawing software. At first, I experimented with the same CAD software I use for 3D part design, Rhino3D:

This approach had several advantages. First, I was already familiar with the software. Second, CAD tools are tailored for technical drawings: it’s a breeze to precisely align shapes, parametrically transform and duplicate objects, and so forth. At the same time, while the schematics looked more readable, they were nothing to write home about.

In a quest for software that would allow me to give the schematics a more organic look, I eventually came across Excalidraw. Excalidraw is an exceedingly simple, web-based vector drawing tool. It’s limited and clunky, but with time, I’ve gotten good at working around many of its flaws:

What I learned from these two tools is that consistency is key. There is a temptation to start every new diagram with a clean slate, but it’s almost always the wrong call. You need to develop a set of conventions you follow every time: scale, line thickness, font colors, a library of reusable design elements to copy-and-paste into new designs. This both makes the tool faster to use — rivaling any EDA package — and allows you to refine the style over time, discarding failed ideas and preserving the tricks that worked well.

This brings us to Affinity. Affinity is a “grown-up” image editing suite that supports bitmap and vector files; I’ve been using it for photo editing ever since Adobe moved to a predatory subscription model for Photoshop. It took me longer to figure out the vector features, in part because of the overwhelming feature set. This is where the lessons from Rhino3D and Excalidraw paid off: on the latest attempt, I knew not to get distracted and to focus on a simple, reusable workflow first.

This allowed me to finally get in the groove and replicate the hand-drawn vibe I’ve been after. The new style hasn’t been featured in any recent articles yet, but I’ve gone ahead and updated some older posts. For example, the earlier microphone amplifier circuit now looks the following way:

Explanatory illustrations

Electronic schematics are about the simplest case of technical illustrations. They’re just a map of connections between standard symbols, laid out according to simple rules. There’s no need to make use of depth, color, or motion.

Many other technical drawings aren’t as easy; the challenge isn’t putting lines on paper, it’s figuring out the most effective way to convey the information in the first place. You need to figure out which elements you want to draw the attention to, and how to provide visual hints of the dynamics you’re trying to illustrate.

I confess that I wasn’t putting much thought into it early on. For example, here’s the original 2024 illustration for an article on photodiodes:

It’s not unusable, but it’s also not good. It’s hard to read and doesn’t make a clear distinction between different materials (solid color) and an electrical region that forms at the junction (hatched overlay).

Here’s my more recent take:

Once again, the trick isn’t pulling off a single illustration like this; it’s building a standardized workflow that lets you crank out dozens of them. You need to converge on backgrounds, line styles, shading, typefaces, arrows, and so on. With this done, you can take an old and janky illustration, such as the following visual from an article on magnetism:

…and then turn it into the following:

As hinted earlier, in many 2D drawings, it’s a challenge to imply a specific three-dimensional order of objects or to suggest that some of them are in motion. Arrows and annotations don’t always cut it. After a fair amount of trial and error, I settled on subtle outlines, nonlinear shadows, and “afterimages”, as shown in this illustration of a simple rotary encoder:

The next time you see a blog illustration that doesn’t look like 💩 and wasn’t cranked out by AI, remember that more time might have gone into making that single picture than into writing all of the surrounding text.