Tags: howtos, research

« Previous post: Maintaining your mental health in … — Next post: A visual introduction to Morse theory »

In machine learning, the term manifold is thrown around a lot—in fact, there’s even a branch of machine learning dealing with learning the structure of manifolds. Said branch is aptly named manifold learning. Inspired by fantastic visualizations of shapelet mining algorithms, this post aims to give a visual introduction to manifolds that is accessible for non-mathematicians and mathematicians alike.

Let’s start with a rough definition: a $d$-dimensional manifold is something that locally looks like a $d$-dimensional Euclidean space, i.e. like some $\mathbb{R}^d$. To show that this is less complicated than it sounds, we need to build some intuition by providing examples. First of all, when mathematicians say that something ’locally looks’ like something else, they consider viewing that object from the perspective of an extremely tiny bug that ’lives’ on said object (and cannot move away from it—but more about that later).

Dimension 1: Moving along lines

According to the definition above a $1$-dimensional manifold would thus be something that ’locally looks’ like $\mathbb{R}^1$. But it turns out that we know $\mathbb{R}^1$ or $\mathbb{R}$ rather well: it is nothing but the line of real numbers¹. This is how it looks from the ‘outside’:

And this is how it looks from the perspective of a small bug ’living’ on said line:

Hence, from the perspective of the bug, the line stretches onwards to infinity, i.e. into your screen, without having any other extents. In fact, the bug can only move ‘forward’ and ‘backward’, as there are no other directions. The number line is thus an example of a $1$-dimensional manifold. However, it is an exceptionally boring example. We can find a better one by considering a circle. Here it is, shown with its small inhabitant:

If you think about it, from the point of the bug, the circle and number line ’look’ the same. The bug can move forward or backward; there’s no way of telling² whether it is living on $\mathbb{R}$ or on $S^1$, as mathematicians like to call the circle. The reason for this notation is that the circle is seen as a $1$-dimensional sphere. Notice that mathematicians do not consider a sphere to be filled—it is only the outer line that we are interested in³. Think of the surface of a balloon (we will come to that presently) rather than its interior.

Dimension 2: Plain planes

Moving on to the second dimension, we are—according to our definition—dealing with an object that locally looks like $\mathbb{R}^2$, the two-dimensional Euclidean space. It turns out that we do know this space also from high school as the Euclidean plane, or a Cartesian coordinate system. Usually, you denote the two axes with $x$ and $y$:

If your education went anything like mine, you spent a lot of hours, some happy and some unhappy, by adding mathematical objects to these two axes. From the perspective of our small bug, $\mathbb{R}^2$ looks like this:

In other words, the bug can move in two independent directions here, and its world will appear to be a large plane, much like the Great Plains would appear to us. Of course, the world of the bug does not have any edges. Again, this is a somewhat boring example of a $2$-manifold⁴. To get something better, we can extend our previous example of the $1$-sphere into the $2$-sphere $S^2$:

That looks awfully familiar—in fact, the $2$-sphere is nothing else than the surface of a ‘ball’ in a $3$-dimensional space, just like the $1$-sphere is the surface of a ‘disk’ in a $2$-dimensional space (note the nice and symmetrical shift in dimensions here). Interestingly, our poor bug has no easy way of checking whether it is living on a sphere or on an infinite plane because it cannot move outside, i.e. it does not have access to rocket technology. Moreover, we assume that the bug is really small and does not notice the curvature of the land. One might say that the bug has good reasons to believe in a flat earth because everything is flat from its perspective.

Dimension 3: We know this!

At this point, you probably know where this is going, and you shall not be disappointed: let us go to the third dimension, then! To represent three dimensions, we again use a Cartesian coordinate system:

The order of the axes does not matter, but I like $y$ to point upwards because I think of it as a ‘height’. Three dimensions should sound somewhat familiar to you—it turns out that this time, we are the bug! From our perspective, our universe has three spatial dimensions⁵, at least locally, so we are probably living in (on?) a $3$-manifold. It is thus our turn to try to discover some facts about the manifold we are living in, i.e. the shape of the universe.

Given that our universe might be a $3$-manifold, what examples of these manifolds are there? Again, the classical one would be the $3$-sphere $S^3$. However, if you followed the discussion from above, you know that the $3$-sphere ’lives’ in $\mathbb{R}^4$, a $4$-dimensional space. These are typically hard to visualize—but it turns out that there is a neat way to describe spheres of all dimensions. To see this, let’s start with $d = 1$, the circle. With a radius of $r = 1$, also known as unit radius, the circle contains all points that satisfy $x^2

• y^2 = 1$:

These equations can be extended to higher dimensions without so much as batting an eyelid! Hence, points on $S^2$, the $2$-sphere, are characterized by satisfying $x^2 + y^2 + z^2 = 1$, while points on $S^3$, the $3$-sphere, are characterized through $x^2 + y^2 + z^2 + w^2 = 1$. In other words, we need four coordinates to describe them. While this is not surprising, it at least gives us an intuitive glimpse about how to treat these objects analytically, even if we are incapable of visualizing them correctly. At least at this point, you are now able to describe the coordinates of any $d$-dimensional manifold.

How to tell one manifold from another

All this knowledge about manifolds now needs to be put to some use. Already in the $1$-dimensional example, we asked the question how a bug might be able to figure out that they are living in $S^1$ or in $\mathbb{R}$. One easy algorithm requires only two ingredients:

1. An infinitely long rope
2. An infinite amount of time

By the description of these two ingredients we can already see that we are now in mathematical territory! The algorithm is straightforward: a bug that wants to figure out the shape of its manifold merely needs to ‘fasten’ the rope at one point and ‘unspool’ it. Next, it merely needs to walk on in a randomly-selected direction, which must never be changed, though. If the bug is living in $S^1$, at some point⁶, it will stumble over its rope. At this point, the algorithm can terminate and the poor bug can finally rest.

Note that this algorithm will only terminate if the bug is indeed living in some $S^1$, no matter how large its radius. If the bug is indeed living in $\mathbb{R}$, its journey will continue forever. A somewhat dire fate. How might we extend this to $d = 2$? Obviously, the walking is a little bit more difficult here, as there are two independent directions. Nonetheless, if the bug suspects that it is living in $S^2$, it could pick one direction at random, refer to it as ‘North’, and follow it. At some point, it will re-cross its path and thus figure out the shape of its universe.

There is a more elegant way, though. If you think back to high school mathematics, you probably proved that the angles in a triangle sum to 180 degrees⁷. It turns out that you can construct a triangle whose angle sum is 270 degrees, provided you are living in $S^2$, and not in $\mathbb{R}^2$. Thus, our little bug could boldly declare that its current location is the pole of its universe and follow these steps:

1. Walk some distance in a straight line. This can be ensured by (again) using copious amounts of rope.
2. Turn left by 90 degrees. This requires a bug-sized protractor, for example, but we can imagine that a bug civilization that is smart enough to develop mathematics is also smart enough to build tools for their species.
3. Walk some distance in a straight line again.
4. Turn left by 90 degrees and get back to where the bug started.

If this is repeated multiple times and the bug decides to use a sufficiently large distance, it will be possible to describe—or rather inscribe—a triangle with three right angles. Since this cannot work in the plane, the bug is left with the conclusion that its civilization is living in $S^2$! Here is an illustration of how such a triangle might look:

The bug does not even have to go to that extreme, though, because it is perfectly sufficient to find a single triangle whose angle sum is more than 180 degrees in order to conclude that one is not living in $\mathbb{R}^2$. Of course, if the side lengths are sufficiently small, the effects of living in such a curved space are negligible.

This little thought experiment illustrates curvature, which is one of the fundamental concepts for describing manifolds. If you enjoyed this foray into the fascinating world of manifolds, I can recommend the book The Shape of Space by Jeffrey R. Weeks. His writing style is very entertaining and on geometrygames.org, he provides numerous programs to visualize and experience numerous manifolds. If you ever wanted to fly through a manifold or play mazes on a torus, this is the perfect website for you.

Until next time!