# What is a manifold?

## Tags: howtos, research

In machine learning, the term *manifold* is thrown around a lot—in
fact, there’s even a branch of machine learning dealing with learning
the structure of manifolds. Said branch is aptly named *manifold
learning*.
Inspired by fantastic visualizations of shapelet mining algorithms, this post aims to give a visual introduction to manifolds that is accessible for non-mathematicians and mathematicians alike.

Let’s start with a rough definition: a $d$-dimensional manifold is
something that *locally looks* like a $d$-dimensional Euclidean space,
i.e. like some $\mathbb{R}^d$. To show that this is less
complicated than it sounds, we need to build some intuition by providing
examples. First of all, when mathematicians say that something ‘locally
looks’ like something else, they consider viewing that object from the
perspective of an extremely tiny bug that ‘lives’ on said
object (and cannot move away from it—but more about that
later).

# Dimension 1: Moving along lines

According to the definition above a $1$-dimensional manifold would thus
be something that ‘locally looks’ like $\mathbb{R}^1$. But it turns out
that we know $\mathbb{R}^1$ or $\mathbb{R}$ rather well: it is nothing but
the line of real numbers^{1}. This is how it looks from the ‘outside’:

And this is how it looks from the perspective of a small bug ‘living’ on said line:

Hence, from the perspective of the bug, the line stretches onwards to
infinity, i.e. *into* your screen, without having any other
extents. In fact, the bug can only move ‘forward’ and ‘backward’, as
there are no other directions. The number line is thus an example of
a $1$-dimensional manifold. However, it is an exceptionally *boring*
example. We can find a better one by considering a *circle*. Here it is,
shown with its small inhabitant:

If you think about it, from the point of the bug, the circle and number
line ‘look’ the same. The bug can move forward or backward; there’s no
way of telling^{2} whether it is living on $\mathbb{R}$ or on $S^1$, as
mathematicians like to call the circle. The reason for this notation is
that the circle is seen as a $1$-dimensional *sphere*. Notice that
mathematicians do *not* consider a sphere to be filled—it is only
the *outer* line that we are interested in^{3}. Think of the surface of
a balloon (we will come to that presently) rather than its interior.

# Dimension 2: Plain planes

Moving on to the second dimension, we are—according to our
definition—dealing with an object that locally looks like
$\mathbb{R}^2$, the two-dimensional Euclidean space. It turns out that
we *do* know this space also from high school as the Euclidean plane,
or a *Cartesian coordinate system*. Usually, you denote the two axes
with $x$ and $y$:

If your education went anything like mine, you spent a lot of hours, some happy and some unhappy, by adding mathematical objects to these two axes. From the perspective of our small bug, $\mathbb{R}^2$ looks like this:

In other words, the bug can move in *two* independent directions here,
and its world will appear to be a large plane, much like the Great
Plains would appear to us.
Of course, the world of the bug does not have any edges. Again, this is
a somewhat boring example of a $2$-manifold^{4}. To get something
better, we can extend our previous example of the $1$-sphere into the
$2$-sphere $S^2$:

That looks awfully familiar—in fact, the $2$-sphere is nothing
else than the surface of a ‘ball’ in a $3$-dimensional space, just like
the $1$-sphere is the surface of a ‘disk’ in a $2$-dimensional
space (note the nice and symmetrical shift in dimensions here).
Interestingly, our poor bug has no easy way of checking whether it is
living on a sphere or on an infinite plane because it cannot move
*outside*, i.e. it does not have access to rocket technology.
Moreover, we assume that the bug is really small and does not notice the
curvature of the land. One might say that the bug has good reasons to
believe in a flat earth because everything is flat from its perspective.

# Dimension 3: We know this!

At this point, you probably know where this is going, and you shall not be disappointed: let us go to the third dimension, then! To represent three dimensions, we again use a Cartesian coordinate system:

The order of the axes does not matter, but I like $y$ to point upwards
because I think of it as a ‘height’. Three dimensions should sound
somewhat familiar to you—it turns out that this time, *we* are
the bug! From our perspective, our universe has three spatial
dimensions^{5}, at least *locally*, so we are probably living
in (on?) a $3$-manifold. It is thus *our* turn to try to discover
some facts about the manifold we are living in, i.e. the shape
of the universe.

Given that our universe *might* be a $3$-manifold, what examples of
these manifolds are there? Again, the classical one would be the
$3$-sphere $S^3$. However, if you followed the discussion from above, you know
that the $3$-sphere ‘lives’ in $\mathbb{R}^4$, a $4$-dimensional space.
These are typically hard to visualize—but it turns out that there
is a neat way to describe spheres of *all* dimensions. To see this,
let’s start with $d = 1$, the circle. With a radius of $r = 1$, also
known as unit radius, the circle contains all points that satisfy $x^2
+ y^2 = 1$:

These equations can be extended to higher dimensions without so much as
batting an eyelid! Hence, points on $S^2$, the $2$-sphere, are
characterized by satisfying $x^2 + y^2 + z^2 = 1$, while points on
$S^3$, the $3$-sphere, are characterized through $x^2 + y^2 + z^2 + w^2
= 1$. In other words, we need *four* coordinates to describe them. While
this is not surprising, it at least gives us an intuitive glimpse about
how to treat these objects analytically, even if we are incapable of
visualizing them correctly. At least at this point, you are now able to
describe the coordinates of *any* $d$-dimensional manifold.

# How to tell one manifold from another

All this knowledge about manifolds now needs to be put to some use. Already in the $1$-dimensional example, we asked the question how a bug might be able to figure out that they are living in $S^1$ or in $\mathbb{R}$. One easy algorithm requires only two ingredients:

- An infinitely long rope
- An infinite amount of time

By the description of these two ingredients we can already see that we
are now in mathematical territory! The algorithm is straightforward: a
bug that wants to figure out the shape of its manifold merely needs to
‘fasten’ the rope at one point and ‘unspool’ it. Next, it merely needs
to walk on in a randomly-selected direction, which must never be
changed, though. If the bug is living in $S^1$, at some point^{6}, it will
stumble over its rope. At this point, the algorithm can terminate and
the poor bug can finally rest.

Note that this algorithm will only terminate *if* the bug is indeed
living in some $S^1$, no matter how large its radius. If the bug is
indeed living in $\mathbb{R}$, its journey will continue forever. A
somewhat dire fate. How might we extend this to $d = 2$? Obviously,
the walking is a little bit more difficult here, as there are *two*
independent directions. Nonetheless, if the bug suspects that it is
living in $S^2$, it could pick one direction at random, refer to it
as ‘North’, and follow it. At some point, it will re-cross its path
and thus figure out the shape of its universe.

There is a more elegant way, though. If you think back to high school
mathematics, you probably proved that the angles in a triangle sum to
180 degrees^{7}. It turns out that you can construct a triangle whose
angle sum is 270 degrees, provided you are living in $S^2$, and *not*
in $\mathbb{R}^2$. Thus, our little bug could boldly declare that its
current location is *the* pole of its universe and follow these steps:

- Walk some distance in a straight line. This can be ensured by (again) using copious amounts of rope.
- Turn left by 90 degrees. This requires a bug-sized protractor, for
example, but we can imagine that a bug civilization that is smart
enough to develop mathematics is
*also*smart enough to build tools for their species. - Walk some distance in a straight line again.
- Turn left by 90 degrees and get back to where the bug started.

If this is repeated multiple times and the bug decides to use
a sufficiently large distance, it will be possible to describe—or
rather *inscribe*—a triangle with three right angles. Since this
cannot work in the plane, the bug is left with the conclusion that its
civilization is living in $S^2$! Here is an illustration of how such
a triangle might look:

The bug does not even have to go to *that* extreme, though, because it
is perfectly sufficient to find a *single* triangle whose angle sum is
more than 180 degrees in order to conclude that one is *not* living in
$\mathbb{R}^2$. Of course, if the side lengths are sufficiently small,
the effects of living in such a curved space are negligible.

This little thought experiment illustrates *curvature*,
which is one of the fundamental concepts for describing manifolds.
If you enjoyed this foray into the fascinating world of manifolds,
I can recommend the book *The Shape of Space* by Jeffrey R. Weeks.
His writing style is very entertaining and on
geometrygames.org, he provides numerous programs to
visualize and experience numerous manifolds. If you ever wanted to fly
*through* a manifold or play mazes on a torus, *this* is the perfect website for you.

Until next time!

- Here’s a brief refresher from high school mathematics: the real numbers contain all kinds of numbers, such as $2$, $\sqrt{2}$, or even the venerable $\pi$. Real numbers are what we used every day to measure distances, amounts of money, and so on.
^{↩} - Using some mathematics, we will later find out that there
*are*some ways of telling a circle and a line apart. Other than the statement that they obviously*look*different to us, that is.^{↩} - In case you are curious, the
*filled*circle is also called a*disk*by mathematicians. This makes sense because it looks like, well, a disk.^{↩} - Unfortunately, it is a pattern in mathematics to pick the
*easiest*example that one can think of. If a mathematician sees a definition that states that something is supposed to ‘locally look like $\mathbb{R}^d$‘, their first impulse will be to say ‘Why not take the entire $\mathbb{R}^d$ as an example, then?‘. While technically correct—which is of course the best kind of correct—it does not serve to enlighten the reader.^{↩} - We leave the discussion about other dimensions for another post. I am not qualified to discuss quantum mechanics or string theory.
^{↩} - Or, in mathematical lingo, after a
*finite*amount of time.^{↩} - If you want to refresh your memory, take a look at one of the proofs from Wikibooks.
^{↩}