Curse of Dimensionality Explained by Nikolay Manchev.

Nikolay uses the following illustration:

And follows with (in part):

…

The curse of dimensionality – as the number of dimensions increases, the number of regions grows exponentially.…

This means we have to use 8’000 observations in three-dimensional space to get the same density as we would get from 20 observations in a one-dimensional space.

This illustrates one of the key effects of the curse of dimensionality – as dimensionality increases the data becomes sparse. We need to gather more observations in order to present the classification algorithm with a good space coverage. If we, however, keep increasing the number of dimensions, the number of required observations quickly goes beyond what we can hope to gather.

…

See Nikolay’s post for more details but I thought the illustration of sparsity induced by dimensions was worth repeating.