Curse of Dimensionality Explained by Nikolay Manchev.
Nikolay uses the following illustration:
And follows with (in part):
…
The curse of dimensionality – as the number of dimensions increases, the number of regions grows exponentially.…
This means we have to use 8’000 observations in three-dimensional space to get the same density as we would get from 20 observations in a one-dimensional space.
This illustrates one of the key effects of the curse of dimensionality – as dimensionality increases the data becomes sparse. We need to gather more observations in order to present the classification algorithm with a good space coverage. If we, however, keep increasing the number of dimensions, the number of required observations quickly goes beyond what we can hope to gather.
…
See Nikolay’s post for more details but I thought the illustration of sparsity induced by dimensions was worth repeating.