Archive for the ‘Principal Component Analysis (PCA)’ Category

Principal Component Analysis – Explained Visually [Examples up to 17D]

Saturday, February 14th, 2015

Principal Component Analysis – Explained Visually by Victor Powell.

From the website:

Principal component analysis (PCA) is a technique used to emphasize variation and bring out strong patterns in a dataset. It’s often used to make data easy to explore and visualize.

Another stunning visualization (2D, 3D and 17D, yes, not a typo, 17D) from Explained Visually.

Probably not the top item in your mind on Valentine’s Day but you should bookmark it and return when you have more time. 😉

I first saw this in a tweet by Mike Loukides.

Online SVD/PCA resources

Monday, March 26th, 2012

Online SVD/PCA resources by Danny Bickson.

From the post:

Last month I was vising Toyota Technological Institure in Chicago, where I was generously hosted by Tamir Hazan and Joseph Keshet. I heard some interesting stuff about large scale SVM from Joseph Keseht which I reported here. Additionally I met with Raman Arora who is working on online SVD. I asked Raman to summarize the state-of-the-art research on online SVD and here is what I got from him:

A very rich listing of resources on single value decomposition and principal component analysis.

k-means Approach to the Karhunen-Loéve Transform (aka PCA – Principal Component Analysis)

Tuesday, September 27th, 2011

k-means Approach to the Karhunen-Loeve Transform by Krzysztof Misztal, Przemyslaw Spurek, and Jacek Tabor.


We present a simultaneous generalization of the well-known Karhunen-Loeve (PCA) and k-means algorithms. The basic idea lies in approximating the data with k affine subspaces of a given dimension n. In the case n=0 we obtain the classical k-means, while for k=1 we obtain PCA algorithm.

We show that for some data exploration problems this method gives better result then either of the classical approaches.

I know, it is a very forbidding title but once you look at the paper you will be glad you did.

First, the authors begin with a graphic illustration of the goal of their technique (no, you have to look at the paper to see it), which even the most “lay” reader can appreciate.

Second, the need for topic maps strikes again as in the third paragraph we learn: “…Karhunen-Loéve transform (called also PCA – Principle Component Analysis)….”

Third, some of the uses of this technique:

  • data mining – we can detect important coordinates and subsets with similar properties;
  • clustering – our modification of k-means can detect different, high dimensional relation in data;
  • image compression and image segmentation;
  • pattern recognition – thanks to detection of relation in data we can use it to assign data to defined before classes.

A sample implementation is available at:

Principal Components Analysis

Friday, November 26th, 2010

A Tutorial on Principal Components Analysis by Lindsay I. Smith.

From Chapter 3:

Finally we come to Principal Components Analysis (PCA). What is it? It is a way of identifying patterns in data, and expressing the data in such a way as to highlight their similarities and differences. Since patterns in data can be hard to find in data of high dimension, where the luxury of graphical representation is not available, PCA is a powerful tool for analysing data.

The other main advantage of PCA is that once you have found these patterns in the data, and you compress the data, ie. by reducing the number of dimensions, without much loss of information. This technique used in image compression, as we will see in a later section.

One of the main application areas for PCA is image analysis, recognition.

Lindsay starts off with a review of the mathematics needed to work through the rest of the material.

Topic maps are a natural fit for pairing up the results of image recognition, for example, and other data. More on that anon.