Archive for the ‘Hierarchical Temporal Memory (HTM)’ Category

Hierarchical Temporal Memory related Papers and Books

Friday, October 14th, 2011

Hierarchical Temporal Memory related Papers and Books

From the post:

I’m writing a report about using Hierarchical Temporal Memory to model kids behaviour learning a second Language. I have Googled many times to find related works. But I noticed that there are just some works related to the HTM. I’ll upload them all here to have a quick reference. I didn’t put link to the original materials to have always a copy of the originals and to be affected by web-site changes. Take note that some of the uploaded contents (in special Numenta Inc. published articles) are licensed and must be used according to the respective License.

I haven’t explored the area, yet, but this is as good a starting point as any.

Hierarchical Temporal Memory

Friday, October 14th, 2011

Hierarchical Temporal Memory: How a Theory of the Neocortex May Lead to Truly Intelligent Machines by Jeff Hawkins.

Don’t skip because of the title!

Hawkins covers his theory of the neocortex but however you feel about that, 2/3 of the presentation is on algorithms, completely new material.

Very cool presentation on “Fixed Sparsity Distributed Representation” and lots of neural science stuff. Need to listen to it again and then read the books/papers.

What I liked about it was the notion that even in very noisy or missing data contexts, that highly reliable identifications can be made.

True enough, Hawkins was talking about vision, etc., but he didn’t bring up any reasons why that could not work in other data environments.

In other words, when can a program treat extra data about a subject as noise and recognize it anyway?

Or if some information is missing about a subject, have a program reliably recognize it.

Or if we only want to store some information and yet have reliable recognition?

Don’t know if any, some or all of those are possible but it is certainly worth finding out.


Jeff Hawkins (Numenta founder) presents as part of the UBC Department of Computer Science’s Distinguished Lecture Series, March 18, 2010.

Coaxing computers to perform basic acts of perception and robotics, let alone high-level thought, has been difficult. No existing computer can recognize pictures, understand language, or navigate through a cluttered room with anywhere near the facility of a child. Hawkins and his colleagues have developed a model of how the neocortex performs these and other tasks. The theory, called Hierarchical Temporal Memory, explains how the hierarchical structure of the neocortex builds a model of its world and uses this model for inference and prediction. To turn this theory into a useful technology, Hawkins has created a company called Numenta. In this talk Hawkins will describe the theory, its biological basis, and progress in applying Hierarchical Temporal Memory to machine learning problems.

Part of this theory was described in Hawkins’ 2004 book, On Intelligence. Further information can be found at