Hadoop on a Raspberry Pi by Isaac Lopez
From the post:
Looking for a fun side project this winter? Jamie Whitehorn has an idea for you. He put Hadoop on a cluster of Raspberry Pi mini-computers. Sound ridiculous? For a student trying to learn Hadoop, it could be ridiculously cool.
For those who don’t know what a Raspberry Pi is, think of it as a computer on a credit card meets Legos. They’re little chunks of computing technology, complete with a Linux operating system, a 700MHz ARM11 processor, a low-power video processor and up to 512MB of Memory. Tinkerers can use it as the computing brains behind any number of applications that they design to their heart’s content. In a recent example, a Raspberry Pi enthusiast built a Raspberry Pi mini PC, which he used to control a mini CNC Laser engraver made out of an old set of salvaged DVD drives and $10 dollars in parts of eBay. Ideas range from building a web server, a weather station, home automation systems, mini arcades – the list of projects is endless.
At the Strata + Hadoop World conference last month, Jamie Whitehorn shared his Hadoop Raspberry Pi creation with an audience. He discussed the challenges a student has in learning the Hadoop system. Chiefly, it’s a distributed architecture that requires multiple computers to operate. Someone looking to build Hadoop skills in a test environment would need several machines, and quite an electricity bill to get a cluster up – a prospect that can be very expensive for a student.
Whitehorn makes the point that while it’s true that this can all be avoided using a Hadoop cloud service, he says that defeats the point, which is understanding the interaction between the software and the hardware. The whole point of the exercise, he explains, is to face the complexity of the project and overcome it.
…
Whitehorn says that he’s learned a lot about Hadoop from attempting the project, and encourages others to get in on the action. For anyone who is interested in doing that, he has posted a blog entry that discusses his approach and some of the nuances that can be found here.
If you want to learn Hadoop close to the metal, or closer than usual, this is the project for you!