Archive for the ‘Akka’ Category

Where Akka Came From

Saturday, January 19th, 2013

Where Akka Came From

From the post:

Sparked by the recent work on an Akka article on wikipedia, Jonas, Viktor and Yours Truly sat down to think back to the early days and how it all came about (I was merely an intrigued listener for the most part). While work on the article is ongoing, we thought it would be instructive to share a list of references to papers, talks and concepts which influenced the design—made Akka what it is today and what still is to come.

As you already know, I have a real weakness for source documentation, both ancient as well as more recent.

Enjoy!

Posted in Actor-Based, Akka | No Comments »

Akka Documentation Release 2.1.0

Monday, January 7th, 2013

Akka Documentation Release 2.1.0 from Typesafe Inc. (PDF file)

The documentation answers the question, “What is Akka?” as follows:

Scalable real-time transaction processing

We believe that writing correct concurrent, fault-tolerant and scalable applications is too hard. Most of the time it’s because we are using the wrong tools and the wrong level of abstraction. Akka is here to change that. Using the Actor Model we raise the abstraction level and provide a better platform to build correct, concurrent, and scalable applications. For fault-tolerance we adopt the “Let it crash” model which the telecom industry has used with great success to build applications that self-heal and systems that never stop. Actors also provide the abstraction for transparent distribution and the basis for truly scalable and fault-tolerant applications.

Chris Cundill says “it’s virtually a book!,” which at 424 pages I think is a fair statement. ;-)

Just skimming this looks quite readable!

I first saw this at This week in #Scala (04/01/2013) by Chris Cundill.

Posted in Actor-Based, Akka, Programming | No Comments »

Akka 2.1.0 Released

Monday, January 7th, 2013

Akka 2.1.0 Released

From the post:

We—the Akka committers—are pleased to be able to announce the availability of Akka 2.1.0 ‘Mingus’. We are proud to include the work of 17 external committers, plus the work done by our great community in reporting and helping to diagnose bugs along the way.

This release refines and builds upon version 2.0, which was published a bit over nine months ago. The most prominent new features are

  • cluster support (experimental, including cluster membership logic & death watch and cluster-aware routers, see more below)
  • integration with Scala standard library (SIP-14 Futures, dataflow as add-on module, akka-actor.jar will be part of the Scala distribution)
  • Akka Camel support (Raymond Roestenburg & Piotr Gabryanczyk)
  • Encrypted Akka Remoting using SSL/TLS (Peter Badenhorst)
  • OSGi meta-information for most bundles (excluding samples and tests, Gert Vanthienen)
  • an ActorDSL for more concise actor declarations, e.g. in the REPL
  • a module for multi-node testing (to support you in developing clustered applications, experimental in the same sense as cluster support)
  • a Java API for the TestKit

In addition there have been a great number of small fixes and improvements, documentation updates (including a whole new section on message delivery guarantees), an area for contributions—akka-contrib—where community developments can mature and prove themselves and many more. A series of blog posts high-lighting the new features has been published over the past weeks on this blog, see this tag.

Looking forward to exploring the new features in this release!

Akka website

Akka downloads

I first saw this at This week in #Scala (04/01/2013) by Chris Cundill.

Posted in Actor-Based, Akka | No Comments »

Discovering message flows in actor systems with the Spider Pattern

Sunday, September 2nd, 2012

Discovering message flows in actor systems with the Spider Pattern by Raymond Rostenberg.

From the post:

In this post I’m going to show a pattern that can be used to discover facts about an actor system while it is running. It can be used to understand how messages flow through the actors in the system. The main reason why I built this pattern is to understand what is going on in a running actor system that is distributed across many machines. If I can’t picture it, I can’t understand it (and I’m in good company with that quote :)

Building actor systems is fun but debugging them can be difficult, you mostly end up browsing through many log files on several machines to find out what’s going on. I’m sure you have browsed through logs and thought, “Hey, where did that message go?”, “Why did this message cause that effect” or “Why did this actor never get a message?”

This is where the Spider pattern comes in.

I would think the better quote would be: “If I can’t see it, I can’t understand it.” But each to their own.

Message passing systems remind me of Newcomb’s requirement for having audit trails for merging behavior.

Not necessary for every use case but when it is necessary, it is nice to know robust auditing is possible.

Or perhaps the better way to put it is that auditing is adjustable.

We can go from tracking every operation at one extreme to a middle ground of some tracking but protecting political appointees or career servants or even to a wide open system (sort of like Twitter or Facebook).

Posted in Actor-Based, Akka, Messaging | No Comments »

Distributed (in-memory) graph processing with Akka

Friday, August 31st, 2012

Distributed (in-memory) graph processing with Akka by Adelbert Chang.

From the post:

Graphs have always been an interesting structure to study in both mathematics and computer science (among other fields), and have become even more interesting in the context of online social networks such as Facebook and Twitter, whose underlying network structures are nicely represented by graphs.

These graphs are typically “big”, even when sub-graphed by things such as location or school. With “big” graphs comes the desire to extract meaningful information from these graphs. In the age of multi-core CPU’s and distributed computing, concurrent processing of graphs proves to be an important topic.

Luckily, many graph analysis algorithms are trivially parallelizable. One example that comes to mind is all-pairs shortest path. In the case of an undirected, unweighted graph, we can consider each vertex individually, and do a full BFS from each vertex.

In this post I detail a general framework for distributed graph processing. I do not use any particular graph library, so my graph class will simply be called Graph. Popular graph libraries for Scala can be found in Twitter’s Cassovary project or the Graph for Scala project.

I will also make use of Derek Wyatt’s submission to the Akka Summer of Blog—”Balancing Workloads Across Nodes with Akka 2“—which provides a nice and simple implementation of a BalancingDispatcher in the context of distributed processing.

If you like Akka, graphs, or both, you will enjoy this post.

Posted in Akka, Graphs | No Comments »

Dempsy – a New Real-time Framework for Processing BigData

Friday, May 4th, 2012

Dempsy – a New Real-time Framework for Processing BigData by Boris Lublinsky.

From the post:

Real time processing of BigData seems to be one of the hottest topics today. Nokia has just released a new open-source project – Dempsy. Dempsy is comparable to Storm, Esper, Streambase, HStreaming and Apache S4. The code is released under the Apache 2 license

Dempsy is meant to solve the problem of processing large amounts of "near real time" stream data with the lowest lag possible; problems where latency is more important that "guaranteed delivery." This class of problems includes use cases such as:

  • Real time monitoring of large distributed systems
  • Processing complete rich streams of social networking data
  • Real time analytics on log information generated from widely distributed systems
  • Statistical analytics on real-time vehicle traffic information on a global basis

The important properties of Dempsy are:

  • It is Distributed. That is to say a Dempsy application can run on multiple JVMs on multiple physical machines.
  • It is Elastic. That is, it is relatively simple to scale an application to more (or fewer) nodes. This does not require code or configuration changes but done by dynamic insertion or removal of processing nodes.
  • It implements Message Processing. Dempsy is based on message passing. It moves messages between Message processors, which act on the messages to perform simple atomic operations such as enrichment, transformation, etc. In general, an application is intended to be broken down into more smaller simpler processors rather than fewer large complex processors.
  • It is a Framework. It is not an application container like a J2EE container, nor a simple library. Instead, like the Spring Framework, it is a collection of patterns, the libraries to enable those patterns, and the interfaces one must implement to use those libraries to implement the patterns.

Dempsy’ programming model is based on message processors communicating via messages and resembles a distributed actor framework . While not strictly speaking an actor framework in the sense of Erlang or Akka actors, where actors explicitely direct messages to other actors, Dempsy’s Message Processors are "actor like POJOs" similar to Processor Elements in S4 and to some extent Bolts in Storm. Message processors are similar to actors in that they operate on a single message at a time, and need not deal with concurrency directly. Unlike actors, Message Processors also are relieved of the the need to know the destination(s) for their output messages, as this is handled inside by Dempsy based on the message properties.

In short Dempsy is a framework to enable the decomposing of a large class of message processing problems into flows of messages between relatively simple processing units implemented as POJOs. 

The Dempsy Tutorial contains more information.

See the post for an interview with Dempsy’s creator, NAVTEQ Fellow Jim Carroll.

Will the “age of data” mean that applications and their code will also be viewed and processed as data? The capabilities you have are those you request for a particular data set? Would like to see topic maps on the leading (and not dragging) edge of that change.

Posted in Akka, Apache S4, Dempsy, Erlang, Esper, HStreaming, Storm, Streambase | No Comments »

50 million messages per second – on a single machine

Sunday, April 8th, 2012

50 million messages per second – on a single machine

From the post:

50 million messages per second on a single machine is mind blowing!

We have measured this for a micro benchmark of Akka 2.0.

As promised in Scalability of Fork Join Pool I will here describe one of the tuning settings that can be used to achieve even higher throughput than the amazing numbers presented previously. Using the same benchmark as in Scalability of Fork Join Pool and only changing the configuration we go from 20 to 50 million messages per second.

The micro benchmark use pairs of actors sending messages to each other, classical ping-pong. All sharing the same fork join dispatcher.

Fairly sure the web scale folks will just sniff and move on. It’s not like every Facebook user sending individual messages to all of their friends and their friend’s friends, all at the same time.

On the other hand, 50 million messages per second per machine, on enough machines, and you are talking about a real pile of message. ;-)

Are we approaching the point of data being responsible for processing itself and reporting the results? Or at least reporting itself to the nearest processor with the appropriate inputs? Perhaps by broadcasting a message itself?

Closer to home, could a topic map infrastructure be built using message passing that reports a TMDM based data model? For use by query or constraint languages? That is it presents a TMDM API as it were, although behind the scenes the reported API is the result of message passing and processing.

That would make the data model or API if you prefer, a matter of what message passing had been implemented.

More malleable and flexible than a relational database scheme or Cyc based ontology. An enlightened data structure, for a new age.

Posted in Akka, Marketing | No Comments »

Typesafe Stack

Tuesday, December 27th, 2011

Typesafe Stack

From the website:

Scala. Akka. Simple.

A 100% open source, integrated distribution offering Scala, Akka, sbt, and the Scala IDE for Eclipse.

The Typesafe Stack makes it easy for developers to get started building scalable software systems with Scala and Akka. The Typesafe Stack is based on the most recent stable versions of Scala and Akka, and provides all of the major components needed to develop and deploy Scala and Akka applications.

Go ahead! You need something new to put on your new, shiny 5TB disk drive. ;-)

Posted in Akka, Scala | No Comments »

Let It Crash

Sunday, December 25th, 2011

Let It Crash

Who else but the Akka team would choose a blog title like: Let It Crash. ;-)

An early post? Read on:

Location Transparency: Remoting in Akka 2.0

The remoting capabilities of Akka 2.0 are really powerful. Something that not has been as powerful is the documentation of the Akka remoting. We are constantly striving on improving it and this blog post will, hopefully, shed some light on the topic.

The remoting contains functionality not only to lookup a remote actor and send messages to it but also to deploy actors on remote nodes. These two types of interaction are referred to as:

  • Lookup
  • Creation

In the section below the two different approaches will be explained.
(It may be worth pointing out that a combination of the two ways is, of course, also feasible)

(see the post for the rest of it)

Encouraging because the team realizes that its documentation leaves something to be desired and just as importantly, it wants to do something about it.

Looking forward to more posts like this one.

Drop by and leave an encouraging word.

Posted in Akka | No Comments »

Scala Videos (and ebook)

Friday, October 21st, 2011

Scala Videos (and ebook)

While looking for something else (isn’t that always the case?) I ran across this collection of Scala videos and a free ebook, Scala for the Impatient, at Typesafe.

Something to enjoy over the weekend!

Posted in Akka, Scala | No Comments »

Akka

Thursday, September 8th, 2011

Akka

From the webpage:

Akka is the platform for the next generation event-driven, scalable and fault-tolerant architectures on the JVM

We believe that writing correct concurrent, fault-tolerant and scalable applications is too hard. Most of the time it’s because we are using the wrong tools and the wrong level of abstraction.

Akka is here to change that.

Using the Actor Model together with Software Transactional Memory we raise the abstraction level and provide a better platform to build correct concurrent and scalable applications.

For fault-tolerance we adopt the “Let it crash” / “Embrace failure” model which have been used with great success in the telecom industry to build applications that self-heal, systems that never stop.

Actors also provides the abstraction for transparent distribution and the basis for truly scalable and fault-tolerant applications.

Akka is Open Source and available under the Apache 2 License.

I am increasingly convinced that “we are using the wrong tools and the wrong level of abstraction.”

Everyone seems to agree on that part. Where they differ is on the right tool and right level of abstraction. ;-)

I suspect the disagreement isn’t going away. But I mention Akka in case it seems like the right tool and right level of abstraction to you.

I would be mindful that the marketplace for non-concurrent, not so scalable semantic applications is quite large. Think of it this way, someone has the be the “Office” of semantic applications. May as well be you. Leave the high-end, difficult stuff to others.

Posted in Actor-Based, Akka | No Comments »