Archive for the ‘Apache Camel’ Category

Integrating with Apache Camel

Thursday, September 26th, 2013

Integrating with Apache Camel by Charles Mouillard.

From the post:

Since its creation by the Apache community in 2007, the open source integration framework Apache Camel has become a developer favourite. It is recognised as a key technology to design SOA / Integration projects and address complex enterprise integration use cases. This article, the first part of a series, will reveal how the framework generates, from the Domain Specific Language, routes where exchanges take place, how they are processed according to the patterns chosen, and finally how integration occurs.

This series will be a good basis to continue onto ‘Enterprise Integration Patterns‘ and compare that to topic maps.

How should topic maps be modified (if at all) to fit into enterprise integration patterns?

Apache Camel tunes its core with new release

Friday, September 20th, 2013

Apache Camel tunes its core with new release by Lucy Carey.

From the post:

The community around open-source integration framework Apache Camel is a prolific little hub, and in the space of just four and a half months, has put together a shiny new release – Apache Camel 2.12 – the 53rd Camel version to date.

On the menu for developers is a total of 17 new components, four new examples, and souped-up performance in simple or bean languages and general routing. More than three hundred JIRA tickets have been solved, and a lot of bug swatting and general fine tuning has taken place. Reflecting the hugely active community around the platform, around half of these new components come courtesy of external contributors, and the rest from Camel team developers.

Fulltime Apache Camel committer Claus Ibsen notes in his blog that this is the first release where steps have been taken to “allow Camel components documentation in the source code which gets generated and included in the binaries.” He also writes that “a Camel component can offer endpoint completion which allows tooling to offer smart completion”, citing the hawtio web console as an example of the ways in which this enables functions like auto completion for JMS queue names, file directory names, bean names in the registry.

Camel homepage.

If you are looking for a variety of explanations about Camel, the Camel homepage recommends a discussion at StackOver.

Not quite the blind men with the elephant but enough differences in approaches to be amusing.

Indexing data in Solr…

Saturday, June 29th, 2013

Indexing data in Solr from disparate sources using Camel by Bilgin Ibryam.

From the post:

Apache Solr is ‘the popular, blazing fast open source enterprise search platform’ built on top of Lucene. In order to do a search (and find results) there is the initial requirement of data ingestion usually from disparate sources like content management systems, relational databases, legacy systems, you name it… Then there is also the challenge of keeping the index up to date by adding new data, updating existing records, removing obsolete data. The new sources of data could be the same as the initial ones, but could also be sources like twitter, AWS or rest endpoints.

Solr can understand different file formats and provides fair amount of options for data indexing:

  1. Direct HTTP and remote streaming – allows you to interact with Solr over HTTP by posting a file for direct indexing or the path to the file for remote streaming.
  2. DataImportHandler – is a module that enables both full and incremental delta imports from relational databases or file system.
  3. SolrJ – a java client to access Solr using Apache Commons HTTP Client.

But in real life, indexing data from different sources with millions of documents, dozens of transformations, filtering, content enriching, replication, parallel processing requires much more than that. One way to cope with such a challenge is by reinventing the wheel: write few custom applications, combine them with some scripts or run cronjobs. Another approach would be to use a tool that is flexible and designed to be configurable and plugable, that can help you to scale and distribute the load with ease. Such a tool is Apache Camel which has also a Solr connector now.


Avoid reinventing the wheel: check mark

Robust software: check mark

Name recognition of Lucene/Solr: check mark

Name recognition of Camel: check mark

Do you see any negatives?

BTW, the examples that round out Bilgin’s post are quite useful!

Apache Camel 2.11.0 Release

Wednesday, May 1st, 2013

Apache Camel 2.11.0 Release by Christian Mueller.

From the post:

The Apache Camel project is a powerful open source integration framework based on known Enterprise Integration Patterns.

The Camel community announces the immediate availability of a new minor release camel-2.11.0. This release is issued after 9 months of intense efforts. During this period the camel community continued to support previous versions and issued various patch releases as well.

The camel-2.11.0 release comes with an impressive 679 issues fixed. Camel is the open source integration framework with the largest support of protocols and data formats on the market. This release adds another 12 components, supporting technologies like cmis, couchdb, elasticsearch, redis, rx and “Springless” JMS integration.

The artifacts are published and ready for you to download either from the Apache mirrors or from the Central Maven repository.

For more details please take a look at the release notes.

Many thanks to the Camel community for making this release possible.

Spring time upgrades are underway!

Apache Camel meets Redis

Saturday, February 23rd, 2013

Apache Camel meets Redis by Bilgin Ibryam.

From the post:

The Lamborghini of Key-Value stores

Camel is the best of bread Integration framework and in this post I’m going to show you how to make it even more powerful by leveraging another great project – Redis. Camel 2.11 is on its way to be released soon with lots of new features, bug fixes and components. Couple of these new components are authored by me, redis-component being my favourite one. Redis – a ligth key/value store is an amazing piece of Italian software designed for speed (same as Lamborghini – a two-seater Italian car designed for speed). Written in C and having an in-memory closer to the metal nature, Redis performs extremely well (Lamborgini’s motto is “Closer to the Road”). Redis is often referred to as a data structure server since keys can contain strings, hashes, lists and sorted sets. A fast and light data structure server is like a super sportscars for software engineers – it just flies. If you want to find out more about Redis’ and Lamborghini’s unique performance characteristics google around and you will see for yourself.

Idempotent Repository

The term idempotent is used in mathematics to describe a function that produces the same result if it is applied to itself. In Messaging this concepts translates into the a message that has the same effect whether it is received once or multiple times. In Camel this pattern is implemented using the IdempotentConsumer class which uses an Expression to calculate a unique message ID string for a given message exchange; this ID can then be looked up in the IdempotentRepository to see if it has been seen before; if it has the message is consumed; if its not then the message is processed and the ID is added to the repository. RedisIdempotentRepository is using a set structure to store and check for existing Ids.

If you have or are considering a message passing topic map application, this may be of interest.

CamelOne 2012 (videos/presentations) Boston, MA

Tuesday, January 29th, 2013

CamelOne 2012 (videos/presentations) Boston, MA

Videos and presentations for your enjoyment from the CamelOne 2012 conference.

As usual I was looking for something else and found more than I bargained for! 😉

Camel Essential Components

Wednesday, January 16th, 2013

Camel Essential Components by Christian Posta. (new Dzone Refcard)

From the webpage:

What is Apache Camel?

Camel is an open-source, lightweight, integration library that allows your applications to accomplish intelligent routing, message transformation, and protocol mediation using the established Enterprise Integration Patterns and out-of-the-box components with a highly expressive Domain Specific Language (Java, XML, or Scala). With Camel you can implement integration solutions as part of an overarching ESB solution, or as individual routes deployed to any container such as Apache Tomcat, Apache ServiceMix, JBoss AS, or even a stand-alone java process.

Why use Camel?

Camel simplifies systems integrations with an easy-to-use DSL to create routes that clearly identify the integration intentions and endpoints. Camel’s out of the box integration components are modeled after the Enterprise Integration Patterns cataloged in Gregor Hohpe and Bobby Wolf’s book ( You can use these EIPs as pre-packaged units, along with any custom processors or external adapters you may need, to easily assemble otherwise complex routing and transformation routes. For example, this route takes an XML message from a queue, does some processing, and publishes to another queue:

No explicit handling of subject identity but that’s what future releases are for. 😉

Building superior integrated applications with open source Apache Camel (Webinar)

Tuesday, October 30th, 2012

Webinar – Building superior integrated applications with open source Apache Camel by Claus Ibsen.

From the post:

I am scheduled to host a free webinar on building integrated applications using Apache Camel.

Date: November 6th, 2012 (moved due Sandy hurricane)
Time: 3:00 PM (Central European Time) – 10:00 AM (EDT)
Duration: 1h15m

This webinar will show you how to build integrated applications with open source Apache Camel. Camel is one of the most frequently downloaded projects, and it is changing the way teams approach integration. The webinar will start with the basics, continue with examples and how to get started, and conclude with live demo. We will cover

  • Enterprise Integration Patterns
  • Domain Specific Languages
  • Maven and Eclipse tooling
  • Java, Spring, OSGi Blueprint, Scala and Groovy
  • Deployment options
  • Extending Camel by building custom Components
  • Q and A

Before we open for QA at the end of the session, we will share links where you can go and read and learn more about Camel. Don’t miss this informative session!

You can register for the webinar at this link.

Definitely on my list to attend.


Apache Camel 2.11 – Neo4j and more new components

Tuesday, October 30th, 2012

Apache Camel 2.11 – Neo4j and more new components by Claus Ibsen.

From the post:

As usual the Camel community continues to be very active. For the upcoming Camel 2.11 release we have already five new components in the works

All five components started by members of the community, and not by people from the Camel team. For example the camel-neo4j, and camel-couchdb components is kindly donated to ASF by Stephen Samuel. Bilgin Ibryam contributed the camel-cmis component. And Cedric Vidal donated the camel-elastichsearch component. And lastly Scott Sullivan donated the camel-sjms component. 

Just in case you live in a world where Enterprise Integration Patterns are relevant. 😉

If you are not familiar with Camel: Camel in Action, Chapter 1 (direct link) free chapter 1 of the Camel in Action book.

I first saw this at DZone.

Apache Camel at 5 [2.10 release]

Friday, July 6th, 2012

Apache Camel celebrates 5 years in development with 2.10 release by Chris Mayer.

Chris writes:

Off the back of celebrating its fifth birthday at CamelOne 2012, the Apache Camel team have put the finishing touches to their next release, Apache Camel 2.10, adding in an array of new components to the Apache enterprise application integration platform.

No less than 483 issues have been resolved this time round, but the real draw is the 18 components added to the package, including Websocket and Twitter, allowing for deeper cohesive messaging for users. With the Twitter component, based on the Twitter4J library, users may obtain direct, polling, or event-driven consumption of timelines, users, trends, and direct messages. An example of combining the two can be found here.

Other additions to the component catalogue include support for HBase, CDI, MongoDB, Apache Avro, DynamoDB on AWS, Google GSON and Guava. Java 7 support is much more thorough now, as is support for Spring 3.1.x and Netty. A full list of all resolved issues can be found here.

The Twitter Websocket example reminds me of something I have been meaning to write about Twitter, topic maps and public data streams.

But more on that next week.

Getting Started with Apache Camel

Saturday, June 16th, 2012

Getting Started with Apache Camel

6/28/2012 10:00 AM EST

From the webpage:


This session will teach you how to get a good start with Apache Camel. It will cover the basic concepts of Camel such as Enterprise Integration Patterns and Domain Specific Languages, all explained with simple examples demonstrating the theory applied in practice using code. We will then discuss how you can get started developing with Camel and how to setup a new project from scratch—using Maven and Eclipse tooling. This session includes live demos that show how to build Camel applications in Java, Spring, OSGi Blueprint and alternative languages such as Scala and Groovy. We demonstrate how to build custom components and we will share highlights of the upcoming Apache Camel 2.10 release.


Claus Ibsen has worked on Apache Camel for years and he shares a great deal of his expertise as a co-author of Manning’s Camel in Action book. He is a principal engineer working for FuseSource specializing in the enterprise integration space. He lives in Sweden near Malmo with his wife and dog.

Data integration talents are great, but coupled with integration tools, they are even better! See you are the webinar!

Apache Camel Tutorial

Wednesday, June 6th, 2012

If you haven’t seen Apache Camel Tutorial Business Partners (other tutorials here), you need to give it a close look:

So there’s a company, which we’ll call Acme. Acme sells widgets, in a fairly unusual way. Their customers are responsible for telling Acme what they purchased. The customer enters into their own systems (ERP or whatever) which widgets they bought from Acme. Then at some point, their systems emit a record of the sale which needs to go to Acme so Acme can bill them for it. Obviously, everyone wants this to be as automated as possible, so there needs to be integration between the customer’s system and Acme.

Sadly, Acme’s sales people are, technically speaking, doormats. They tell all their prospects, “you can send us the data in whatever format, using whatever protocols, whatever. You just can’t change once it’s up and running.”

The result is pretty much what you’d expect. Taking a random sample of 3 customers:

  • Customer 1: XML over FTP
  • Customer 2: CSV over HTTP
  • Customer 3: Excel via e-mail

Now on the Acme side, all this has to be converted to a canonical XML format and submitted to the Acme accounting system via JMS. Then the Acme accounting system does its stuff and sends an XML reply via JMS, with a summary of what it processed (e.g. 3 line items accepted, line item #2 in error, total invoice 123.45). Finally, that data needs to be formatted into an e-mail, and sent to a contact at the customer in question (“Dear Joyce, we received an invoice on 1/2/08. We accepted 3 line items totaling 123.45, though there was an error with line items #2 [invalid quantity ordered]. Thank you for your business. Love, Acme.”).

You don’t have to be a “doormat” to take data as you find it.

Intercepted communications are unlikely to use your preferred terminology for locations or actions. Ditto for web/blog pages.

If you are thinking about normalization of data streams by producing subject-identity enhanced data streams, then you are thinking what I am thinking about Apache Camel.

For further information:

Apache Camel Documentation

Apache Camel homepage