From the post:
These are exciting times at Splunk, and for Big Data. During the 2011 Hadoop World, we announced our initiative to combine Splunk and Hadoop in a new offering. The heart of this new offering is an open source component called Shep. Shep is what will enable seamless two-way data-flow across the the systems, as well as opening up two-way compute operations across data residing in both systems.
The thing that intrigues us most is the synergy between Splunk and Hadoop. The ways to integrate are numerous, and as the field evolves and the project progresses, we can see more and more opportunities to provide powerful solutions to common problems.
Many of our customers are indexing terabytes per day, and have also spun up Hadoop initiatives in other parts of the business. Splunk integration with Hadoop is part of a broader goal at Splunk to break down barriers to data-silos, and open them up to availability across the enterprise, no matter what the source. To itemize some categories we’re focused on, listed here are some key use cases:
- Query both Splunk and Hadoop data, using Splunk as a “single-pane-of-glass”
- Data transformation utilizing Splunk search commands
- Real-time analytics of data streams going to mutliple destinations
- Splunk as data warehouse/marts for targeted exploration of HDFS data
- Data acquisition from logs and apis via Splunk Universal Forwarder
Read the post to learn the features that are supported now or soon will be in Shep.
Now in private beta but it sounds worthy of a “heads up!”