HBase + Hadoop + Xceivers by Lars George.
From the post:
Introduction
Some of the configuration properties found in Hadoop have a direct effect on clients, such as HBase. One of those properties is called “dfs.datanode.max.xcievers”, and belongs to the HDFS subproject. It defines the number of server side threads and – to some extent – sockets used for data connections. Setting this number too low can cause problems as you grow or increase utilization of your cluster. This post will help you to understand what happens between the client and server, and how to determine a reasonable number for this property.
The Problem
Since HBase is storing everything it needs inside HDFS, the hard upper boundary imposed by the ”dfs.datanode.max.xcievers” configuration property can result in too few resources being available to HBase, manifesting itself as IOExceptions on either side of the connection.
This is a true sysadmin type post.
Error messages say “DataXceiver,” but set the “dfs.datanode.max.xcievers” property. Post notes “xcievers” is misspelled.
Detailed coverage of the nature of the problem, complete with sample log entries. Along with suggested solutions.
And, word of current work to improve the current situation.
If you are using HBase and Hadoop, put a copy of this with your sysadmin stuff.
[…] } #themeHeader #titleAndDescription * { color: black; } tm.durusau.net – Today, 6:41 […]
Pingback by HBase + Hadoop + Xceivers « Another Word For It | Gestion de l'Information - Gouvernance et Big Data | Scoop.it — March 15, 2012 @ 5:42 am