Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> zookeeper-3.3.2 has default maxClientCnxns set to 10?


Copy link to this message
-
zookeeper-3.3.2 has default maxClientCnxns set to 10?
Hello,

I've set up a test HBase+Hadoop cluster yesterday and got the following
error in logs during running MR job (which internally creates HTable for
Reducer):

KeeperErrorCode = ConnectionLoss for /hbase

Then I went to Zookeeper logs and found this:

2011-03-24 22:41:49,884 - WARN  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from
/x.x.x.x - max is 10

As far as I know HBase configured to initiate up to 30 connections by
default, and maxClientCnxns for Zookeeper was meant to be 30 as well. Is
this a bug that I have it set to 10 by default? I use a version:
zookeeper-3.3.2-CDH3B4. Sorry if this issue is specific to CDH (then I
probably should post it in the correspondent ML).

After setting maxClientCnxns explicitly to 30 in zoo.cfg things work well.

Just to prevent some Qs, I have this in code:

when starting job:
    Configuration conf = HBaseConfiguration.create();
    Job job = new Job(conf, aggRawDataJobClass.getName());
    TableMapReduceUtil.initTableMapperJob(rawLogsTableName, scan,
            mapperClass, ImmutableBytesWritable.class, mappedAggregateClass,
job);

in reducer:
public static class CustomReducer<T> extends
TableReducer<ImmutableBytesWritable, T, Writable> {
    @Override
    protected void setup(Context context) throws IOException,
InterruptedException {
       ...
       hTable = new HTable(context.getConfiguration(), tableName);
       ...
    }
}

Alex Baranau
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBase
Hadoop ecosystem search :: http://search-hadoop.com/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB