Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper, mail # user - zookeeper-3.3.2 has default maxClientCnxns set to 10?


Copy link to this message
-
zookeeper-3.3.2 has default maxClientCnxns set to 10?
Alex Baranau 2011-03-25, 19:36
Hello,

I've set up a test HBase+Hadoop cluster yesterday and got the following
error in logs during running MR job (which internally creates HTable for
Reducer):

KeeperErrorCode = ConnectionLoss for /hbase

Then I went to Zookeeper logs and found this:

2011-03-24 22:41:49,884 - WARN  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from
/x.x.x.x - max is 10

As far as I know HBase configured to initiate up to 30 connections by
default, and maxClientCnxns for Zookeeper was meant to be 30 as well. Is
this a bug that I have it set to 10 by default? I use a version:
zookeeper-3.3.2-CDH3B4. Sorry if this issue is specific to CDH (then I
probably should post it in the correspondent ML).

After setting maxClientCnxns explicitly to 30 in zoo.cfg things work well.

Just to prevent some Qs, I have this in code:

when starting job:
    Configuration conf = HBaseConfiguration.create();
    Job job = new Job(conf, aggRawDataJobClass.getName());
    TableMapReduceUtil.initTableMapperJob(rawLogsTableName, scan,
            mapperClass, ImmutableBytesWritable.class, mappedAggregateClass,
job);

in reducer:
public static class CustomReducer<T> extends
TableReducer<ImmutableBytesWritable, T, Writable> {
    @Override
    protected void setup(Context context) throws IOException,
InterruptedException {
       ...
       hTable = new HTable(context.getConfiguration(), tableName);
       ...
    }
}

Alex Baranau
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBase
Hadoop ecosystem search :: http://search-hadoop.com/
+
Stack 2011-03-25, 20:00
+
Alex Baranau 2011-03-25, 20:57
+
Mathias Herberts 2011-03-25, 21:22
+
Alex Baranau 2011-03-25, 22:21
+
Todd Lipcon 2011-03-26, 01:07
+
Dmitriy Lyubimov 2011-03-26, 09:04
+
Dmitriy Lyubimov 2011-03-26, 09:08
+
Todd Lipcon 2011-03-26, 20:43