Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Concurrently Reading Still Got Exceptions


Copy link to this message
-
Concurrently Reading Still Got Exceptions
Dear all,

Some exceptions are raised when I concurrently read data from HBase.
The version of HBase I used is 0.92.0.

I cannot fix the problem. Could you please help me?

Thanks so much!

Best wishes,
Bing

      Feb 6, 2013 12:21:31 AM
org.apache.hadoop.hbase.ipc.HBaseClient$Connection run
      WARNING: Unexpected exception receiving call responses
java.lang.NullPointerException
          at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:521)
          at org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:297)
          at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:593)
          at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)
      Feb 6, 2013 12:21:31 AM
org.apache.hadoop.hbase.client.ScannerCallable close
      WARNING: Ignore, probably already closed
      java.io.IOException: Call to greatfreeweb/127.0.1.1:60020
failed on local exception: java.io.IOException: Unexpected exception
receiving call responses
          at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:934)
          at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:903)
          at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
          at $Proxy6.close(Unknown Source)
          at org.apache.hadoop.hbase.client.ScannerCallable.close(ScannerCallable.java:112)
          at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:74)
          at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:39)
          at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1325)
          at org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1167)
          at org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1296)
          at org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1356)
          at com.greatfree.hbase.rank.NodeRankRetriever.loadNodeGroupNodeRankRowKeys(NodeRankRetriever.java:348)
          at com.greatfree.ranking.PersistNodeGroupNodeRanksThread.run(PersistNodeGroupNodeRanksThread.java:29)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:662) Caused by:
java.io.IOException: Unexpected exception receiving call responses
          at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:509)
      Caused by: java.lang.NullPointerException
          at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:521)
          at org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:297)
          at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:593)
          at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)

I read data from HBase concurrently with the following code.

        ...
                ExecutorService threadPool = Executors.newFixedThreadPool(100);
                LoadNodeGroupNodeRankRowKeyThread thread;
                Set<String> groupKeys;
                for (String nodeKey : nodeKeys)
                {
                        groupKeys = NodeCache.WWW().getGroupKeys(nodeKey);
                        for (String groupKey : groupKeys)
                        {
                                // Threads are initialized and executed here.
                                thread = new
LoadNodeGroupNodeRankRowKeyThread(nodeKey, groupKey,
TimingScale.PERMANENTLY);
                                threadPool.execute(thread);
                        }
                }
                Scanner in = new Scanner(System.in);
                in.nextLine();
                threadPool.shutdownNow();
        ...

The code of LoadNodeGroupNodeRankRowKeyThread is as follows,

        ...
        public void run()
        {
                NodeRankRetriever retriever = new NodeRankRetriever();
                Set<String> rowKeys retriever.loadNodeGroupNodeRankRowKeys(this.hostNodeKey,
this.groupKey, this.timingScale);
                if (rowKeys.size() > 0)
                {
                        for (String rowKey : rowKeys)
                        {
                                System.out.println(rowKey);
                        }
                }
                else
                {
                        System.out.println("No data loaded");
                }
                retriever.dispose();
        }
        ...

The constructor of NodeRankRetriever() just got an instance of HTable
from HTablePool from the following method.

        ...
        public HTableInterface getTable(String tableName)
        {
                return this.hTablePool.getTable(tableName);
        }
        ...

The method dispose() of NodeRankRetriever() just close the
HTableInterface created by HTablePool.

        ...
        public void dispose()
        {
                try
                {
                        this.rankTable.close();
                }
                catch (IOException e)
                {
                        e.printStackTrace();
                }
        }
        ...
+
Ted Yu 2013-03-02, 15:59
+
Anoop John 2013-03-02, 17:44
+
ramkrishna vasudevan 2013-03-02, 17:48
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB