Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Is "synchronized" required?


+
Bing Li 2013-02-04, 20:20
+
Harsh J 2013-02-04, 20:21
+
Ted Yu 2013-02-04, 20:25
+
Bing Li 2013-02-04, 20:32
+
Haijia Zhou 2013-02-04, 20:42
+
Adrien Mogenet 2013-02-04, 21:13
+
Nicolas Liochon 2013-02-04, 21:31
+
Bing Li 2013-02-04, 22:40
+
Nicolas Liochon 2013-02-04, 22:49
+
Bing Li 2013-02-05, 16:54
+
lars hofhansl 2013-02-06, 05:05
Copy link to this message
-
Re: Is "synchronized" required?
Dear Lars,

Some exceptions are raised when I concurrently read data from HBase.
Each thread to read is assigned a HTable instance. The version of
HBase I used is 0.92.0.

I cannot fix the problem. Could you please help me?

Thanks so much!

Best wishes,
Bing

      Feb 6, 2013 12:21:31 AM
org.apache.hadoop.hbase.ipc.HBaseClient$Connection run
      WARNING: Unexpected exception receiving call responses
java.lang.NullPointerException
          at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:521)
          at org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:297)
          at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:593)
          at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)
      Feb 6, 2013 12:21:31 AM
org.apache.hadoop.hbase.client.ScannerCallable close
      WARNING: Ignore, probably already closed
      java.io.IOException: Call to greatfreeweb/127.0.1.1:60020
failed on local exception: java.io.IOException: Unexpected exception
receiving call responses
          at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:934)
          at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:903)
          at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
          at $Proxy6.close(Unknown Source)
          at org.apache.hadoop.hbase.client.ScannerCallable.close(ScannerCallable.java:112)
          at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:74)
          at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:39)
          at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1325)
          at org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1167)
          at org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1296)
          at org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1356)
          at com.greatfree.hbase.rank.NodeRankRetriever.loadNodeGroupNodeRankRowKeys(NodeRankRetriever.java:348)
          at com.greatfree.ranking.PersistNodeGroupNodeRanksThread.run(PersistNodeGroupNodeRanksThread.java:29)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:662) Caused by:
java.io.IOException: Unexpected exception receiving call responses
          at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:509)
      Caused by: java.lang.NullPointerException
          at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:521)
          at org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:297)
          at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:593)
          at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:505)

I read data from HBase concurrently with the following code.

        ...
                ExecutorService threadPool = Executors.newFixedThreadPool(100);
                LoadNodeGroupNodeRankRowKeyThread thread;
                for (String nodeKey : nodeKeys)
                {
                       // Threads are initialized and executed here.
                       thread = new LoadNodeGroupNodeRankRowKeyThread(nodeKey);
                       threadPool.execute(thread);
                }
                Scanner in = new Scanner(System.in);
                in.nextLine();
                threadPool.shutdownNow();
        ...

The code of LoadNodeGroupNodeRankRowKeyThread is as follows,

        ...
        public void run()
        {
                NodeRankRetriever retriever = new NodeRankRetriever();

// The following line reader data from HBase.
                Set<String> rowKeys retriever.loadNodeGroupNodeRankRowKeys(this.hostNodeKey);
                if (rowKeys.size() > 0)
                {
                        for (String rowKey : rowKeys)
                        {
                                System.out.println(rowKey);
                        }
                }
                else
                {
                        System.out.println("No data loaded");
                }
                retriever.dispose();
        }
        ...

The constructor of NodeRankRetriever() just got an instance of HTable
from HTablePool from the following method.

        ...
        public HTableInterface getTable(String tableName)
        {
                return this.hTablePool.getTable(tableName);
        }
        ...

The method dispose() of NodeRankRetriever() just close the
HTableInterface created by HTablePool.

        ...
        public void dispose()
        {
                try
                {
                        this.rankTable.close();
                }
                catch (IOException e)
                {
                        e.printStackTrace();
                }
        }
        ...

On Wed, Feb 6, 2013 at 1:05 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
+
lars hofhansl 2013-02-07, 17:24
+
Bing Li 2013-02-06, 06:36
+
Adrien Mogenet 2013-02-06, 07:45
+
lars hofhansl 2013-02-06, 07:44
+
Bing Li 2013-02-06, 10:31
+
lars hofhansl 2013-02-06, 18:54