Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> issue about search speed and rowkey


Copy link to this message
-
issue about search speed and rowkey
hi,all:

        i have problem in running following code,MoveData method is used to
get data from source table ,and modify each row rowkey ,and insert into
destination table,and i always get error,anyone can help?

 public static void writeRow(HTablePool htp,String tablename, String
rowkey,String cf,String col,String value) {
        try {
         HTableInterface table = htp.getTable(Bytes.toBytes(tablename));
            Put put = new Put(Bytes.toBytes(rowkey));
                put.add(Bytes.toBytes(cf),
                  Bytes.toBytes(col),
                  Long.parseLong(rowkey),
                  Bytes.toBytes(value));

                table.put(put);
            table.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

public static void MoveData(String src_t,String dest_t){

    try{
     HTable tabsrc = new HTable(conf, src_t);
     HTable tabdest = new HTable(conf, dest_t);
     tabsrc.setAutoFlush(false);
     tabdest.setAutoFlush(false);
     HTablePool tablePool = new HTablePool(conf, 5);
     Scan scan = new Scan();
     scan.setCaching(10000);
        scan.setBatch(10);
        ResultScanner rs = tabsrc.getScanner(scan);

           for (Result r : rs){
            ArrayList al = new ArrayList();
            HashMap hm = new HashMap();
            for (KeyValue kv : r.raw()){

             hm.put(new String(kv.getQualifier()), new
String(kv.getValue()));
             al.add(new String(kv.getQualifier()));
            }

               for (int i = 0; i < al.size(); i++) {

writeRow(tablePool,dest_t,hm.get("date").toString(),"info",al.get(i).toString(),hm.get(al.get(i)).toString());
               }
         }

           rs.close();
           tabsrc.close();
           tabdest.close();

    }catch(IOException e){
     e.printStackTrace();
    }

   }

2013-08-07 16:43:31,250 WARN  [main] conf.Configuration
(Configuration.java:warnOnceIfDeprecated(824)) - hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
java.lang.RuntimeException:
org.apache.hadoop.hbase.client.ScannerTimeoutException: 123891ms passed
since the last invocation, timeout is currently set to 120000
 at
org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:44)
 at com.testme.demo.HBaseTest.MoveData(HBaseTest.java:186)
 at com.testme.demo.HBaseTest.main(HBaseTest.java:314)
Caused by: org.apache.hadoop.hbase.client.ScannerTimeoutException: 123891ms
passed since the last invocation, timeout is currently set to 120000
 at
org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:283)
 at
org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:41)
 ... 2 more
Caused by: org.apache.hadoop.hbase.UnknownScannerException:
org.apache.hadoop.hbase.UnknownScannerException: Name: -1792412350530007203
 at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2464)
 at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
 at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)

 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
Source)
 at java.lang.reflect.Constructor.newInstance(Unknown Source)
 at
org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
 at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:143)
 at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:42)
 at
org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:163)
 at
org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:274)
 ... 3 more
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB