Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Error while doing multi get from HBase


+
anand nalya 2013-04-11, 15:23
+
Ted Yu 2013-04-11, 21:26
+
anand nalya 2013-04-12, 13:47
+
anand nalya 2013-04-12, 13:52
Copy link to this message
-
Re: Error while doing multi get from HBase
and whats your block cache size?

there are two possible reasons:
1. result is too big
2. GC options are not optimized.

can you paste your gc options here?

--Send from my Sony mobile.
On Apr 12, 2013 9:53 PM, "anand nalya" <[EMAIL PROTECTED]> wrote:

> Hi Azuryy,
>
> I'm using the default cache size of 100 for scanner. For mutigets, I've
> tried with 1 (13ms), 10(356ms), 100(1135ms), 1000(4330ms), and
> 10000(17744ms) keys. Normal workload will be around 10000 keys at a time.
>
> Are there any optimization that can be done for multigets. Is HBase a good
> candidate for usecase?
>
> Thanks,
> Anand
>
>
> On 12 April 2013 19:17, anand nalya <[EMAIL PROTECTED]> wrote:
>
> > Hi Ted,
> >
> > The region servers are not loaded. It is showing 5% CPU usage. The
> > datanode is showing around 50% CPU utilization. disk IO is aroung 7Mbps.
> >
> > There is nothing noticeable in GC log.
> >
> > Thanks,
> > Anand
> >
> >
> > On 12 April 2013 02:56, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> >> How loaded were the region servers when the query was running ?
> >>
> >> Did you check GC log ?
> >>
> >> Thanks
> >>
> >> On Thu, Apr 11, 2013 at 8:23 AM, anand nalya <[EMAIL PROTECTED]>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > I'm using HBase 0.94.5 with thrift server. I'm trying to get the rows
> >> from
> >> > HBase using
> >> >
> >>
> org.apache.hadoop.hbase.thrift.generated.Hbase.Client.getRows(ByteBuffer,
> >> > List<ByteBuffer>, Map<ByteBuffer, ByteBuffer>) but it is giving
> results
> >> > very slowly (around 2 mins for 100 rows). For larger number of
> records,
> >> > there is no response.
> >> >
> >> > I've two region server and a total of 128 regions. Total data size is
> >> > around 250GB (250 million records) uniformly distributed across
> regions.
> >> >
> >> > Regionserver only show the following in its log:
> >> >
> >> > 2013-04-11 19:53:44,535 ERROR
> >> > org.apache.hadoop.hbase.regionserver.HRegionServer:
> >> > org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call
> >> > multi(org.apache.hadoop.hbase.client.MultiAction@49ac272), rpc
> >> version=1,
> >> > client version=29, methodsFingerPrint=-1368823753 from
> >> > 192.168.145.195:52277after 74994 ms, since caller disconnected
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3723)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3643)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3626)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3664)
> >> >     at
> >> org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4576)
> >> >     at
> >> org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4549)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2042)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3516)
> >> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> >     at
> >> >
> >> >
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >> >     at
> >> >
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> >     at java.lang.reflect.Method.invoke(Method.java:597)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
> >> >     at
> >> >
> >>
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> >> > 2013-04-11 19:53:46,121 ERROR
> >> > org.apache.hadoop.hbase.regionserver.HRegionServer:
> >> > org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call
+
anand nalya 2013-04-12, 14:29
+
Azuryy Yu 2013-04-12, 14:33
+
Azuryy Yu 2013-04-12, 01:30
+
anand nalya 2013-04-11, 15:20
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB