Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Error while doing multi get from HBase


Copy link to this message
-
Re: Error while doing multi get from HBase
and whats your block cache size?

there are two possible reasons:
1. result is too big
2. GC options are not optimized.

can you paste your gc options here?

--Send from my Sony mobile.
On Apr 12, 2013 9:53 PM, "anand nalya" <[EMAIL PROTECTED]> wrote:

> Hi Azuryy,
>
> I'm using the default cache size of 100 for scanner. For mutigets, I've
> tried with 1 (13ms), 10(356ms), 100(1135ms), 1000(4330ms), and
> 10000(17744ms) keys. Normal workload will be around 10000 keys at a time.
>
> Are there any optimization that can be done for multigets. Is HBase a good
> candidate for usecase?
>
> Thanks,
> Anand
>
>
> On 12 April 2013 19:17, anand nalya <[EMAIL PROTECTED]> wrote:
>
> > Hi Ted,
> >
> > The region servers are not loaded. It is showing 5% CPU usage. The
> > datanode is showing around 50% CPU utilization. disk IO is aroung 7Mbps.
> >
> > There is nothing noticeable in GC log.
> >
> > Thanks,
> > Anand
> >
> >
> > On 12 April 2013 02:56, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> >> How loaded were the region servers when the query was running ?
> >>
> >> Did you check GC log ?
> >>
> >> Thanks
> >>
> >> On Thu, Apr 11, 2013 at 8:23 AM, anand nalya <[EMAIL PROTECTED]>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > I'm using HBase 0.94.5 with thrift server. I'm trying to get the rows
> >> from
> >> > HBase using
> >> >
> >>
> org.apache.hadoop.hbase.thrift.generated.Hbase.Client.getRows(ByteBuffer,
> >> > List<ByteBuffer>, Map<ByteBuffer, ByteBuffer>) but it is giving
> results
> >> > very slowly (around 2 mins for 100 rows). For larger number of
> records,
> >> > there is no response.
> >> >
> >> > I've two region server and a total of 128 regions. Total data size is
> >> > around 250GB (250 million records) uniformly distributed across
> regions.
> >> >
> >> > Regionserver only show the following in its log:
> >> >
> >> > 2013-04-11 19:53:44,535 ERROR
> >> > org.apache.hadoop.hbase.regionserver.HRegionServer:
> >> > org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call
> >> > multi(org.apache.hadoop.hbase.client.MultiAction@49ac272), rpc
> >> version=1,
> >> > client version=29, methodsFingerPrint=-1368823753 from
> >> > 192.168.145.195:52277after 74994 ms, since caller disconnected
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3723)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3643)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3626)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3664)
> >> >     at
> >> org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4576)
> >> >     at
> >> org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4549)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2042)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3516)
> >> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> >     at
> >> >
> >> >
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >> >     at
> >> >
> >> >
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> >     at java.lang.reflect.Method.invoke(Method.java:597)
> >> >     at
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
> >> >     at
> >> >
> >>
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> >> > 2013-04-11 19:53:46,121 ERROR
> >> > org.apache.hadoop.hbase.regionserver.HRegionServer:
> >> > org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call