Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Error while doing multi get from HBase


+
anand nalya 2013-04-11, 15:23
+
Ted Yu 2013-04-11, 21:26
+
anand nalya 2013-04-12, 13:47
+
anand nalya 2013-04-12, 13:52
+
Azuryy Yu 2013-04-12, 14:00
+
anand nalya 2013-04-12, 14:29
Copy link to this message
-
Re: Error while doing multi get from HBase
your CMS is not tuned, please find how tune CMS on java web site.

then Xmn is too small, this is not suitable for frequent multi get. please
change Xmn=1g

--Send from my Sony mobile.
On Apr 12, 2013 10:29 PM, "anand nalya" <[EMAIL PROTECTED]> wrote:

> the block cache size is 0.25
>
> Each row holds around 2KB data, so size should not be an issue at least
> till the number of records is less than 1000.
>
> Also,
> HBASE_HEAPSIZE=8000
> HBASE_OPTS="-XX:+UseConcMarkSweepGC"
> HBASE_REGIONSERVER_OPTS="-Xmx4g -Xms4g -Xmn256m -XX:+UseParNewGC
> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -verbose:gc
> -XX:+PrintGCDetails -XX:+PrintGCTimeStamps"
>
> On 12 April 2013 19:30, Azuryy Yu <[EMAIL PROTECTED]> wrote:
>
> > and whats your block cache size?
> >
> > there are two possible reasons:
> > 1. result is too big
> > 2. GC options are not optimized.
> >
> > can you paste your gc options here?
> >
> > --Send from my Sony mobile.
> > On Apr 12, 2013 9:53 PM, "anand nalya" <[EMAIL PROTECTED]> wrote:
> >
> > > Hi Azuryy,
> > >
> > > I'm using the default cache size of 100 for scanner. For mutigets, I've
> > > tried with 1 (13ms), 10(356ms), 100(1135ms), 1000(4330ms), and
> > > 10000(17744ms) keys. Normal workload will be around 10000 keys at a
> time.
> > >
> > > Are there any optimization that can be done for multigets. Is HBase a
> > good
> > > candidate for usecase?
> > >
> > > Thanks,
> > > Anand
> > >
> > >
> > > On 12 April 2013 19:17, anand nalya <[EMAIL PROTECTED]> wrote:
> > >
> > > > Hi Ted,
> > > >
> > > > The region servers are not loaded. It is showing 5% CPU usage. The
> > > > datanode is showing around 50% CPU utilization. disk IO is aroung
> > 7Mbps.
> > > >
> > > > There is nothing noticeable in GC log.
> > > >
> > > > Thanks,
> > > > Anand
> > > >
> > > >
> > > > On 12 April 2013 02:56, Ted Yu <[EMAIL PROTECTED]> wrote:
> > > >
> > > >> How loaded were the region servers when the query was running ?
> > > >>
> > > >> Did you check GC log ?
> > > >>
> > > >> Thanks
> > > >>
> > > >> On Thu, Apr 11, 2013 at 8:23 AM, anand nalya <[EMAIL PROTECTED]
> >
> > > >> wrote:
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> > I'm using HBase 0.94.5 with thrift server. I'm trying to get the
> > rows
> > > >> from
> > > >> > HBase using
> > > >> >
> > > >>
> > >
> org.apache.hadoop.hbase.thrift.generated.Hbase.Client.getRows(ByteBuffer,
> > > >> > List<ByteBuffer>, Map<ByteBuffer, ByteBuffer>) but it is giving
> > > results
> > > >> > very slowly (around 2 mins for 100 rows). For larger number of
> > > records,
> > > >> > there is no response.
> > > >> >
> > > >> > I've two region server and a total of 128 regions. Total data size
> > is
> > > >> > around 250GB (250 million records) uniformly distributed across
> > > regions.
> > > >> >
> > > >> > Regionserver only show the following in its log:
> > > >> >
> > > >> > 2013-04-11 19:53:44,535 ERROR
> > > >> > org.apache.hadoop.hbase.regionserver.HRegionServer:
> > > >> > org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting
> > call
> > > >> > multi(org.apache.hadoop.hbase.client.MultiAction@49ac272), rpc
> > > >> version=1,
> > > >> > client version=29, methodsFingerPrint=-1368823753 from
> > > >> > 192.168.145.195:52277after 74994 ms, since caller disconnected
> > > >> >     at
> > > >> >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436)
> > > >> >     at
> > > >> >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3723)
> > > >> >     at
> > > >> >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3643)
> > > >> >     at
> > > >> >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3626)
> > > >> >     at
> > > >> >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3664)
+
Azuryy Yu 2013-04-12, 01:30
+
anand nalya 2013-04-11, 15:20