Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - scan very slow in hbase


Copy link to this message
-
Re: scan very slow in hbase
Azuryy Yu 2013-08-07, 01:23
Try to set scan cache to 200. and what size of your block cache? did you
check the GC log during slow scan?
On Wed, Aug 7, 2013 at 9:20 AM, ch huang <[EMAIL PROTECTED]> wrote:

> i set the cache still no help ,here is my code
>
>
>         HTable table = new HTable(conf, tablename);
>         Scan scan = new Scan();
>
>         scan.setTimeRange(NumberUtils.toLong("1369108800515"),
> NumberUtils.toLong("1369108810515"));
>
>         scan.setCaching(10000);
>
>         System.out.println("start get scan");
>            ResultScanner rs = table.getScanner(scan);
>
>            while(true){
>             try {
>              Result k = rs.next();
>              if(k == null) break;
>              System.out.println("get rowkey:" + new String(k.getRow()));
>             }catch (Exception e){
>              e.printStackTrace();
>              break;
>             }
>            }
>
>
>
> On Wed, Aug 7, 2013 at 12:29 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
> > In addition to that, make sure you have scanner caching enabled
> > (Scan.setCaching).
> > By default (in 0.94) every call to scanner.next() causes its own RPC
> > request.
> >
> > -- Lars
> >
> >
> >
> > ________________________________
> >  From: Stack <[EMAIL PROTECTED]>
> > To: Hbase-User <[EMAIL PROTECTED]>
> > Sent: Tuesday, August 6, 2013 7:38 AM
> > Subject: Re: scan very slow in hbase
> >
> >
> > I would suggest you search the mail archives before posting first (you
> will
> > usually get your answer faster if you go this route).
> >
> > The below has been answered in the recent past.  See
> > http://search-hadoop.com/m/5tk8QnhFqw
> >
> > Thanks,
> > St.Ack
> >
> >
> > On Tue, Aug 6, 2013 at 12:39 AM, ch huang <[EMAIL PROTECTED]> wrote:
> >
> > > my workmate  tell me hbase is very slow for scan something ,i check the
> > > region server find the following information,anyone can help?
> > >
> > >
> > > 13/08/06 15:30:34 WARN ipc.HBaseServer: IPC Server listener on 60020:
> > > readAndProcess threw exception java.io.IOException: Connection reset by
> > > peer. Count of bytes read: 0
> > > java.io.IOException: Connection reset by peer
> > >         at sun.nio.ch.FileDispatcher.read0(Native Method)
> > >         at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
> > >         at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198)
> > >         at sun.nio.ch.IOUtil.read(IOUtil.java:171)
> > >         at
> sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:245)
> > >         at
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer.channelRead(HBaseServer.java:1796)
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1179)
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
> > >         at
> > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >         at
> > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >         at java.lang.Thread.run(Thread.java:662)
> > > 13/08/06 15:30:34 ERROR regionserver.HRegionServer:
> > > org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call
> > > next(1916648340315433886, 1), rpc version=1, client version=29,
> > > methodsFingerPrint=-1368823753 from 192.168.2.209:1150 after 8504 ms,
> > > since
> > > caller disconnected
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436)
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3856)
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3776)