Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> scan very slow in hbase


Copy link to this message
-
Re: scan very slow in hbase
Try to set scan cache to 200. and what size of your block cache? did you
check the GC log during slow scan?
On Wed, Aug 7, 2013 at 9:20 AM, ch huang <[EMAIL PROTECTED]> wrote:

> i set the cache still no help ,here is my code
>
>
>         HTable table = new HTable(conf, tablename);
>         Scan scan = new Scan();
>
>         scan.setTimeRange(NumberUtils.toLong("1369108800515"),
> NumberUtils.toLong("1369108810515"));
>
>         scan.setCaching(10000);
>
>         System.out.println("start get scan");
>            ResultScanner rs = table.getScanner(scan);
>
>            while(true){
>             try {
>              Result k = rs.next();
>              if(k == null) break;
>              System.out.println("get rowkey:" + new String(k.getRow()));
>             }catch (Exception e){
>              e.printStackTrace();
>              break;
>             }
>            }
>
>
>
> On Wed, Aug 7, 2013 at 12:29 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
> > In addition to that, make sure you have scanner caching enabled
> > (Scan.setCaching).
> > By default (in 0.94) every call to scanner.next() causes its own RPC
> > request.
> >
> > -- Lars
> >
> >
> >
> > ________________________________
> >  From: Stack <[EMAIL PROTECTED]>
> > To: Hbase-User <[EMAIL PROTECTED]>
> > Sent: Tuesday, August 6, 2013 7:38 AM
> > Subject: Re: scan very slow in hbase
> >
> >
> > I would suggest you search the mail archives before posting first (you
> will
> > usually get your answer faster if you go this route).
> >
> > The below has been answered in the recent past.  See
> > http://search-hadoop.com/m/5tk8QnhFqw
> >
> > Thanks,
> > St.Ack
> >
> >
> > On Tue, Aug 6, 2013 at 12:39 AM, ch huang <[EMAIL PROTECTED]> wrote:
> >
> > > my workmate  tell me hbase is very slow for scan something ,i check the
> > > region server find the following information,anyone can help?
> > >
> > >
> > > 13/08/06 15:30:34 WARN ipc.HBaseServer: IPC Server listener on 60020:
> > > readAndProcess threw exception java.io.IOException: Connection reset by
> > > peer. Count of bytes read: 0
> > > java.io.IOException: Connection reset by peer
> > >         at sun.nio.ch.FileDispatcher.read0(Native Method)
> > >         at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
> > >         at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198)
> > >         at sun.nio.ch.IOUtil.read(IOUtil.java:171)
> > >         at
> sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:245)
> > >         at
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer.channelRead(HBaseServer.java:1796)
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1179)
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
> > >         at
> > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> > >         at
> > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> > >         at java.lang.Thread.run(Thread.java:662)
> > > 13/08/06 15:30:34 ERROR regionserver.HRegionServer:
> > > org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting call
> > > next(1916648340315433886, 1), rpc version=1, client version=29,
> > > methodsFingerPrint=-1368823753 from 192.168.2.209:1150 after 8504 ms,
> > > since
> > > caller disconnected
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Call.throwExceptionIfCallerDisconnected(HBaseServer.java:436)
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3856)
> > >         at
> > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3776)