Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> How many column families in one table ?


Copy link to this message
-
Re: How many column families in one table ?
Hi Lars,
I am using Hadoop version - 1.1.2  and Hbase version - 0.94.7.
Yes , I have enabled scanner caching with value 10K but performance is not
too good. :(
On Mon, Jul 1, 2013 at 4:48 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> Which version of HBase?
> Did you enable scanner caching? Otherwise each call to next() is a RPC
> roundtrip and you are basically measuring your networks RTT.
>
> -- Lars
>
>
> ________________________________
>  From: Vimal Jain <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Monday, July 1, 2013 4:11 AM
> Subject: Re: How many column families in one table ?
>
>
> Can someone please reply ?
> Also what is  the typical read/write speed of hbase and how much deviation
> would be there in my scenario mentioned above (14 cf , total 140 columns )
> ?
> I am asking this because i am not simply printing out the scanned values ,
> instead i am applying some logic on the data retrieved per row basis. So
> was just curious to find if that small logic in my code is contributing
> towards the long time taken to scan the table.
>
>
> On Mon, Jul 1, 2013 at 2:41 PM, Vimal Jain <[EMAIL PROTECTED]> wrote:
>
> > I scanned it during normal traffic hours.There was no I/O load on the
> > server.
> > I dont see any GC locks too.
> > Also i have given 1.5G to RS , 512M to each Master and Zookeeper.
> >
> > One correction in the post above :
> > Actual time to scan whole table is even more , it takes 10 mins to scan
> > 0.1 million rows ( so total of 2.5 hours to scan 1.6 million rows) .
> > The time i mentioned in previous post was for different type of
> > lookup.Please ignore that.
> >
> >
> > On Mon, Jul 1, 2013 at 2:24 PM, Viral Bajaria <[EMAIL PROTECTED]
> >wrote:
> >
> >> When you did the scan, did you check what the bottleneck was ? Was it
> I/O
> >> ?
> >> Did you see any GC locks ? How much RAM are you giving to your RS ?
> >>
> >> -Viral
> >>
> >> On Mon, Jul 1, 2013 at 1:44 AM, Vimal Jain <[EMAIL PROTECTED]> wrote:
> >>
> >> > To completely scan the table for all 140 columns  , it takes around
> >> 30-40
> >> > minutes.
> >> >
> >>
> >
> >
> >
> > --
> > Thanks and Regards,
> > Vimal Jain
> >
>
>
>
> --
> Thanks and Regards,
> Vimal Jain
>

--
Thanks and Regards,
Vimal Jain