Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> How many column families in one table ?


Copy link to this message
-
Re: How many column families in one table ?
Hi Lars,
I am using Hadoop version - 1.1.2  and Hbase version - 0.94.7.
Yes , I have enabled scanner caching with value 10K but performance is not
too good. :(
On Mon, Jul 1, 2013 at 4:48 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> Which version of HBase?
> Did you enable scanner caching? Otherwise each call to next() is a RPC
> roundtrip and you are basically measuring your networks RTT.
>
> -- Lars
>
>
> ________________________________
>  From: Vimal Jain <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Monday, July 1, 2013 4:11 AM
> Subject: Re: How many column families in one table ?
>
>
> Can someone please reply ?
> Also what is  the typical read/write speed of hbase and how much deviation
> would be there in my scenario mentioned above (14 cf , total 140 columns )
> ?
> I am asking this because i am not simply printing out the scanned values ,
> instead i am applying some logic on the data retrieved per row basis. So
> was just curious to find if that small logic in my code is contributing
> towards the long time taken to scan the table.
>
>
> On Mon, Jul 1, 2013 at 2:41 PM, Vimal Jain <[EMAIL PROTECTED]> wrote:
>
> > I scanned it during normal traffic hours.There was no I/O load on the
> > server.
> > I dont see any GC locks too.
> > Also i have given 1.5G to RS , 512M to each Master and Zookeeper.
> >
> > One correction in the post above :
> > Actual time to scan whole table is even more , it takes 10 mins to scan
> > 0.1 million rows ( so total of 2.5 hours to scan 1.6 million rows) .
> > The time i mentioned in previous post was for different type of
> > lookup.Please ignore that.
> >
> >
> > On Mon, Jul 1, 2013 at 2:24 PM, Viral Bajaria <[EMAIL PROTECTED]
> >wrote:
> >
> >> When you did the scan, did you check what the bottleneck was ? Was it
> I/O
> >> ?
> >> Did you see any GC locks ? How much RAM are you giving to your RS ?
> >>
> >> -Viral
> >>
> >> On Mon, Jul 1, 2013 at 1:44 AM, Vimal Jain <[EMAIL PROTECTED]> wrote:
> >>
> >> > To completely scan the table for all 140 columns  , it takes around
> >> 30-40
> >> > minutes.
> >> >
> >>
> >
> >
> >
> > --
> > Thanks and Regards,
> > Vimal Jain
> >
>
>
>
> --
> Thanks and Regards,
> Vimal Jain
>

--
Thanks and Regards,
Vimal Jain
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB