Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - How many column families in one table ?


+
Vimal Jain 2013-06-28, 05:20
+
Michel Segel 2013-06-28, 05:37
+
Ted Yu 2013-06-28, 12:27
+
Vimal Jain 2013-06-28, 12:40
+
Otis Gospodnetic 2013-06-28, 18:36
+
Vimal Jain 2013-07-01, 08:44
+
Viral Bajaria 2013-07-01, 08:54
+
Vimal Jain 2013-07-01, 09:11
+
Vimal Jain 2013-07-01, 11:11
+
lars hofhansl 2013-07-01, 11:18
Copy link to this message
-
Re: How many column families in one table ?
Vimal Jain 2013-07-01, 11:27
Hi Lars,
I am using Hadoop version - 1.1.2  and Hbase version - 0.94.7.
Yes , I have enabled scanner caching with value 10K but performance is not
too good. :(
On Mon, Jul 1, 2013 at 4:48 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> Which version of HBase?
> Did you enable scanner caching? Otherwise each call to next() is a RPC
> roundtrip and you are basically measuring your networks RTT.
>
> -- Lars
>
>
> ________________________________
>  From: Vimal Jain <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Monday, July 1, 2013 4:11 AM
> Subject: Re: How many column families in one table ?
>
>
> Can someone please reply ?
> Also what is  the typical read/write speed of hbase and how much deviation
> would be there in my scenario mentioned above (14 cf , total 140 columns )
> ?
> I am asking this because i am not simply printing out the scanned values ,
> instead i am applying some logic on the data retrieved per row basis. So
> was just curious to find if that small logic in my code is contributing
> towards the long time taken to scan the table.
>
>
> On Mon, Jul 1, 2013 at 2:41 PM, Vimal Jain <[EMAIL PROTECTED]> wrote:
>
> > I scanned it during normal traffic hours.There was no I/O load on the
> > server.
> > I dont see any GC locks too.
> > Also i have given 1.5G to RS , 512M to each Master and Zookeeper.
> >
> > One correction in the post above :
> > Actual time to scan whole table is even more , it takes 10 mins to scan
> > 0.1 million rows ( so total of 2.5 hours to scan 1.6 million rows) .
> > The time i mentioned in previous post was for different type of
> > lookup.Please ignore that.
> >
> >
> > On Mon, Jul 1, 2013 at 2:24 PM, Viral Bajaria <[EMAIL PROTECTED]
> >wrote:
> >
> >> When you did the scan, did you check what the bottleneck was ? Was it
> I/O
> >> ?
> >> Did you see any GC locks ? How much RAM are you giving to your RS ?
> >>
> >> -Viral
> >>
> >> On Mon, Jul 1, 2013 at 1:44 AM, Vimal Jain <[EMAIL PROTECTED]> wrote:
> >>
> >> > To completely scan the table for all 140 columns  , it takes around
> >> 30-40
> >> > minutes.
> >> >
> >>
> >
> >
> >
> > --
> > Thanks and Regards,
> > Vimal Jain
> >
>
>
>
> --
> Thanks and Regards,
> Vimal Jain
>

--
Thanks and Regards,
Vimal Jain
+
Ted Yu 2013-07-01, 11:39
+
Vimal Jain 2013-07-01, 11:44
+
lars hofhansl 2013-07-01, 15:38
+
Vimal Jain 2013-07-01, 17:03
+
Vimal Jain 2013-07-01, 17:06
+
Viral Bajaria 2013-07-01, 19:18
+
Vimal Jain 2013-08-04, 06:28
+
Kevin Odell 2013-08-04, 14:44
+
Inder Pall 2013-08-04, 14:55
+
Kevin Odell 2013-08-04, 15:04
+
Inder Pall 2013-08-04, 15:05
+
lars hofhansl 2013-08-04, 16:03
+
Rohit Kelkar 2013-08-04, 20:55
+
Pablo Medina 2013-08-05, 14:00
+
Kevin Odell 2013-08-05, 14:44
+
Michael Segel 2013-06-28, 12:42