Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - ANN: The third hbase 0.94.0 release candidate is available for download


+
lars hofhansl 2012-04-18, 23:20
+
Jonathan Hsieh 2012-04-23, 17:46
+
lars hofhansl 2012-04-23, 22:37
+
Stack 2012-04-23, 17:54
+
lars hofhansl 2012-05-01, 23:26
+
Todd Lipcon 2012-05-02, 04:35
+
lars hofhansl 2012-05-02, 05:20
+
Elliott Clark 2012-05-02, 21:59
+
Ted Yu 2012-05-02, 22:01
+
Elliott Clark 2012-05-02, 22:07
+
Ted Yu 2012-05-02, 22:10
+
Mikael Sitruk 2012-05-03, 07:15
+
Elliott Clark 2012-05-03, 17:36
+
Elliott Clark 2012-05-04, 22:04
+
Ted Yu 2012-05-04, 22:07
+
Elliott Clark 2012-05-04, 22:14
+
Ted Yu 2012-05-05, 02:42
+
Elliott Clark 2012-05-07, 17:42
+
Todd Lipcon 2012-05-07, 17:47
+
Elliott Clark 2012-05-07, 18:07
+
Enis Söztutar 2012-05-07, 20:43
+
Elliott Clark 2012-05-08, 17:38
+
lars hofhansl 2012-05-08, 20:21
+
Elliott Clark 2012-05-08, 21:47
+
lars hofhansl 2012-05-10, 00:57
+
Ramkrishna.S.Vasudevan 2012-05-10, 15:41
+
Todd Lipcon 2012-05-10, 17:01
+
rama krishna 2012-05-10, 17:17
+
Andrew Purtell 2012-05-10, 16:09
+
Stack 2012-05-11, 04:15
+
Mikael Sitruk 2012-05-11, 05:04
+
Stack 2012-05-11, 05:18
+
Mikael Sitruk 2012-05-11, 10:24
+
Stack 2012-05-12, 05:02
+
Mikael Sitruk 2012-05-12, 17:14
+
Stack 2012-05-12, 21:50
+
Mikael Sitruk 2012-05-12, 22:14
+
lars hofhansl 2012-05-12, 05:26
+
Stack 2012-05-12, 05:39
+
lars hofhansl 2012-05-13, 17:22
+
Ramkrishna.S.Vasudevan 2012-05-14, 06:18
+
Ramkrishna.S.Vasudevan 2012-05-14, 13:59
+
Ted Yu 2012-05-14, 14:17
+
lars hofhansl 2012-05-14, 15:15
Copy link to this message
-
Re: ANN: The third hbase 0.94.0 release candidate is available for download
Todd Lipcon 2012-05-14, 15:17
On Mon, May 14, 2012 at 8:15 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
> It's default off. I'd say we just say it's an experimental feature in the release notes.

+1 for calling it experimental in notes and docs, and not removing it.
Replication was in an experimental state for quite some time, too, and
we didn't rip that out - I think shipping things off-by-default with
clear labeling is one of the best ways to sand down rough edges.

>
>
> Are you saying we should have another RC?
> There was other stuff that went into 0.94 after I cut the RC, so that would potentially need to stabilize if I cut a new RC now.
>
> -- Lars
>
> ________________________________
> From: Ted Yu <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Monday, May 14, 2012 7:17 AM
> Subject: Re: ANN: The third hbase 0.94.0 release candidate is available for download
>
> Thanks for sharing this information, Ramkrishna.
>
> Dictionary WAL compression makes replication not functional - see details
> in https://issues.apache.org/jira/browse/HBASE-5778
>
> I would vote for the removal of Dictionary WAL compression until we make it
> more robust and consuming much less memory.
>
> On Mon, May 14, 2012 at 6:59 AM, Ramkrishna.S.Vasudevan <
> [EMAIL PROTECTED]> wrote:
>
>> Hi
>>
>> One small observation after giving +1 on the RC.
>> The WAL compression feature causes OOME and causes Full GC.
>>
>> The problem is, if we have 1500 regions and I need to create
>> recovered.edits
>> for each of the region (I don’t have much data in the regions (~300MB)).
>> Now when I try to build the dictionary there is a Node object getting
>> created.
>> Each node object occupies 32 bytes.
>> We have 5 such dictionaries.
>>
>> Initially we create indexToNodes array and its size is 32767.
>>
>> So now we have 32*5*32767 = ~5MB.
>>
>> Now I have 1500 regions.
>>
>> So 5MB*1500 = ~7GB.(Excluding actual data).  This seems to a very high
>> initial memory foot print and this never allows me to split the logs and I
>> am not able to make the cluster up at all.
>>
>> Our configured heap size was 8GB, tested in 3 node cluster with 5000
>> regions, very less data( 1GB in hdfs cluster including replication), some
>> small data is spread evenly across all regions.
>>
>> The formula is 32(Node object size)*5(No of dictionary)*32767(no of node
>> objects)*noofregions.
>>
>> I think this initial memory needs to be documented (documentation should do
>> for now)or has to be fixed with some workarounds.
>>
>> So pls give your thoughts on this.
>>
>> Regards
>> Ram
>>
>>
>>
>>
>> > -----Original Message-----
>> > From: Ramkrishna.S.Vasudevan [mailto:[EMAIL PROTECTED]]
>> > Sent: Monday, May 14, 2012 11:48 AM
>> > To: [EMAIL PROTECTED]; 'lars hofhansl'
>> > Subject: RE: ANN: The third hbase 0.94.0 release candidate is available
>> > for download
>> >
>> > Hi
>> > We (it includes the test team here) 0.94 RC and carried out various
>> > operations on it.
>> > Puts, Scans, and all the restart scenarios (using kill -9 also).  Even
>> > the
>> > encoding stuffs were tested and carried out our basic test scenarios.
>> > Seems
>> > to work fine.
>> >
>> > Did not test rolling restart with 0.92. By this week we may try to do
>> > some
>> > performance comparison with 0.92.
>> > Also Lars and I agreed for a point release too.
>> > So I am +1 on the RC.
>> >
>> > Regards
>> > Ram
>> >
>> > > -----Original Message-----
>> > > From: lars hofhansl [mailto:[EMAIL PROTECTED]]
>> > > Sent: Sunday, May 13, 2012 10:53 PM
>> > > To: [EMAIL PROTECTED]
>> > > Subject: Re: ANN: The third hbase 0.94.0 release candidate is
>> > available
>> > > for download
>> > >
>> > > OK, I'll change my tactic :)
>> > >
>> > > If there are no -1's by Wed, May 16th, I'll release RC4 as 0.94.0.
>> > >
>> > > -- Lars
>> > >
>> > >
>> > >
>> > > ----- Original Message -----
>> > > From: Stack <[EMAIL PROTECTED]>
>> > > To: lars hofhansl <[EMAIL PROTECTED]>
>> > > Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>

Todd Lipcon
Software Engineer, Cloudera
+
Ramkrishna.S.Vasudevan 2012-05-14, 15:30
+
Jonathan Hsieh 2012-05-16, 16:04
+
Ramkrishna.S.Vasudevan 2012-05-16, 16:22