Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> ANN: The third hbase 0.94.0 release candidate is available for download


Copy link to this message
-
Re: ANN: The third hbase 0.94.0 release candidate is available for download
On Mon, May 14, 2012 at 8:15 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
> It's default off. I'd say we just say it's an experimental feature in the release notes.

+1 for calling it experimental in notes and docs, and not removing it.
Replication was in an experimental state for quite some time, too, and
we didn't rip that out - I think shipping things off-by-default with
clear labeling is one of the best ways to sand down rough edges.

>
>
> Are you saying we should have another RC?
> There was other stuff that went into 0.94 after I cut the RC, so that would potentially need to stabilize if I cut a new RC now.
>
> -- Lars
>
> ________________________________
> From: Ted Yu <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Monday, May 14, 2012 7:17 AM
> Subject: Re: ANN: The third hbase 0.94.0 release candidate is available for download
>
> Thanks for sharing this information, Ramkrishna.
>
> Dictionary WAL compression makes replication not functional - see details
> in https://issues.apache.org/jira/browse/HBASE-5778
>
> I would vote for the removal of Dictionary WAL compression until we make it
> more robust and consuming much less memory.
>
> On Mon, May 14, 2012 at 6:59 AM, Ramkrishna.S.Vasudevan <
> [EMAIL PROTECTED]> wrote:
>
>> Hi
>>
>> One small observation after giving +1 on the RC.
>> The WAL compression feature causes OOME and causes Full GC.
>>
>> The problem is, if we have 1500 regions and I need to create
>> recovered.edits
>> for each of the region (I don’t have much data in the regions (~300MB)).
>> Now when I try to build the dictionary there is a Node object getting
>> created.
>> Each node object occupies 32 bytes.
>> We have 5 such dictionaries.
>>
>> Initially we create indexToNodes array and its size is 32767.
>>
>> So now we have 32*5*32767 = ~5MB.
>>
>> Now I have 1500 regions.
>>
>> So 5MB*1500 = ~7GB.(Excluding actual data).  This seems to a very high
>> initial memory foot print and this never allows me to split the logs and I
>> am not able to make the cluster up at all.
>>
>> Our configured heap size was 8GB, tested in 3 node cluster with 5000
>> regions, very less data( 1GB in hdfs cluster including replication), some
>> small data is spread evenly across all regions.
>>
>> The formula is 32(Node object size)*5(No of dictionary)*32767(no of node
>> objects)*noofregions.
>>
>> I think this initial memory needs to be documented (documentation should do
>> for now)or has to be fixed with some workarounds.
>>
>> So pls give your thoughts on this.
>>
>> Regards
>> Ram
>>
>>
>>
>>
>> > -----Original Message-----
>> > From: Ramkrishna.S.Vasudevan [mailto:[EMAIL PROTECTED]]
>> > Sent: Monday, May 14, 2012 11:48 AM
>> > To: [EMAIL PROTECTED]; 'lars hofhansl'
>> > Subject: RE: ANN: The third hbase 0.94.0 release candidate is available
>> > for download
>> >
>> > Hi
>> > We (it includes the test team here) 0.94 RC and carried out various
>> > operations on it.
>> > Puts, Scans, and all the restart scenarios (using kill -9 also).  Even
>> > the
>> > encoding stuffs were tested and carried out our basic test scenarios.
>> > Seems
>> > to work fine.
>> >
>> > Did not test rolling restart with 0.92. By this week we may try to do
>> > some
>> > performance comparison with 0.92.
>> > Also Lars and I agreed for a point release too.
>> > So I am +1 on the RC.
>> >
>> > Regards
>> > Ram
>> >
>> > > -----Original Message-----
>> > > From: lars hofhansl [mailto:[EMAIL PROTECTED]]
>> > > Sent: Sunday, May 13, 2012 10:53 PM
>> > > To: [EMAIL PROTECTED]
>> > > Subject: Re: ANN: The third hbase 0.94.0 release candidate is
>> > available
>> > > for download
>> > >
>> > > OK, I'll change my tactic :)
>> > >
>> > > If there are no -1's by Wed, May 16th, I'll release RC4 as 0.94.0.
>> > >
>> > > -- Lars
>> > >
>> > >
>> > >
>> > > ----- Original Message -----
>> > > From: Stack <[EMAIL PROTECTED]>
>> > > To: lars hofhansl <[EMAIL PROTECTED]>
>> > > Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>

Todd Lipcon
Software Engineer, Cloudera
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB