All,
  Matt rolled a 0.20.204.0rc1, but I think it got lost in the previous vote thread. Unfortunately, it had the version as 0.20.204 and didn't update the release notes. I've updated it, run the regression tests and I think we should release it. I've put the tarball up at:

http://people.apache.org/~omalley/hadoop-0.20.204.0-rc2

Please check it out and vote on it.

-- Owen

On Aug 9, 2011, at 8:55 AM, Owen O'Malley wrote:

> All,
>  Matt rolled a 0.20.204.0rc1, but I think it got lost in the previous vote thread. Unfortunately, it had the version as 0.20.204 and didn't update the release notes. I've updated it, run the regression tests and I think we should release it. I've put the tarball up at:
>
> http://people.apache.org/~omalley/hadoop-0.20.204.0-rc2
>
> Please check it out and vote on it.

-1 (non-binding)

https://issues.apache.org/jira/browse/HDFS-2244

On Aug 9, 2011, at 8:55 AM, Owen O'Malley wrote:

> All,
>  Matt rolled a 0.20.204.0rc1, but I think it got lost in the previous vote thread. Unfortunately, it had the version as 0.20.204 and didn't update the release notes. I've updated it, run the regression tests and I think we should release it. I've put the tarball up at:
>
> http://people.apache.org/~omalley/hadoop-0.20.204.0-rc2
>
> Please check it out and vote on it.
After playing with it this afternoon, it would be good if other people did too.  I've already talked to Owen about some of the weirdness I'm seeing, but it would be useful to see if other people can independently verify.  I'm not going to tell you what they are... but run a sizable job and see if you see anything.... odd. :)

On Aug 9, 2011, at 8:55 AM, Owen O'Malley wrote:

> All,
>  Matt rolled a 0.20.204.0rc1, but I think it got lost in the previous vote thread. Unfortunately, it had the version as 0.20.204 and didn't update the release notes. I've updated it, run the regression tests and I think we should release it. I've put the tarball up at:
>
> http://people.apache.org/~omalley/hadoop-0.20.204.0-rc2

This vote is still running with no votes other than mine.

I've tested with and without security on a 60 node cluster and I'm seeing some failures, but not that many. On a terasort with 15,000 maps and 200 reduces, I ran the following cases:

security + linux task controller : 2 failures (both mr-2651)

no security + default task controller : 6-7 failures (seems to be a race condition in clean up)

Even in the no security case, it is only losing 0.05% of the time.

It isn't perfect, but this is the code that Yahoo is currently running. I think we should release it.

-- Owen

On Aug 18, 2011, at 12:28 AM, Owen O'Malley wrote:
>
> This vote is still running with no votes other than mine.
>
> I've tested with and without security on a 60 node cluster and I'm seeing some failures, but not that many. On a terasort with 15,000 maps and 200 reduces, I ran the following cases:
>
> security + linux task controller : 2 failures (both mr-2651)
>
> no security + default task controller : 6-7 failures (seems to be a race condition in clean up)
>
> Even in the no security case, it is only losing 0.05% of the time.

We're seeing much much higher failure rates.  In the 5-10% area.  It might very well be because we have more cores/faster boxes.

> It isn't perfect, but this is the code that Yahoo is currently running. I think we should release it.

Y! can afford the task failures.  The rest of us can't.  So -1.

>> This vote is still running with no votes other than mine.

The vote was started 9 days ago and, if it kept running after Allen's
vote, it would have ended three days ago with the result of 1-0 to
release.  As such, let's call this release 2Owen3.

Jokes aside,
  I intend to try it out on a set of 10 nodes or so and see if it works or not. If it works fine (barring a few task failures), I dont see any reason why this cannot be released.
 
thanks
mahadev

On Aug 18, 2011, at 11:35 AM, Jakob Homan wrote:

>>> This vote is still running with no votes other than mine.
>
> The vote was started 9 days ago and, if it kept running after Allen's
> vote, it would have ended three days ago with the result of 1-0 to
> release.  As such, let's call this release 2Owen3.

+1 (non-binding)
 
I downloaded the release candidate 2 onto a 12-nod cluster performed the following tests:
 
##################################################################
Create files using S-Live:
     hadoop --config $HADOOP_CONF_DIR org.apache.hadoop.fs.slive.SliveTest -appendSize 1,67108864 -append 0,uniform -baseDir /user/$USER/S-Live -blockSize 67108864,67108864 -create 100,uniform -delete 0,uniform -dirSize 16 -duration 300 -files 1024 -ls 0,uniform -maps 20 -mkdir 0,uniform -ops 10000 -packetSize 65536 -readSize 1,4294967295 -read 0,uniform -reduces 5 -rename 0,uniform -replication 1,3 -resFile $outFile -seed 12345678 -sleep 100,1000 -writeSize 1,67108864
     Output:
Basic report for operation type CreateOp
-------------
Measurement "bytes_written" = 32046140254
Measurement "milliseconds_taken" = 233234
Measurement "op_count" = 7929
Rate for measurement "bytes_written" = 131.031 MB/sec
Rate for measurement "op_count" = 33.996 operations/sec

##########################################################
 
Do random deletes, reads, mkdirs, lists, and renames using S-Live:
    hadoop --config $HADOOP_CONF_DIR org.apache.hadoop.fs.slive.SliveTest -appendSize 1,67108864 -append 0,uniform -baseDir /user/$USER/S-Live -blockSize 67108864,67108864 -create 0,uniform -delete 20,uniform -dirSize 16 -duration 300 -files 1024 -ls 20,uniform -maps 20 -mkdir 20,uniform -ops 10000 -packetSize 65536 -readSize 1,4294967295 -read 20,uniform -reduces 5 -rename 20,uniform -replication 1,3 -resFile $outFile -seed 12345678 -sleep 100,1000 -writeSize 1,67108864
     Output:
Basic report for operation type DeleteOp
-------------
Measurement "milliseconds_taken" = 6008007
Measurement "op_count" = 27148
Rate for measurement "op_count" = 4.519 operations/sec
-------------

#########################################################################
 
     I also ran randomwriter.
 
Thanks,
-Eric Payne
 
 
------ Forwarded Message
From: Owen O'Malley <[EMAIL PROTECTED]>
Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Date: Thu, 18 Aug 2011 00:28:20 -0700
To: Owen O'Malley <[EMAIL PROTECTED]>
Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Subject: Re: [VOTE] Should we release 0.20.204.0rc2?
On Aug 9, 2011, at 8:55 AM, Owen O'Malley wrote:

> All,
>  Matt rolled a 0.20.204.0rc1, but I think it got lost in the previous vote
thread. Unfortunately, it had the version as 0.20.204 and didn't update the
release notes. I've updated it, run the regression tests and I think we should
release it. I've put the tarball up at:
>
> http://people.apache.org/~omalley/hadoop-0.20.204.0-rc2

This vote is still running with no votes other than mine.

I've tested with and without security on a 60 node cluster and I'm seeing
some failures, but not that many. On a terasort with 15,000 maps and 200
reduces, I ran the following cases:

security + linux task controller : 2 failures (both mr-2651)

no security + default task controller : 6-7 failures (seems to be a race
condition in clean up)

Even in the no security case, it is only losing 0.05% of the time.

It isn't perfect, but this is the code that Yahoo is currently running. I
think we should release it.

-- Owen

------ End of Forwarded Message

All,
   I've fixed the issues that Allen observed in the previous rc for 0.20.204 and rolled the new bundled up in http://people.apache.org/~omalley/hadoop-0.20.204.0-rc3 . Please download the tarball, compile it,  and try it out. All of the tests pass, and I've run several 1TB sorts with 15,000 maps and 110 reduces with only 1 task failures out of 3 runs.

Thanks,
   Owen

+1 (non-binding), tested several map reduce jobs on 50 nodes with security enabled.  No failure observed in tera gen and tera sort tasks.  RPM tested on CentOS 5.6, and Deb package tested on Ubuntu 11.4.

regards,
Eric

On Aug 25, 2011, at 5:12 PM, Owen O'Malley wrote:

> All,
>   I've fixed the issues that Allen observed in the previous rc for 0.20.204 and rolled the new bundled up in http://people.apache.org/~omalley/hadoop-0.20.204.0-rc3 . Please download the tarball, compile it,  and try it out. All of the tests pass, and I've run several 1TB sorts with 15,000 maps and 110 reduces with only 1 task failures out of 3 runs.
>
> Thanks,
>   Owen

+1

I installed the 204 release on my machine (non secure setup). Did HDFS
sanity tests using command line. Tested namenode and jobtracker web ui
pages. Ran TestDFSIO write and read tests and mrbench tests.

+1 for 204 from me.

Regards,
Suresh

+1.

Downloaded the release, brought up a single node cluster and ran a  randomwriter job. Also ran ant test and it passed.

thanks
mahadev

On Aug 26, 2011, at 10:08 AM, Eric Yang wrote:

> +1 (non-binding), tested several map reduce jobs on 50 nodes with security enabled.  No failure observed in tera gen and tera sort tasks.  RPM tested on CentOS 5.6, and Deb package tested on Ubuntu 11.4.
>
> regards,
> Eric
>
> On Aug 25, 2011, at 5:12 PM, Owen O'Malley wrote:
>
>> All,
>>  I've fixed the issues that Allen observed in the previous rc for 0.20.204 and rolled the new bundled up in http://people.apache.org/~omalley/hadoop-0.20.204.0-rc3 . Please download the tarball, compile it,  and try it out. All of the tests pass, and I've run several 1TB sorts with 15,000 maps and 110 reduces with only 1 task failures out of 3 runs.
>>
>> Thanks,
>>  Owen
>

+1

Downloaded bits, verified checksums and ran a single node cluster.

Arun

On Aug 25, 2011, at 5:12 PM, Owen O'Malley wrote:

> All,
>   I've fixed the issues that Allen observed in the previous rc for 0.20.204 and rolled the new bundled up in http://people.apache.org/~omalley/hadoop-0.20.204.0-rc3 . Please download the tarball, compile it,  and try it out. All of the tests pass, and I've run several 1TB sorts with 15,000 maps and 110 reduces with only 1 task failures out of 3 runs.
>
> Thanks,
>   Owen

+1
  downloaded, ran tests, ran job on 1 node cluster on my machine.

sanjay

On Aug 25, 2011, at 5:12 PM, Owen O'Malley wrote:

> All,
>   I've fixed the issues that Allen observed in the previous rc for 0.20.204 and rolled the new bundled up in http://people.apache.org/~omalley/hadoop-0.20.204.0-rc3 . Please download the tarball, compile it,  and try it out. All of the tests pass, and I've run several 1TB sorts with 15,000 maps and 110 reduces with only 1 task failures out of 3 runs.
>
> Thanks,
>   Owen

WIth 6 +1's (4 binding) and no -1's the vote passes and we have an official
0.20.204.0. Thanks to everyone who contributed code or testing to the
release!

-- Owen

+1 (non-binding)
 
I downloaded the patch and installed it on a 10-node cluster.
 
I successfully ran randomwriter twice and the following 2 SLive tests:
 
hadoop --config $HADOOP_CONF_DIR org.apache.hadoop.fs.slive.SliveTest \
    -appendSize 1,67108864 -append 0,uniform -baseDir /user/$USER/S-Live \
    -blockSize 67108864,67108864 -create 0,uniform -delete 20,uniform -dirSize 16 \
    -duration 300 -files 1024 -ls 20,uniform -maps 20 -mkdir 20,uniform -ops 10000 \
    -packetSize 65536 -readSize 1,4294967295 -read 20,uniform -reduces 5 \
    -rename 20,uniform -replication 1,3 -resFile $RESFILE \
    -seed 12345678 -sleep 100,1000 -writeSize 1,67108864

hadoop --config $HADOOP_CONF_DIR org.apache.hadoop.fs.slive.SliveTest \
    -appendSize 1,67108864 -append 0,uniform -baseDir /user/$USER/S-Live \
    -blockSize 67108864,67108864 -create 100,uniform -delete 0,uniform -dirSize 16 \
    -duration 300 -files 1024 -ls 0,uniform -maps 20 -mkdir 0,uniform -ops 10000 \
    -packetSize 65536 -readSize 1,4294967295 -read 0,uniform -reduces 5 \
    -rename 0,uniform -replication 1,3 -resFile $RESFILE -seed 12345678 \
    -sleep 100,1000 -writeSize 1,67108864

Thanks,
-Eric Payne
 
----------------------
From: Owen O'Malley [[EMAIL PROTECTED]]
Sent: Thu 8/25/2011 7:12 PM
To: [EMAIL PROTECTED]
Subject: [VOTE] Should we release 0.20.204.0-rc3?
 
 
All,
   I've fixed the issues that Allen observed in the previous rc for 0.20.204 and rolled the new bundled up in http://people.apache.org/~omalley/hadoop-0.20.204.0-rc3. Please download the tarball, compile it,  and try it out. All of the tests pass, and I've run several 1TB sorts with 15,000 maps and 110 reduces with only 1 task failures out of 3 runs.
 
Thanks,
   Owen