Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - Homestretch for hadoop-0.22


Copy link to this message
-
Re: Homestretch for hadoop-0.22
Konstantin Boudnik 2011-10-28, 17:53
Konstantin,

Great news! The progress is pretty spectacular I'd say.

I am slowly working on Jetty back-port. Unfortunately, other things require my
attention too. Hopefully, it will be finished over next a couple days or so.

Cos

On Fri, Oct 28, 2011 at 02:29AM, Konstantin Shvachko wrote:
> Hi everybody,
>
> We reached the homestretch for hadoop-0.22 release.
> The testing proceeds very well.
> - The builds are building
> - HDFS and MR are running under load well now.
> - We applied pretty simple custom patches to make Hive (7) and Pig (8)
> work with 0.22.
> - HBase (92), which is expected to be released in the near future as I
> hear, is fully compatible with 0.22.
>
> There are 4 main items remaining on my plate.
>
> 1. We are down to four blockers, which represent three issues:
> HADOOP-7035. Document incompatible API changes between releases
> MAPREDUCE-1100 / 1716. User's task-logs filling up local disks on the
> TaskTrackers
> HDFS-2498. TestParallelRead times out
>
> I'd like also to include
> HDFS-1900. Clean up config key collision between common and HDFS
> and potentially further bug fixes.
>
> 2. Assemble build is needed.
> By assemble build I mean a Jenkins target which produces a tarball
> that includes all three sub-projects. It is like a daily build. One of
> this builds is assigned to be release a candidate and then the
> release. We can do it in traditional way, based on scripts developed
> in HADOOP-6846. Or we can try to do it with BigTop.
>
> 3. Jetty upgrade.
> 0.22 branch is on Jetty 6.1.14, while everything else moved to 6.1.26.
> Based on experiments I ran with Jetty 6.1.14 tasks hang forever if
> things go wrong, while with Jetty 6.1.26 you get a lot of "fetch
> failure"s. The latter at least fails fast, and does not need manual
> intervention, so I'd go for the upgrade.
>
> 4. Documentation.
> In addition to HADOOP-7035, which according to Tom needs a rerun of
> the tool, we need to confirm the documentation is up to date. I think
> it is, but re-checking would be good.
>
> I seek help from the good people of Hadoopland on these last steps.
> If you have suggestions for inclusion or plan to help with the items,
> please reply to the thread or drop me a line. I understand the next
> two weeks will be busy with conferences. I plan to release after
> Apachecon by mid November. If the assemble build is ready soon people
> will be able to try it early.
>
> Thanks,
> --Konstantin