-Homestretch for hadoop-0.22
Konstantin Shvachko 2011-10-28, 09:29
We reached the homestretch for hadoop-0.22 release.
The testing proceeds very well.
- The builds are building
- HDFS and MR are running under load well now.
- We applied pretty simple custom patches to make Hive (7) and Pig (8)
work with 0.22.
- HBase (92), which is expected to be released in the near future as I
hear, is fully compatible with 0.22.
There are 4 main items remaining on my plate.
1. We are down to four blockers, which represent three issues:
HADOOP-7035. Document incompatible API changes between releases
MAPREDUCE-1100 / 1716. User's task-logs filling up local disks on the
HDFS-2498. TestParallelRead times out
I'd like also to include
HDFS-1900. Clean up config key collision between common and HDFS
and potentially further bug fixes.
2. Assemble build is needed.
By assemble build I mean a Jenkins target which produces a tarball
that includes all three sub-projects. It is like a daily build. One of
this builds is assigned to be release a candidate and then the
release. We can do it in traditional way, based on scripts developed
in HADOOP-6846. Or we can try to do it with BigTop.
3. Jetty upgrade.
0.22 branch is on Jetty 6.1.14, while everything else moved to 6.1.26.
Based on experiments I ran with Jetty 6.1.14 tasks hang forever if
things go wrong, while with Jetty 6.1.26 you get a lot of "fetch
failure"s. The latter at least fails fast, and does not need manual
intervention, so I'd go for the upgrade.
In addition to HADOOP-7035, which according to Tom needs a rerun of
the tool, we need to confirm the documentation is up to date. I think
it is, but re-checking would be good.
I seek help from the good people of Hadoopland on these last steps.
If you have suggestions for inclusion or plan to help with the items,
please reply to the thread or drop me a line. I understand the next
two weeks will be busy with conferences. I plan to release after
Apachecon by mid November. If the assemble build is ready soon people
will be able to try it early.