Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> Rejuvenate Hadoop 0.22 effort

Copy link to this message
Rejuvenate Hadoop 0.22 effort
Hi everybody,

I think there is no need to change anything drastically with the plans
for Hadoop 0.22 release, so I'll continue  along the lines previously
rendered by Nigel, discussed, and agreed upon within the community.

1. First thing, we need to resurrect Hadoop-0.22 Jenkins builds ASAP.
Does anybody want to help with that? Any help is greatly appreciated.

2. I will start sorting out jiras currently assigned to 0.22. There
are 10 blockers (again) over the three projects. Details below. My
plan is to get the release candidate out late October.

3. Let's start discussing what else people think needs to be included
in 0.22. I will include issues based on the following priorities
- build fixes,
- test failures,
- bug fixes,
- documentation
- compatibility issues directed to making H-0.22 work with other
project (HBase, Pig, Hive, Oozie)
- minor improvements (irritating for users but simple)
- no new features in 0.22.0, but I'd like to have a list of things
which people would've considered for 0.22.1

4. I will use the following filter to watch the jiras assigned to the release:
project in (HADOOP, HDFS, MAPREDUCE) AND resolution = Unresolved AND
fixVersion = "0.22.0" ORDER BY priority DESC
If you think an issue should be considered for inclusion please set
fixVersion = "0.22.0". I will mark them as blockers based on the
priorities above and my common sense.
Note, if the jira is consciously assigned to a contributor it has high
chance to make into the blockers.

== TESTING =5. I think Steve's idea of integrating 0.22 with Apache BigTop is
great. Will be glad to see any steps in this direction.

6. Hadoop-0.22 is being tested since January 2011. We conducted some
internal testing lately. Testing is proceeding now on a dev cluster.
If anybody plans to setup a cluster for testing and wants to
coordinate the efforts please ping me.

== 10 BLOCKERS =7. There are 10 official blockers.

Key Assignee Summary
MAPREDUCE-1991 Todd Lipcon taskcontroller allows stealing permissions
on any local file
MAPREDUCE-2178 Devaraj Das Race condition in LinuxTaskController
permissions handling
MAPREDUCE-2266 Unassigned JvmManager sleeps between SIGTERM and
SIGKILL while holding many TT locks

I will unblock TaskController issues as per discussion related to

MAPREDUCE-1100 Vinod Kumar User's task-logs filling up local disks on
the TaskTrackers
MAPREDUCE-1716 Vinod Kumar MAPREDUCE-1100 Truncate logs of finished
tasks to prevent node thrash due to excessive logging

Don't see any activity from Vinod. Any volunteers to port this to 0.22?

HADOOP-7035 Tom White Document incompatible API changes between releases

Looks like close to completion. Tom are you still on it?

MAPREDUCE-2268 Todd Lipcon With JVM reuse, JvmManager doesn't delete
last workdir properly

Todd, is it a blocker? Do you plan to fix it soon?

MAPREDUCE-1506 Unassigned Assertion failure in TestTaskTrackerMemoryManager

Will unblock, as no volunteers emerged.

HDFS-1967 Unassigned HDFS-1852 TestHDFSTrash failing on trunk and 22
HDFS-2012 Unassigned Recurring failure of TestBalancer on branch-0.22

Don't see failures anymore. Will follow up when Jenkins builds are restored

HDFS-2290 Benoy Antony Block with corrupt replica is not getting replicated

Close to completion.