Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # dev >> Patch for v1.1.2 (if this tree is under development)


+
Karl Gierach 2013-05-16, 00:25
Copy link to this message
-
Re: Patch for v1.1.2 (if this tree is under development)
Hello Karl,

Thank you for investigating the issue and preparing a patch!  Yes, the 1.x
line is still maintained.  (We just released 1.2.0.)  All patches do
require a corresponding issue in JIRA, so I recommend logging in to
https://issues.apache.org/jira/ and submitting this information in a new
issue there.  Additional details on contribution are available here:

http://wiki.apache.org/hadoop/HowToContribute

Regarding trunk, it appears that the current version of the code also would
suffer the same problem.  See
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/MapReduceChildJVM.java,
method getVMCommand:

https://github.com/apache/hadoop-common/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/MapReduceChildJVM.java#L156

The full scope of your JIRA issue likely would need to cover fixes for both
trunk and branch-1, as well as a unit test that covers this case.

Chris Nauroth
Hortonworks
http://hortonworks.com/

On Wed, May 15, 2013 at 5:25 PM, Karl Gierach <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Below is a patch for Hadoop v1.1.2.  I'm new to this list, so if I need to
> write up a JIRA ticket for this, please let me know.
>
> The defect scenario is that if you enter any white space within values in
> this file:
>
>     /etc/hadoop/mapred-site.xml
>
>
> e.g.: (a white space prior to the -X...)
>
>
>   <property>
>     <name>mapred.reduce.child.java.opts</name>
>     <value> -Xmx1G</value>
>   </property>
>
>
> All of the child jobs fail, and each child gets an error in the stderr log
> like:
>
> Could not find the main class: . Program will exit.
>
>
> The root cause is obvious in the patch below - the split on the value was
> done on whitespace, and any preceding whitespace ultimately becomes a
> zero-length entry on the child jvm command line, causing the jvm to think
> that a '' argument is the main class.   The patch just skips over any
> zero-length entries prior to adding them to the jvm vargs list.  I looked
> in trunk as well, to see if the patch would apply there but it looks like
> Tasks were refactored and this code file is not present any more.
>
>
> This error occurred on Open JDK, Centos 6.2, 32 bit.
>
>
> Regards,
> Karl
>
>
> Index: src/mapred/org/apache/hadoop/mapred/TaskRunner.java
> ==================================================================> --- src/mapred/org/apache/hadoop/mapred/TaskRunner.java    (revision
> 1482686)
> +++ src/mapred/org/apache/hadoop/mapred/TaskRunner.java    (working copy)
> @@ -437,7 +437,9 @@
>        vargs.add("-Djava.library.path=" + libraryPath);
>      }
>      for (int i = 0; i < javaOptsSplit.length; i++) {
> -      vargs.add(javaOptsSplit[i]);
> +      if( javaOptsSplit[i].trim().length() > 0 ) {
> +        vargs.add(javaOptsSplit[i]);
> +      }
>      }
>
>      Path childTmpDir = createChildTmpDir(workDir, conf, false);