Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # dev >> Patch for v1.1.2 (if this tree is under development)


+
Karl Gierach 2013-05-16, 00:25
Copy link to this message
-
Re: Patch for v1.1.2 (if this tree is under development)
Hello Karl,

Thank you for investigating the issue and preparing a patch!  Yes, the 1.x
line is still maintained.  (We just released 1.2.0.)  All patches do
require a corresponding issue in JIRA, so I recommend logging in to
https://issues.apache.org/jira/ and submitting this information in a new
issue there.  Additional details on contribution are available here:

http://wiki.apache.org/hadoop/HowToContribute

Regarding trunk, it appears that the current version of the code also would
suffer the same problem.  See
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/MapReduceChildJVM.java,
method getVMCommand:

https://github.com/apache/hadoop-common/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/MapReduceChildJVM.java#L156

The full scope of your JIRA issue likely would need to cover fixes for both
trunk and branch-1, as well as a unit test that covers this case.

Chris Nauroth
Hortonworks
http://hortonworks.com/

On Wed, May 15, 2013 at 5:25 PM, Karl Gierach <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Below is a patch for Hadoop v1.1.2.  I'm new to this list, so if I need to
> write up a JIRA ticket for this, please let me know.
>
> The defect scenario is that if you enter any white space within values in
> this file:
>
>     /etc/hadoop/mapred-site.xml
>
>
> e.g.: (a white space prior to the -X...)
>
>
>   <property>
>     <name>mapred.reduce.child.java.opts</name>
>     <value> -Xmx1G</value>
>   </property>
>
>
> All of the child jobs fail, and each child gets an error in the stderr log
> like:
>
> Could not find the main class: . Program will exit.
>
>
> The root cause is obvious in the patch below - the split on the value was
> done on whitespace, and any preceding whitespace ultimately becomes a
> zero-length entry on the child jvm command line, causing the jvm to think
> that a '' argument is the main class.   The patch just skips over any
> zero-length entries prior to adding them to the jvm vargs list.  I looked
> in trunk as well, to see if the patch would apply there but it looks like
> Tasks were refactored and this code file is not present any more.
>
>
> This error occurred on Open JDK, Centos 6.2, 32 bit.
>
>
> Regards,
> Karl
>
>
> Index: src/mapred/org/apache/hadoop/mapred/TaskRunner.java
> ==================================================================> --- src/mapred/org/apache/hadoop/mapred/TaskRunner.java    (revision
> 1482686)
> +++ src/mapred/org/apache/hadoop/mapred/TaskRunner.java    (working copy)
> @@ -437,7 +437,9 @@
>        vargs.add("-Djava.library.path=" + libraryPath);
>      }
>      for (int i = 0; i < javaOptsSplit.length; i++) {
> -      vargs.add(javaOptsSplit[i]);
> +      if( javaOptsSplit[i].trim().length() > 0 ) {
> +        vargs.add(javaOptsSplit[i]);
> +      }
>      }
>
>      Path childTmpDir = createChildTmpDir(workDir, conf, false);
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB