Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Building and Deploying MRv2


Copy link to this message
-
Re: Building and Deploying MRv2

Thanks to the group. Finally, I got the build and deploying going with
the following exception during the common build

      [exec] validate-sitemap:
      [exec]
/home/praveensripati/Installations/apache-forrest-0.8/main/webapp/resources/schema/relaxng/sitemap-v06.rng:72:31:
error: datatype library "http://www.w3.org/2001/XMLSchema-datatypes" not
recognized

      [exec] BUILD FAILED
      [exec]
/home/praveensripati/Installations/apache-forrest-0.8/main/targets/validate.xml:158:
Validation failed, messages should have been provided.

------

The INSTALL doc  
(http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL)
is incomplete. Here are the additional steps

- Dependency on the apache-forrest-0.8.tar.gz, apache-forrest-0.9.tar.gz
is causing ClassNotFound Exceptions for the Common build.

- The hadoop-mapred-examples-0.22.0-SNAPSHOT.jar is not there in the
$HADOOP_MAPRED_HOME/build. "ant examples -Dresolvers=internal" had to be
run.

- The configuration files in the below folder have to be copied to a
configuration folder and the same folder should be set to the
HADOOP_CONF_DIR variable and exported.

     - MR-279/common/conf
     - MR-279/hdfs/conf
     - MR-279/mapreduce/conf
     - $YARN_INSTALL/conf

- Had to install autoconf 'sudo apt-get install autoconf"on Ubuntu 11.04.

Thanks,
Praveen
On Saturday 18 June 2011 05:22 PM, Praveen Sripati wrote:
> Hi,
>
> I have got the code from the svn into into
> /home/praveensripati/Hadoop/ directory and untar'd the
> hadoop-mapreduce-1.0-SNAPSHOT-all.tar.gz file in the
> /home/praveensripati/Hadoop folder. The INSTALL document says to
> export the following variables. What should the HADOOP_CONF_DIR
> variable be set to - common or hdfs or mapreduce conf folder? The
> YARN_CONF_DIR is again pointing to the HADOOP_CONF_DIR.
>
> export HADOOP_MAPRED_HOME=/home/praveensripati/Hadoop/MR-279/mapreduce
> export HADOOP_COMMON_HOME=/home/praveensripati/Hadoop/MR-279/common
> export HADOOP_HDFS_HOME=/home/praveensripati/Hadoop/MR-279/hdfs
> export YARN_HOME=/home/praveensripati/Hadoop/hadoop-mapreduce-1.0-SNAPSHOT
> export HADOOP_CONF_DIR> export YARN_CONF_DIR=$HADOOP_CONF_DIR
> Thanks,
> Praveen
>
> On Saturday 18 June 2011 08:52 AM, Praveen Sripati wrote:
>>
>> Hi,
>>
>> Finally, got all the jars built. Now is the time to run the MRv2.
>>
>> It would be nice if the below documentation gets updated
>>
>> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL
>>
>> ----------
>>
>> yarn build
>>
>> Ubuntu had an older version of the protoc binary, I did put the
>> latest protoc binary which I build in the PATH and the following
>> error is not coming
>>
>> yarn_protos.proto:4:8: Option "java_generate_equals_and_hash" unknown.
>>
>> Then had to install autoconf 'sudo apt-get install autoconf" and the
>> hadoop-mapreduce-1.0-SNAPSHOT-all.tar.gz file got generated.
>>
>> ----------
>>
>> common build
>>
>> After including the forrestor 8, the following error is not coming
>> (HADOOP-7394 has been created for the same)
>>
>>       [exec] Exception in thread "main"
>> java.lang.NoClassDefFoundError: org/apache/fop/messaging/MessageHandler
>>       [exec]     at
>> org.apache.cocoon.serialization.FOPSerializer.configure(FOPSerializer.java:122)
>>
>>
>> But, now getting the following error (the jars are getting generated
>> in the build folder)
>>
>>      [exec] validate-sitemap:
>>      [exec]
>> /home/praveensripati/Installations/apache-forrest-0.8/main/webapp/resources/schema/relaxng/sitemap-v06.rng:72:31:
>> error: datatype library "http://www.w3.org/2001/XMLSchema-datatypes"
>> not recognized
>>
>>      [exec] BUILD FAILED
>>      [exec]
>> /home/praveensripati/Installations/apache-forrest-0.8/main/targets/validate.xml:158:
>> Validation failed, messages should have been provided.
>> Thanks,
>> Praveen
>>
>> On Saturday 18 June 2011 03:44 AM, Siddharth Seth wrote:
>>> Ubuntu seems to install the protocol buffer library (protobuf-compiler) as
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB