Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Maven Cloudera Configuration problem


Copy link to this message
-
Re: Maven Cloudera Configuration problem
You need to configure your namenode and jobtracker information in the
configuration files within you application. Only set the relevant
properties in the copy of the files that you are bundling in your job. For
rest the default values would be used from the default configuration files
(core-default.xml, mapred-default.xml) already bundled in the lib/jar
provided by cloudera/hadoop. The assumption is that this is for MRv1.

Anyway, you should go through this for details
http://hadoop.apache.org/docs/stable/cluster_setup.html

*core-site.xml *(teh security ones are optional and if you are not using
anything special you can remove them and rely on the defaults which is also
'simple'.

<configuration>

  <property>

    <name>fs.defaultFS</name>

    <value>hdfs://server:8020</value>

  </property>

  <property>

    <name>hadoop.security.authentication</name>

    <value>simple</value>

  </property>

  <property>

    <name>hadoop.security.auth_to_local</name>

    <value>DEFAULT</value>

  </property>

</configuration>
*map-red.xml*

</configuration>

  <property>

    <name>mapred.job.tracker</name>

    <value>http://server:</value>

  </property>

</configuration>*
*
Regards,
Shahab

*
*
On Tue, Aug 13, 2013 at 7:19 AM, Pavan Sudheendra <[EMAIL PROTECTED]>wrote:

> Hi,
> I'm currently using maven to build the jars necessary for my
> map-reduce program to run and it works for a single node cluster..
>
> For a multi node cluster, how do i specify my map-reduce program to
> ingest the cluster settings instead of localhost settings?
> I don't know how to specify this using maven to build my jar.
>
> I'm using the cdh distribution by the way..
> --
> Regards-
> Pavan
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB