Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Need YARN - Test job.


Copy link to this message
-
Re: Need YARN - Test job.
Hi Dhahasekaran,

You can use the Hadoop workload to do this testing. The key is to set the
HADOOP_CONF_DIR envar to the right location:

*Yarn*

export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
hadoop fs -rm -r /terasort/TS_input2
yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
teragen  -D dfs.replication=1 -Dmapred.map.tasks=144    10000
/terasort/TS_input2

*MapReduce*
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.mapreduce1
hadoop fs -rm -r /terasort/TS_input2
yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
teragen  -D dfs.replication=1 -Dmapred.map.tasks=144    10000
/terasort/TS_input2
I hope this helps.
Thanks,

Gunnar
On Thu, Jan 2, 2014 at 3:03 PM, Dhanasekaran Anbalagan
<[EMAIL PROTECTED]>wrote:

> Hi Guys,
>
> we recently installed CDH5 in our test cluster. we need any test job for
> YARN framework.
>
> we are able to run mapreduce job successfully on YARN framework without
> code change.
>
> But we need to test yarn job functionality. can you please guide me.
>
> we tired https://github.com/hortonworks/simple-yarn-app it's not help us.
> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
> com.hortonworks.simpleyarnapp.Client /bin/date 2
> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
> dvcloudlab231/192.168.70.231:8032
> Submitting application application_1388687890867_0007
> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
> 192.168.70.231:8032
> Application application_1388687890867_0007 finished with state FAILED at
> 1388699348344
>
> Note:
> In Resource manager node. I don't see any container logs.
>
> -Dhanasekaran.
>
> Did I learn something today? If not, I wasted it.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+[EMAIL PROTECTED].
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>

--
Thanks,

Gunnar
*If you think you can you can, if you think you can't you're right.*
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB