Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Need YARN - Test job.


Copy link to this message
-
Re: Need YARN - Test job.
Hi Dhahasekaran,

You can use the Hadoop workload to do this testing. The key is to set the
HADOOP_CONF_DIR envar to the right location:

*Yarn*

export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
hadoop fs -rm -r /terasort/TS_input2
yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
teragen  -D dfs.replication=1 -Dmapred.map.tasks=144    10000
/terasort/TS_input2

*MapReduce*
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.mapreduce1
hadoop fs -rm -r /terasort/TS_input2
yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
teragen  -D dfs.replication=1 -Dmapred.map.tasks=144    10000
/terasort/TS_input2
I hope this helps.
Thanks,

Gunnar
On Thu, Jan 2, 2014 at 3:03 PM, Dhanasekaran Anbalagan
<[EMAIL PROTECTED]>wrote:

> Hi Guys,
>
> we recently installed CDH5 in our test cluster. we need any test job for
> YARN framework.
>
> we are able to run mapreduce job successfully on YARN framework without
> code change.
>
> But we need to test yarn job functionality. can you please guide me.
>
> we tired https://github.com/hortonworks/simple-yarn-app it's not help us.
> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
> com.hortonworks.simpleyarnapp.Client /bin/date 2
> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
> dvcloudlab231/192.168.70.231:8032
> Submitting application application_1388687890867_0007
> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
> 192.168.70.231:8032
> Application application_1388687890867_0007 finished with state FAILED at
> 1388699348344
>
> Note:
> In Resource manager node. I don't see any container logs.
>
> -Dhanasekaran.
>
> Did I learn something today? If not, I wasted it.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+[EMAIL PROTECTED].
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>

--
Thanks,

Gunnar
*If you think you can you can, if you think you can't you're right.*