Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Re: Need YARN - Test job.


+
Gunnar Tapper 2014-01-02, 22:27
+
Gunnar Tapper 2014-01-03, 15:08
Copy link to this message
-
Re: Need YARN - Test job.
Alejandro Abdelnur 2014-01-03, 17:44
Gaurav,

[BCCing user@h.a.o to move the thread out of it]

It seems you  are missing some step when reconfiguring your cluster in
Cloudera Manager, you should have to modify things by hand in your setup.
Adding the Cloudera Manager user alias.

Thanks.
On Fri, Jan 3, 2014 at 7:11 AM, Gaurav Shankhdhar <
[EMAIL PROTECTED]> wrote:

> yes, i am setting it but still still hangs there..
> Also there are no failure logs, it just hangs without erroring out.
>
> Any log location you want me to look at?
>
>
> On Friday, January 3, 2014 8:38:16 PM UTC+5:30, Gunnar Tapper wrote:
>
>> Hi Guarav,
>>
>> What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you
>> don't set it as:
>>
>>  export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
>>
>> Gunnar
>>
>>
>> On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <[EMAIL PROTECTED]>wrote:
>>
>>> Folks,
>>>
>>> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
>>> no luck.
>>> What i have figured out till now:
>>>
>>> 1. By default Demo VM run the MR in MRv1 uses JobTracker and
>>> TaskTracker.
>>> 2. You can either run MRv1 or MRv2 but not both.
>>> 3. I disabled MRv1 and configured YARN by increasing the alternatives
>>> priority for YARN, deployed the Client Configuration (Using Cloudera
>>> Manager).
>>> 4. Submitted the teragen program and can see the YARN framework is in
>>> action from Web UI's.
>>> 5. But it hangs at Map 0% and Reduce %.
>>>
>>> Tried multiple option to no avail.
>>>
>>> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in
>>> CDH4.5?
>>>
>>> Attached doc with screen shots.
>>>
>>> Reference:
>>> http://www.cloudera.com/content/cloudera-content/cloudera-
>>> docs/CM4Ent/latest/Cloudera-Manager-Managing-Clusters/cmmc_adding_YARN_
>>> MRv2.html
>>>
>>> Regards
>>> Gaurav
>>>
>>> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
>>> wrote:
>>>>
>>>> Hi Guys,
>>>>
>>>> we recently installed CDH5 in our test cluster. we need any test job
>>>> for YARN framework.
>>>>
>>>> we are able to run mapreduce job successfully on YARN framework without
>>>> code change.
>>>>
>>>> But we need to test yarn job functionality. can you please guide me.
>>>>
>>>> we tired https://github.com/hortonworks/simple-yarn-app it's not help
>>>> us.
>>>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>>>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>>>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>>>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>>>> dvcloudlab231/192.168.70.231:8032
>>>> Submitting application application_1388687890867_0007
>>>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>>>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>>>> 192.168.70.231:8032
>>>> Application application_1388687890867_0007 finished with state FAILED
>>>> at 1388699348344
>>>>
>>>> Note:
>>>> In Resource manager node. I don't see any container logs.
>>>>
>>>> -Dhanasekaran.
>>>>
>>>> Did I learn something today? If not, I wasted it.
>>>>
>>>  --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "CDH Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to cdh-user+[EMAIL PROTECTED].
>>>
>>> For more options, visit https://groups.google.com/a/
>>> cloudera.org/groups/opt_out.
>>>
>>
>>
>>
>> --
>> Thanks,
>>
>> Gunnar
>> *If you think you can you can, if you think you can't you're right.*
>>
>  --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+[EMAIL PROTECTED].
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>

--
Alejandro