Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Re: Help writing a YARN application


+
Rahul Bhattacharjee 2013-09-21, 04:08
+
Arun C Murthy 2013-09-23, 23:46
+
Pradeep Gollakota 2013-09-24, 05:19
Copy link to this message
-
Re: Help writing a YARN application
Personal 2013-09-24, 05:39
Hi Pradeep,
You can also look at Weave - A simpler thread abstraction over YARN. 
http://github.com/continuuity/weave
Runs on multiple HDs.
Nitin

On Mon, Sep 23, 2013 at 10:20 PM, Pradeep Gollakota <[EMAIL PROTECTED]>
wrote:

> Hi Arun,
> Thanks so much for your reply. I looked at the app you linked, it's way
> easier to understand. Unfortunately, I can't use YarnClient class. It
> doesn't seem to be in the 2.0.0 branch. Our production environment is using
> cdh-4.x and it looks like they're tracking the 2.0.x branch.
> However, I have looked at other implementations (notably Kitten and Giraph)
> for inspiration and able to get my containers successfully launching.
> Thanks for the help!
> - Pradeep
> On Mon, Sep 23, 2013 at 4:46 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>> I looked at your code, it's using really old apis/protocols which are
>> significantly different now. See hadoop-2.1.0-beta release for latest
>> apis/protocols.
>>
>> Also, you should really be using yarn client module rather than raw
>> protocols.
>>
>> See https://github.com/hortonworks/simple-yarn-app for a simple example.
>>
>> thanks,
>> Arun
>>
>> On Sep 20, 2013, at 11:24 AM, Pradeep Gollakota <[EMAIL PROTECTED]>
>> wrote:
>>
>> Hi All,
>>
>> I've been trying to write a Yarn application and I'm completely lost. I'm
>> using Hadoop 2.0.0-cdh4.4.0 (Cloudera distribution). I've uploaded my
>> sample code to github at https://github.com/pradeepg26/sample-yarn
>>
>> The problem is that my application master is exiting with a status of 1
>> (I'm expecting that since my code isn't complete yet). But I have no logs
>> that I can examine. So I'm not sure if the error I'm getting is the error
>> I'm expecting. I've attached the nodemanger and resourcemanager logs for
>> your reference as well.
>>
>> How can I get started on writing YARN applications beyond the initial
>> tutorial?
>>
>> Thanks for any help/pointers!
>>
>> Pradeep
>> <yarn-yarn-nodemanager-pradeep-gollakota.vm.lithium.com.log>
>> <yarn-yarn-resourcemanager-pradeep-gollakota.vm.lithium.com.log>
>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.