Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: Help writing a YARN application


+
Rahul Bhattacharjee 2013-09-21, 04:08
+
Arun C Murthy 2013-09-23, 23:46
+
Pradeep Gollakota 2013-09-24, 05:19
Copy link to this message
-
Re: Help writing a YARN application
Hi Pradeep,
You can also look at Weave - A simpler thread abstraction over YARN. 
http://github.com/continuuity/weave
Runs on multiple HDs.
Nitin

On Mon, Sep 23, 2013 at 10:20 PM, Pradeep Gollakota <[EMAIL PROTECTED]>
wrote:

> Hi Arun,
> Thanks so much for your reply. I looked at the app you linked, it's way
> easier to understand. Unfortunately, I can't use YarnClient class. It
> doesn't seem to be in the 2.0.0 branch. Our production environment is using
> cdh-4.x and it looks like they're tracking the 2.0.x branch.
> However, I have looked at other implementations (notably Kitten and Giraph)
> for inspiration and able to get my containers successfully launching.
> Thanks for the help!
> - Pradeep
> On Mon, Sep 23, 2013 at 4:46 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>> I looked at your code, it's using really old apis/protocols which are
>> significantly different now. See hadoop-2.1.0-beta release for latest
>> apis/protocols.
>>
>> Also, you should really be using yarn client module rather than raw
>> protocols.
>>
>> See https://github.com/hortonworks/simple-yarn-app for a simple example.
>>
>> thanks,
>> Arun
>>
>> On Sep 20, 2013, at 11:24 AM, Pradeep Gollakota <[EMAIL PROTECTED]>
>> wrote:
>>
>> Hi All,
>>
>> I've been trying to write a Yarn application and I'm completely lost. I'm
>> using Hadoop 2.0.0-cdh4.4.0 (Cloudera distribution). I've uploaded my
>> sample code to github at https://github.com/pradeepg26/sample-yarn
>>
>> The problem is that my application master is exiting with a status of 1
>> (I'm expecting that since my code isn't complete yet). But I have no logs
>> that I can examine. So I'm not sure if the error I'm getting is the error
>> I'm expecting. I've attached the nodemanger and resourcemanager logs for
>> your reference as well.
>>
>> How can I get started on writing YARN applications beyond the initial
>> tutorial?
>>
>> Thanks for any help/pointers!
>>
>> Pradeep
>> <yarn-yarn-nodemanager-pradeep-gollakota.vm.lithium.com.log>
>> <yarn-yarn-resourcemanager-pradeep-gollakota.vm.lithium.com.log>
>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB