Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Correct way to unzip locally an archive in Yarn


Copy link to this message
-
Re: Correct way to unzip locally an archive in Yarn
Yes if you can trace this down I would be very interested.  We are running 0.23.6 without any issues, but that does not mean that there is not some bug in the code that is causing this to happen in your situation.

--Bobby

From: Sebastiano Vigna <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Saturday, February 16, 2013 8:39 AM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: Correct way to unzip locally an archive in Yarn

I will as soon as I can understand what happens on the cluster (no access from home). DistributedCache.getLocalCacheFiles() returns in both cases a local name for the zip file uploaded with -files, but locally my unzip code works, on the cluster it throws a FileNotFoundException.
On 16 February 2013 15:22, Arun C Murthy <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
This could be a bug, mind opening a jira? Thanks.

On Feb 16, 2013, at 2:34 AM, Sebastiano Vigna wrote:

On 15 February 2013 16:57, Robert Evans <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Are you trying to run a Map/Reduce job or are you writing a new YARN
application?  If it is a MR job, then it should work mostly the same as
before (on 1.x). If you are writing a new YARN application then there is a
separate Map in the ContainerLaunchContext that you need to fill in.

It's a MapReduce job (0.23.6). After two days of useless trials, I'm uploading the zip with -files and I wrote a stub to unzip it manually. I was positively unable to get the archive unzipped *to a local directory* in any way.

Unfortunately it works in local but not on the cluster. I have still to discover why. :(

Ciao,
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB