Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re: setLocalResources() on ContainerLaunchContext


Copy link to this message
-
Re: setLocalResources() on ContainerLaunchContext
I tried the following and it works!
String shellScriptPath = "file:///home_/dsadm/kishore/kk.ksh";

But now getting a timestamp error like below, when I passed 0 to
setTimestamp()

13/08/06 08:23:48 INFO ApplicationMaster: Got container status for
containerID= container_1375784329048_0017_01_000002, state=COMPLETE,
exitStatus=-1000, diagnostics=Resource file:/home_/dsadm/kishore/kk.ksh
changed on src filesystem (expected 0, was 1367580580000
On Tue, Aug 6, 2013 at 5:24 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> Can you try passing a fully qualified local path? That is, including the
> file:/ scheme
> On Aug 6, 2013 4:05 PM, "Krishna Kishore Bonagiri" <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Harsh,
>>    The setResource() call on LocalResource() is expecting an argument of
>> type org.apache.hadoop.yarn.api.records.URL which is converted from a
>> string in the form of URI. This happens in the following call of
>> Distributed Shell example,
>>
>> shellRsrc.setResource(ConverterUtils.getYarnUrlFromURI(new URI(
>> shellScriptPath)));
>>
>> So, if I give a local file I get a parsing error like below, which is
>> when I changed it to an HDFS file thinking that it should be given like
>> that only. Could you please give an example of how else it could be used,
>> using a local file as you are saying?
>>
>> 2013-08-06 06:23:12,942 WARN
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
>> Failed to parse resource-request
>> java.net.URISyntaxException: Expected scheme name at index 0:
>> :///home_/dsadm/kishore/kk.ksh
>>         at java.net.URI$Parser.fail(URI.java:2820)
>>         at java.net.URI$Parser.failExpecting(URI.java:2826)
>>         at java.net.URI$Parser.parse(URI.java:3015)
>>         at java.net.URI.<init>(URI.java:747)
>>         at
>> org.apache.hadoop.yarn.util.ConverterUtils.getPathFromYarnURL(ConverterUtils.java:77)
>>         at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourceRequest.<init>(LocalResourceRequest.java:46)
>>
>>
>>
>> On Tue, Aug 6, 2013 at 3:36 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>>> To be honest, I've never tried loading a HDFS file onto the
>>> LocalResource this way. I usually just pass a local file and that
>>> works just fine. There may be something in the URI transformation
>>> possibly breaking a HDFS source, but try passing a local file - does
>>> that fail too? The Shell example uses a local file.
>>>
>>> On Tue, Aug 6, 2013 at 10:54 AM, Krishna Kishore Bonagiri
>>> <[EMAIL PROTECTED]> wrote:
>>> > Hi Harsh,
>>> >
>>> >   Please see if this is useful, I got a stack trace after the error has
>>> > occurred....
>>> >
>>> > 2013-08-06 00:55:30,559 INFO
>>> > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
>>> CWD set
>>> > to
>>> /tmp/nm-local-dir/usercache/dsadm/appcache/application_1375716148174_0004
>>> > >>> >
>>> file:/tmp/nm-local-dir/usercache/dsadm/appcache/application_1375716148174_0004
>>> > 2013-08-06 00:55:31,017 ERROR
>>> > org.apache.hadoop.security.UserGroupInformation:
>>> PriviledgedActionException
>>> > as:dsadm (auth:SIMPLE) cause:java.io.FileNotFoundException: File does
>>> not
>>> > exist: hdfs://isredeng/kishore/kk.ksh
>>> > 2013-08-06 00:55:31,029 INFO
>>> >
>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
>>> > DEBUG: FAILED { hdfs://isredeng/kishore/kk.ksh, 0, FILE, null }, File
>>> does
>>> > not exist: hdfs://isredeng/kishore/kk.ksh
>>> > 2013-08-06 00:55:31,031 INFO
>>> >
>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource:
>>> > Resource hdfs://isredeng/kishore/kk.ksh transitioned from DOWNLOADING
>>> to
>>> > FAILED
>>> > 2013-08-06 00:55:31,034 INFO
>>> >
>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
>>> > Container container_1375716148174_0004_01_000002 transitioned from
>>> > LOCALIZING to LOCALIZATION_FAILED
>>> > 2013-08-06 00:55:31,035 INFO
+
Omkar Joshi 2013-08-06, 18:11
+
Hitesh Shah 2013-08-06, 19:05
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB