Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Re: Yarn Writing ApplicationMaster, Client Exception "URISyntaxException: Expected scheme name at index 0"


Copy link to this message
-
Re: Yarn Writing ApplicationMaster, Client Exception "URISyntaxException: Expected scheme name at index 0"
blah blah 2013-03-26, 09:07
Problem solved. Thank you.

2013/3/26 Harsh J <[EMAIL PROTECTED]>

> YARN does not seem to be checking for a fully qualified path when you
> pass it yours and ends up breaking. The problem is easily reproducible
> with the two transforming calls from ConverterUtils.
>
> Transform the jarPath to a fully qualified one like so, before using
> it anywhere:
>
> Path jarPath = new Path("/Working_HDFS_DIR/"+ appId +"/"+AM_JAR);
> jarPath = fs.makeQualified(jarPath);
>
> And the problem should disappear.
>
>
> On Mon, Mar 25, 2013 at 10:57 PM, blah blah <[EMAIL PROTECTED]> wrote:
> > Hi
> >
> > I am trying to create my own Application Master. I have followed this
> > tutorial
> >
> http://hadoop.apache.org/docs/r2.0.2-alpha/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html
> > However I have problem with reading AM jar as resource at the container.
> > Code for passing AM.jar
> >
> > Path jarPath = new Path("/Working_HDFS_DIR/"+ appId +"/"+AM_JAR);
> > fs.copyFromLocalFile(new Path("/local/src/AM.jar"), jarPath); //
> VALIDATED
> > jar is in HDFS under correct PATH
> > FileStatus jarStatus = fs.getFileStatus(jarPath);
> > LocalResource amJarRsrc = Records.newRecord(LocalResource.class);
> > amJarRsrc.setType(LocalResourceType.FILE);
> > amJarRsrc.setVisibility(LocalResourceVisibility.APPLICATION);
> > amJarRsrc.setResource(ConverterUtils.getYarnUrlFromPath(jarPath));
> > amJarRsrc.setTimestamp(jarStatus.getModificationTime());
> > amJarRsrc.setSize(jarStatus.getLen());
> > localResources.put("AppMaster.jar",  amJarRsrc);
> > amContainer.setLocalResources(localResources);
> >
> > Error logs (nodeManager.log)
> >
> > INFO
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application:
> > Application application_1364219323374_0016 transitioned from INITING to
> > RUNNING
> > INFO
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
> > Got exception parsing AppMaster.jar and value resource {, port: -1, file:
> > "/Working_HDFS_DIR/application_1364219323374_0016/AM.jar", }, size:
> 13940,
> > timestamp: 1364230436600, type: FILE, visibility: APPLICATION,
> > 2013-03-25 17:53:57,391 WARN
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
> > Failed to parse resource-request
> > java.net.URISyntaxException: Expected scheme name at index 0:
> > :///Working_HDFS_DIR/application_1364219323374_0016/AM.jar
> > at java.net.URI$Parser.fail(URI.java:2810)
> > at java.net.URI$Parser.failExpecting(URI.java:2816)
> > at java.net.URI$Parser.parse(URI.java:3008)
> > at java.net.URI.<init>(URI.java:735)
> > at
> >
> org.apache.hadoop.yarn.util.ConverterUtils.getPathFromYarnURL(ConverterUtils.java:70)
> > at
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourceRequest.<init>(LocalResourceRequest.java:46)
> > at
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:501)
> > at
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:472)
> > at
> >
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:382)
> > at
> >
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> > at
> >
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
> > at
> >
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> > at
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:828)
> > at
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:71)
> > at
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:556)