Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Yarn Writing ApplicationMaster, Client Exception "URISyntaxException: Expected scheme name at index 0"


Copy link to this message
-
Re: Yarn Writing ApplicationMaster, Client Exception "URISyntaxException: Expected scheme name at index 0"
Problem solved. Thank you.

2013/3/26 Harsh J <[EMAIL PROTECTED]>

> YARN does not seem to be checking for a fully qualified path when you
> pass it yours and ends up breaking. The problem is easily reproducible
> with the two transforming calls from ConverterUtils.
>
> Transform the jarPath to a fully qualified one like so, before using
> it anywhere:
>
> Path jarPath = new Path("/Working_HDFS_DIR/"+ appId +"/"+AM_JAR);
> jarPath = fs.makeQualified(jarPath);
>
> And the problem should disappear.
>
>
> On Mon, Mar 25, 2013 at 10:57 PM, blah blah <[EMAIL PROTECTED]> wrote:
> > Hi
> >
> > I am trying to create my own Application Master. I have followed this
> > tutorial
> >
> http://hadoop.apache.org/docs/r2.0.2-alpha/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html
> > However I have problem with reading AM jar as resource at the container.
> > Code for passing AM.jar
> >
> > Path jarPath = new Path("/Working_HDFS_DIR/"+ appId +"/"+AM_JAR);
> > fs.copyFromLocalFile(new Path("/local/src/AM.jar"), jarPath); //
> VALIDATED
> > jar is in HDFS under correct PATH
> > FileStatus jarStatus = fs.getFileStatus(jarPath);
> > LocalResource amJarRsrc = Records.newRecord(LocalResource.class);
> > amJarRsrc.setType(LocalResourceType.FILE);
> > amJarRsrc.setVisibility(LocalResourceVisibility.APPLICATION);
> > amJarRsrc.setResource(ConverterUtils.getYarnUrlFromPath(jarPath));
> > amJarRsrc.setTimestamp(jarStatus.getModificationTime());
> > amJarRsrc.setSize(jarStatus.getLen());
> > localResources.put("AppMaster.jar",  amJarRsrc);
> > amContainer.setLocalResources(localResources);
> >
> > Error logs (nodeManager.log)
> >
> > INFO
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application:
> > Application application_1364219323374_0016 transitioned from INITING to
> > RUNNING
> > INFO
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
> > Got exception parsing AppMaster.jar and value resource {, port: -1, file:
> > "/Working_HDFS_DIR/application_1364219323374_0016/AM.jar", }, size:
> 13940,
> > timestamp: 1364230436600, type: FILE, visibility: APPLICATION,
> > 2013-03-25 17:53:57,391 WARN
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
> > Failed to parse resource-request
> > java.net.URISyntaxException: Expected scheme name at index 0:
> > :///Working_HDFS_DIR/application_1364219323374_0016/AM.jar
> > at java.net.URI$Parser.fail(URI.java:2810)
> > at java.net.URI$Parser.failExpecting(URI.java:2816)
> > at java.net.URI$Parser.parse(URI.java:3008)
> > at java.net.URI.<init>(URI.java:735)
> > at
> >
> org.apache.hadoop.yarn.util.ConverterUtils.getPathFromYarnURL(ConverterUtils.java:70)
> > at
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourceRequest.<init>(LocalResourceRequest.java:46)
> > at
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:501)
> > at
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:472)
> > at
> >
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:382)
> > at
> >
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
> > at
> >
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
> > at
> >
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
> > at
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:828)
> > at
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:71)
> > at
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:556)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB