Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: FileNotFoundException When DistributedCache file with YARN


Copy link to this message
-
Re: FileNotFoundException When DistributedCache file with YARN
Hi

  I fixed the problem just add the expression of job.setJar("MyJarName")
,and the job went well.
  But I have no idea about the expression  and  the exception.

Any suggestion will be appreciated.
regards.
2013/5/13 YouPeng Yang <[EMAIL PROTECTED]>

> HI
>
>  I adopt distributed cache  to implement the semi-joins.
>  I am using CDH4.1.2
>
>  the Map side setup  function is as [1]:
>
>
> It works well in my eclipse indigo, howevet it goes wrong when I run it
>  in cli:
> The  exeception in one of the containers refers to[2].
>
> How could I solve this exception?
>
> Regards
>
>
> [1]-------------------------------------------
> @Override
> public void setup(Context context){
>  try {
> //add DistributedCache files to the Mapper.
> //this DistributedCache files are on the HDFS
>  URI[] cacheFiles > DistributedCache.getCacheFiles(context.getConfiguration());
> if (cacheFiles != null && cacheFiles.length > 0){
>  Path path = new Path(cacheFiles[0].toString());
> fs = FileSystem.get(path.toUri(),context.getConfiguration());
>  in = fs.open(path);
>     String line ;
> String[] tokens ;
>     while ((line = in.readLine()) != null ) {
> tokens = StringUtils.split(line,'|');
> if (tokens != null && tokens.length >= 3){
>  joinData.put(tokens[0], tokens[1]);//key:ifid ;value:cmtsid
> }
> }
>  }
> } catch (IOException e) {
> e.printStackTrace();
>  }finally{
> try {
> in.close();
> } catch (IOException e) {
>  e.printStackTrace();
> }
> }
> }
>
>
> [2]----------------------------
> 2013-05-13 16:11:08,444 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
> Downloading public rsrc:{
> hdfs://Hadoop01:8040/user/hadoop/sqoop/CMTSIFTABLE/part-m-00000,
> 1366969672048, FILE }
> 2013-05-13 16:11:08,446 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
> Created localizer for container_1366976077492_0022_01_000001
> 2013-05-13 16:11:08,446 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
> Failed to download rsrc { {
> hdfs://Hadoop01:8040/user/hadoop/sqoop/CMTSIFTABLE/part-m-00000,
> 1366969672048, FILE
> },pending,[(container_1366976077492_0022_01_000001)],9096622200855127,DOWNLOADING}
> java.io.FileNotFoundException: File /tmp/nm-local-dir/filecache does not
> exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:455)
>  at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:919)
> at
> org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:143)
>  at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:189)
> at org.apache.hadoop.fs.FileContext$4.next(FileContext.java:706)
>  at org.apache.hadoop.fs.FileContext$4.next(FileContext.java:703)
> at
> org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2333)
>  at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:703)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:147)
>  at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:49)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>  at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>  at java.lang.Thread.run(Thread.java:662)
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB