Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # dev - PriviledgedActionException as:root (auth:SIMPLE) cause:java.io.IOException: No input paths specified in job


+
ch huang 2013-12-24, 01:27
Copy link to this message
-
Re: PriviledgedActionException as:root (auth:SIMPLE) cause:java.io.IOException: No input paths specified in job
ch huang 2013-12-24, 01:43
I changed the following code in my test code ,it's ok
 FileInputFormat. *addInputPath*(job, *new* Path(args[0])); =>

MyFileInputFormat. *addInputPath*(job, *new* Path(args[0]));
On Tue, Dec 24, 2013 at 9:27 AM, ch huang <[EMAIL PROTECTED]> wrote:

> hi,maillist:
>       i try to understand how the FileInputFormat work,and i do the
> following things,but it seems not work,and i do not know why,hope anyone
> can shed light on it
>
>      i change the wordcount code add one line on run function , and
> MyTextInputFormat just a copy of TextInputFormat
>
> job.setInputFormatClass(MyTextInputFormat.class);
>
> and i also modify the MyTextInputFormat.java , and MyFileInputFormat just
> another copy from FileInputFormat,so i can modify it freely and observe
>
> public class MyTextInputFormat extends FileInputFormat<LongWritable,
> Text>   =>  public class MyTextInputFormat extends
> MyFileInputFormat<LongWritable, Text>
>
> finally i compile and run the program ,but get error like this, i do not
> know why
>
> # hadoop yarndemo/BmAlphaToNum /alex/messages  /alex/output8
> 13/12/24 09:10:15 WARN conf.Configuration: session.id is deprecated.
> Instead, use dfs.metrics.session-id
> 13/12/24 09:10:15 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId> 13/12/24 09:10:15 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 13/12/24 09:10:15 WARN mapred.JobClient: No job jar file set.  User
> classes may not be found. See JobConf(Class) or JobConf#setJar(String).
> 13/12/24 09:10:15 INFO yarndemo.MyFileInputFormat: in getsplits
> 13/12/24 09:10:15 INFO yarndemo.MyFileInputFormat: jumpppppppppp into
> liststatus!!
> 13/12/24 09:10:15 INFO mapred.JobClient: Cleaning up the staging area
> file:/data/temp/mapred/staging/root866865048/.staging/job_local866865048_0001
> 13/12/24 09:10:15 ERROR security.UserGroupInformation:
> PriviledgedActionException as:root (auth:SIMPLE) cause:java.io.IOException:
> No input paths specified in job
> Exception in thread "main" java.io.IOException: No input paths specified
> in job
>         at
> yarndemo.MyFileInputFormat.listStatus(MyFileInputFormat.java:214)
>         at yarndemo.MyFileInputFormat.getSplits(MyFileInputFormat.java:284)
>         at
> org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1063)
>         at
> org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1080)
>         at
> org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:992)
>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:945)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>         at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:945)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
>         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:596)
>         at yarndemo.BmAlphaToNum.run(BmAlphaToNum.java:74)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>         at yarndemo.BmAlphaToNum.main(BmAlphaToNum.java:78)
>
> here is my java file content
>
> package yarndemo;
> import java.io.IOException;
> import java.util.StringTokenizer;
> import org.apache.hadoop.conf.Configuration;
> import org.apache.hadoop.conf.Configured;
> import org.apache.hadoop.fs.Path;
> import org.apache.hadoop.io.IntWritable;
> import org.apache.hadoop.io.Text;
> import org.apache.hadoop.mapreduce.Job;
> import org.apache.hadoop.mapreduce.Mapper;
> import org.apache.hadoop.mapreduce.Reducer;
> import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
> import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
> import org.apache.hadoop.util.Tool;
> import org.apache.hadoop.util.ToolRunner;