Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> different input/output formats


Copy link to this message
-
Re: different input/output formats
PFA.

On Wed, May 30, 2012 at 2:45 AM, Mark question <[EMAIL PROTECTED]> wrote:

> Hi Samir, can you email me your main class.. or if you can check mine, it
> is as follows:
>
> public class SortByNorm1 extends Configured implements Tool {
>
>    @Override public int run(String[] args) throws Exception {
>
>        if (args.length != 2) {
>            System.err.printf("Usage:bin/hadoop jar norm1.jar <inputDir>
> <outputDir>\n");
>            ToolRunner.printGenericCommandUsage(System.err);
>            return -1;
>        }
>        JobConf conf = new JobConf(new Configuration(),SortByNorm1.class);
>        conf.setJobName("SortDocByNorm1");
>        conf.setMapperClass(Norm1Mapper.class);
>        conf.setMapOutputKeyClass(FloatWritable.class);
>        conf.setMapOutputValueClass(Text.class);
>        conf.setNumReduceTasks(0);
>        conf.setReducerClass(Norm1Reducer.class);
>         conf.setOutputKeyClass(FloatWritable.class);
>        conf.setOutputValueClass(Text.class);
>
>        conf.setInputFormat(TextInputFormat.class);
>        conf.setOutputFormat(SequenceFileOutputFormat.class);
>
>        TextInputFormat.addInputPath(conf, new Path(args[0]));
>        SequenceFileOutputFormat.setOutputPath(conf, new Path(args[1]));
>         JobClient.runJob(conf);
>        return 0;
>    }
>    public static void main(String[] args) throws Exception {
>        int exitCode = ToolRunner.run(new SortByNorm1(), args);
>        System.exit(exitCode);
>     }
>
>
> On Tue, May 29, 2012 at 1:55 PM, samir das mohapatra <
> [EMAIL PROTECTED]> wrote:
>
> > Hi Mark
> >    See the out put for that same  Application .
> >    I am  not getting any error.
> >
> >
> > On Wed, May 30, 2012 at 1:27 AM, Mark question <[EMAIL PROTECTED]
> >wrote:
> >
> >> Hi guys, this is a very simple  program, trying to use TextInputFormat
> and
> >> SequenceFileoutputFormat. Should be easy but I get the same error.
> >>
> >> Here is my configurations:
> >>
> >>        conf.setMapperClass(myMapper.class);
> >>        conf.setMapOutputKeyClass(FloatWritable.class);
> >>        conf.setMapOutputValueClass(Text.class);
> >>        conf.setNumReduceTasks(0);
> >>        conf.setOutputKeyClass(FloatWritable.class);
> >>        conf.setOutputValueClass(Text.class);
> >>
> >>        conf.setInputFormat(TextInputFormat.class);
> >>        conf.setOutputFormat(SequenceFileOutputFormat.class);
> >>
> >>        TextInputFormat.addInputPath(conf, new Path(args[0]));
> >>        SequenceFileOutputFormat.setOutputPath(conf, new Path(args[1]));
> >>
> >>
> >> myMapper class is:
> >>
> >> public class myMapper extends MapReduceBase implements
> >> Mapper<LongWritable,Text,FloatWritable,Text> {
> >>
> >>    public void map(LongWritable offset, Text
> >> val,OutputCollector<FloatWritable,Text> output, Reporter reporter)
> >>    throws IOException {
> >>        output.collect(new FloatWritable(1), val);
> >>     }
> >> }
> >>
> >> But I get the following error:
> >>
> >> 12/05/29 12:54:31 INFO mapreduce.Job: Task Id :
> >> attempt_201205260045_0032_m_000000_0, Status : FAILED
> >> java.io.IOException: wrong key class: org.apache.hadoop.io.LongWritable
> is
> >> not class org.apache.hadoop.io.FloatWritable
> >>    at
> >> org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:998)
> >>    at
> >>
> >>
> org.apache.hadoop.mapred.SequenceFileOutputFormat$1.write(SequenceFileOutputFormat.java:75)
> >>    at
> >>
> >>
> org.apache.hadoop.mapred.MapTask$DirectMapOutputCollector.collect(MapTask.java:705)
> >>    at
> >>
> >>
> org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:508)
> >>    at
> >>
> >>
> filter.stat.cosine.preprocess.SortByNorm1$Norm1Mapper.map(SortByNorm1.java:59)
> >>    at
> >>
> >>
> filter.stat.cosine.preprocess.SortByNorm1$Norm1Mapper.map(SortByNorm1.java:1)
> >>    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> >>    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:397)
> >>    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)