Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> different input/output formats


Copy link to this message
-
Re: different input/output formats
PFA.

On Wed, May 30, 2012 at 2:45 AM, Mark question <[EMAIL PROTECTED]> wrote:

> Hi Samir, can you email me your main class.. or if you can check mine, it
> is as follows:
>
> public class SortByNorm1 extends Configured implements Tool {
>
>    @Override public int run(String[] args) throws Exception {
>
>        if (args.length != 2) {
>            System.err.printf("Usage:bin/hadoop jar norm1.jar <inputDir>
> <outputDir>\n");
>            ToolRunner.printGenericCommandUsage(System.err);
>            return -1;
>        }
>        JobConf conf = new JobConf(new Configuration(),SortByNorm1.class);
>        conf.setJobName("SortDocByNorm1");
>        conf.setMapperClass(Norm1Mapper.class);
>        conf.setMapOutputKeyClass(FloatWritable.class);
>        conf.setMapOutputValueClass(Text.class);
>        conf.setNumReduceTasks(0);
>        conf.setReducerClass(Norm1Reducer.class);
>         conf.setOutputKeyClass(FloatWritable.class);
>        conf.setOutputValueClass(Text.class);
>
>        conf.setInputFormat(TextInputFormat.class);
>        conf.setOutputFormat(SequenceFileOutputFormat.class);
>
>        TextInputFormat.addInputPath(conf, new Path(args[0]));
>        SequenceFileOutputFormat.setOutputPath(conf, new Path(args[1]));
>         JobClient.runJob(conf);
>        return 0;
>    }
>    public static void main(String[] args) throws Exception {
>        int exitCode = ToolRunner.run(new SortByNorm1(), args);
>        System.exit(exitCode);
>     }
>
>
> On Tue, May 29, 2012 at 1:55 PM, samir das mohapatra <
> [EMAIL PROTECTED]> wrote:
>
> > Hi Mark
> >    See the out put for that same  Application .
> >    I am  not getting any error.
> >
> >
> > On Wed, May 30, 2012 at 1:27 AM, Mark question <[EMAIL PROTECTED]
> >wrote:
> >
> >> Hi guys, this is a very simple  program, trying to use TextInputFormat
> and
> >> SequenceFileoutputFormat. Should be easy but I get the same error.
> >>
> >> Here is my configurations:
> >>
> >>        conf.setMapperClass(myMapper.class);
> >>        conf.setMapOutputKeyClass(FloatWritable.class);
> >>        conf.setMapOutputValueClass(Text.class);
> >>        conf.setNumReduceTasks(0);
> >>        conf.setOutputKeyClass(FloatWritable.class);
> >>        conf.setOutputValueClass(Text.class);
> >>
> >>        conf.setInputFormat(TextInputFormat.class);
> >>        conf.setOutputFormat(SequenceFileOutputFormat.class);
> >>
> >>        TextInputFormat.addInputPath(conf, new Path(args[0]));
> >>        SequenceFileOutputFormat.setOutputPath(conf, new Path(args[1]));
> >>
> >>
> >> myMapper class is:
> >>
> >> public class myMapper extends MapReduceBase implements
> >> Mapper<LongWritable,Text,FloatWritable,Text> {
> >>
> >>    public void map(LongWritable offset, Text
> >> val,OutputCollector<FloatWritable,Text> output, Reporter reporter)
> >>    throws IOException {
> >>        output.collect(new FloatWritable(1), val);
> >>     }
> >> }
> >>
> >> But I get the following error:
> >>
> >> 12/05/29 12:54:31 INFO mapreduce.Job: Task Id :
> >> attempt_201205260045_0032_m_000000_0, Status : FAILED
> >> java.io.IOException: wrong key class: org.apache.hadoop.io.LongWritable
> is
> >> not class org.apache.hadoop.io.FloatWritable
> >>    at
> >> org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:998)
> >>    at
> >>
> >>
> org.apache.hadoop.mapred.SequenceFileOutputFormat$1.write(SequenceFileOutputFormat.java:75)
> >>    at
> >>
> >>
> org.apache.hadoop.mapred.MapTask$DirectMapOutputCollector.collect(MapTask.java:705)
> >>    at
> >>
> >>
> org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:508)
> >>    at
> >>
> >>
> filter.stat.cosine.preprocess.SortByNorm1$Norm1Mapper.map(SortByNorm1.java:59)
> >>    at
> >>
> >>
> filter.stat.cosine.preprocess.SortByNorm1$Norm1Mapper.map(SortByNorm1.java:1)
> >>    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> >>    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:397)
> >>    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB