Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - HBase MR - key/value mismatch


Copy link to this message
-
Re: HBase MR - key/value mismatch
Shahab Yunus 2013-09-05, 16:46
Ted,

Might be a something very basic that I am missing but why should OP's
reducer's key be of type ImmutableBytesWritable if he is emitting Text in
the mapper? Thanks.

 protected void map(

            ImmutableBytesWritable key,

            Result value,

            org.apache.hadoop.mapreduce.Mapper<ImmutableBytesWritable,
Result, Text, Text>.Context context)

            throws IOException, InterruptedException {

context.write(this.sentenseOriginal, this.sentenseParsed); //sentenseOriginal
is Text
Regards,
Shahab
On Thu, Sep 5, 2013 at 10:34 AM, Ted Yu <[EMAIL PROTECTED]> wrote:

> public class SentimentCalculationHBaseReducer extends
>
>         TableReducer<Text, Text, ImmutableBytesWritable> {
>
> The first type parameter for reducer should be ImmutableBytesWritable
>
> Cheers
>
>
> On Wed, Sep 4, 2013 at 11:16 PM, Omkar Joshi <[EMAIL PROTECTED]
> >wrote:
>
> > I'm trying to execute a MR code over stand-alone HBase(0.94.11). I had
> > read the HBase api and modified my MR code to read data and getting
> > exceptions in the Reduce phase.
> >
> > The exception I get is :
> >
> > 13/09/05 16:16:17 INFO mapred.JobClient:  map 0% reduce 0%
> >
> > 13/09/05 16:23:31 INFO mapred.JobClient: Task Id :
> > attempt_201309051437_0005_m_000000_0, Status : FAILED
> >
> > java.io.IOException: wrong key class: class
> > org.apache.hadoop.hbase.io.ImmutableBytesWritable is not class
> > org.apache.hadoop.io.Text
> >
> >         at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:164)
> >
> >         at
> >
> org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1168)
> >
> >         at
> >
> org.apache.hadoop.mapred.Task$NewCombinerRunner$OutputConverter.write(Task.java:1492)
> >
> >         at
> >
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> >
> >         at
> >
> com.hbase.mapreduce.SentimentCalculationHBaseReducer.reduce(SentimentCalculationHBaseReducer.java:199)
> >
> >         at
> >
> com.hbase.mapreduce.SentimentCalculationHBaseReducer.reduce(SentimentCalculationHBaseReducer.java:1)
> >
> >         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> >
> >         at
> > org.apache.hadoop.mapred.Task$NewCombinerRunner.combine(Task.java:1513)
> >
> >         at
> >
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1436)
> >
> >         at
> > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
> >
> >         at
> >
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:699)
> >
> >         at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
> >
> >         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> >
> >         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> >
> >         at java.security.AccessController.doPrivileged(Native Method)
> >
> >         at javax.security.auth.Subject.doAs(Subject.java:415)
> >
> >         at
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
> >
> >         at org.apache.hadoop.mapred.Child.main(Child.java:249)
> >
> >
> >
> > Providing the partial(excluding the business logic) codes
> >
> >
> > Mapper:
> >
> >
> > public class SentimentCalculationHBaseMapper extends TableMapper<Text,
> > Text> {
> >
> >
> >
> > private Text sentenseOriginal = new Text();
> >
> > private Text sentenseParsed = new Text();
> >
> >
> >
> > @Override
> >
> >     protected void map(
> >
> >             ImmutableBytesWritable key,
> >
> >             Result value,
> >
> >             org.apache.hadoop.mapreduce.Mapper<ImmutableBytesWritable,
> > Result, Text, Text>.Context context)
> >
> >             throws IOException, InterruptedException {
> >
> > context.write(this.sentenseOriginal, this.sentenseParsed);
> >
> > }
> >
> > }
> >
> > Reducer :
> >
> >
> > public class SentimentCalculationHBaseReducer extends
> >
> >         TableReducer<Text, Text, ImmutableBytesWritable