Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - [Error]Finding average using hbase hadoop


Copy link to this message
-
Re: [Error]Finding average using hbase hadoop
Jean-Marc Spaggiari 2013-08-16, 13:48
Hi Manish,

Have you replaced ALL the "symbol".getBytes() and "stocks".getBytes() and
so on?

You should remove all the string.getBytes() and replace them by
Bytes.toBytes(string).

If so, can you please paste you code on pastbin? Same for the exception.

Thanks,

JM

2013/8/16 manish dunani <[EMAIL PROTECTED]>

> Hello jean,
> I did the same as u told me..
> But,still face the same error.Didn't get any idea,what would be wrong??
> If u find then please do more elaboration on it..
>
>
> On Fri, Aug 16, 2013 at 5:02 PM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
> > Hi Manish,
> >
> > First, instead of "stocks".getBytes() you need to use
> > Bytes.toBytes("stocks"). Same for the other strings.
> > Second, on your map task, you creates 2 bytes arrays for those string at
> > each iteration. You don't want that on a production environment. You
> should
> > move that outside of the map method and make them final static.
> >
> > JM
> >
> > 2013/8/16 manish dunani <[EMAIL PROTECTED]>
> >
> > > hello,
> > >
> > > I am using apache hadoop 1.1.2 and hbase 0.94.9 on pseudo distibuted
> > mode.
> > >
> > > I am trying to find Average open stocks values.
> > >
> > > *sample dataset in hbase::**(table name:nyse4)*
> > >
> > >
> > >  2010-02-04           column=stocks:open, timestamp=1376567559424,
> > > value=2.5
> > >  2010-02-04           column=stocks:symbol, timestamp=1376567559424,
> > > value=QXM
> > >  2010-02-05           column=stocks:open, timestamp=1376567559429,
> > > value=2.42
> > >  2010-02-05           column=stocks:symbol, timestamp=1376567559429,
> > > value=QXM
> > >  2010-02-08           column=stocks:open, timestamp=1376567559431,
> > > value=2.33
> > >  2010-02-08           column=stocks:symbol, timestamp=1376567559431,
> > > value=QXM
> > >
> > > *code:*(please ignores the lines that are commenting)
> > >
> > >
> > > > package com.maddy;
> > > >
> > > > import java.io.IOException;
> > > >
> > > > import org.apache.hadoop.conf.Configuration;
> > > > import org.apache.hadoop.fs.Path;
> > > > import org.apache.hadoop.hbase.HBaseConfiguration;
> > > > import org.apache.hadoop.hbase.client.Put;
> > > > import org.apache.hadoop.hbase.client.Result;
> > > > import org.apache.hadoop.hbase.client.Scan;
> > > > import org.apache.hadoop.hbase.filter.FirstKeyOnlyFilter;
> > > > import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
> > > > import org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil;
> > > > import org.apache.hadoop.hbase.mapreduce.TableMapper;
> > > > import org.apache.hadoop.hbase.mapreduce.TableReducer;
> > > > import org.apache.hadoop.hbase.util.Bytes;
> > > > //import org.apache.hadoop.io.DoubleWritable;
> > > > import org.apache.hadoop.io.FloatWritable;
> > > > import org.apache.hadoop.mapreduce.Job;
> > > > import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
> > > >
> > > >
> > > > public class openaveragestock
> > > > {
> > > >     public static class map extends
> > > > TableMapper<ImmutableBytesWritable,FloatWritable>
> > > >     {
> > > >         @Override
> > > >
> > > >         public void map(ImmutableBytesWritable row,Result
> value,Context
> > > > context) throws IOException
> > > >         {
> > > >
> > > >             byte[]
> > > > val=(value.getValue("stocks".getBytes(),"open".getBytes()));
> > > >             //byte[]
> > > > val1=(value.getValue("stocks".getBytes(),"symbol".getBytes()));
> > > >
> > > >
> > > >             ImmutableBytesWritable stock_symbol=new
> > > > ImmutableBytesWritable("symbol".getBytes());
> > > >
> > > >
> > > >             try
> > > >             {
> > > >                 context.write(stock_symbol,new
> > > > FloatWritable(Bytes.toFloat(val)));
> > > >             }
> > > >             catch(InterruptedException e)
> > > >
> > > >             {
> > > >                  throw new IOException(e);
> > > >             }
> > > >
> > > >
> > > >         }
> > > >
> > > >
> > > >     }
> > > >
> > > >
> > > >     public static class reduce extends