Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> [Error]Finding average using hbase hadoop


Copy link to this message
-
Re: [Error]Finding average using hbase hadoop
Hello jean,
I did the same as u told me..
But,still face the same error.Didn't get any idea,what would be wrong??
If u find then please do more elaboration on it..
On Fri, Aug 16, 2013 at 5:02 PM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Hi Manish,
>
> First, instead of "stocks".getBytes() you need to use
> Bytes.toBytes("stocks"). Same for the other strings.
> Second, on your map task, you creates 2 bytes arrays for those string at
> each iteration. You don't want that on a production environment. You should
> move that outside of the map method and make them final static.
>
> JM
>
> 2013/8/16 manish dunani <[EMAIL PROTECTED]>
>
> > hello,
> >
> > I am using apache hadoop 1.1.2 and hbase 0.94.9 on pseudo distibuted
> mode.
> >
> > I am trying to find Average open stocks values.
> >
> > *sample dataset in hbase::**(table name:nyse4)*
> >
> >
> >  2010-02-04           column=stocks:open, timestamp=1376567559424,
> > value=2.5
> >  2010-02-04           column=stocks:symbol, timestamp=1376567559424,
> > value=QXM
> >  2010-02-05           column=stocks:open, timestamp=1376567559429,
> > value=2.42
> >  2010-02-05           column=stocks:symbol, timestamp=1376567559429,
> > value=QXM
> >  2010-02-08           column=stocks:open, timestamp=1376567559431,
> > value=2.33
> >  2010-02-08           column=stocks:symbol, timestamp=1376567559431,
> > value=QXM
> >
> > *code:*(please ignores the lines that are commenting)
> >
> >
> > > package com.maddy;
> > >
> > > import java.io.IOException;
> > >
> > > import org.apache.hadoop.conf.Configuration;
> > > import org.apache.hadoop.fs.Path;
> > > import org.apache.hadoop.hbase.HBaseConfiguration;
> > > import org.apache.hadoop.hbase.client.Put;
> > > import org.apache.hadoop.hbase.client.Result;
> > > import org.apache.hadoop.hbase.client.Scan;
> > > import org.apache.hadoop.hbase.filter.FirstKeyOnlyFilter;
> > > import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
> > > import org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil;
> > > import org.apache.hadoop.hbase.mapreduce.TableMapper;
> > > import org.apache.hadoop.hbase.mapreduce.TableReducer;
> > > import org.apache.hadoop.hbase.util.Bytes;
> > > //import org.apache.hadoop.io.DoubleWritable;
> > > import org.apache.hadoop.io.FloatWritable;
> > > import org.apache.hadoop.mapreduce.Job;
> > > import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
> > >
> > >
> > > public class openaveragestock
> > > {
> > >     public static class map extends
> > > TableMapper<ImmutableBytesWritable,FloatWritable>
> > >     {
> > >         @Override
> > >
> > >         public void map(ImmutableBytesWritable row,Result value,Context
> > > context) throws IOException
> > >         {
> > >
> > >             byte[]
> > > val=(value.getValue("stocks".getBytes(),"open".getBytes()));
> > >             //byte[]
> > > val1=(value.getValue("stocks".getBytes(),"symbol".getBytes()));
> > >
> > >
> > >             ImmutableBytesWritable stock_symbol=new
> > > ImmutableBytesWritable("symbol".getBytes());
> > >
> > >
> > >             try
> > >             {
> > >                 context.write(stock_symbol,new
> > > FloatWritable(Bytes.toFloat(val)));
> > >             }
> > >             catch(InterruptedException e)
> > >
> > >             {
> > >                  throw new IOException(e);
> > >             }
> > >
> > >
> > >         }
> > >
> > >
> > >     }
> > >
> > >
> > >     public static class reduce extends
> > >
> TableReducer<ImmutableBytesWritable,FloatWritable,ImmutableBytesWritable>
> > >     {
> > >
> > >         @Override
> > >         public void reduce(ImmutableBytesWritable
> > > key,Iterable<FloatWritable>values,Context context) throws IOException,
> > > InterruptedException
> > >         {
> > >             float sum=0;
> > >             int count=0;
> > >           //  float average=0;
> > >             for(FloatWritable val:values)
> > >             {
> > >                 sum+=val.get();
> > >                 count++;

Regards

*Manish Dunani*
*Contact No* : +91 9408329137
*skype id* : manish.dunani*
*