Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> BigDecimalColumnInterpreter


Copy link to this message
-
Re: BigDecimalColumnInterpreter
Hi Julian,

I am still failing to understand how the same
BigDecimalColumnInterpreter(BDCI) is working at my end. I am doing the
interpretation of the bytes in getValue method and i havent faced any
trouble yet. There has to be some difference between our set-up or you have
taken a very different approach using BDCI. I have created a util class
which will create a table in Hbase and then run a aggregation. Here is the
link to class: https://dl.dropbox.com/u/64149128/TestBigDecimalCI.java .
Please make sure that you use the correct libraries at client and server
side. You can run this code directly from the client side. I am using HBase
0.92.1(cdh4.0.0). Let me know the outcome or if you face any error in
running that class. I dont have the cluster access right now. If possible,
tomorrow in morning i will also try to provide the stack trace of method
call of BigDecimalColumnInterpreter.getValue().

@Ted: It would be of great help if you can also give the above mentioned
utility a try on your own set-up and let me know the outcome.

Thanks,
Anil Gupta

On Wed, Sep 12, 2012 at 12:56 PM, Julian Wissmann
<[EMAIL PROTECTED]>wrote:

> Cool!
> I'm sure I'll find some time to digg into it early next week if nobody else
> lusts after it ;-)
>
> Cheers
>
> 2012/9/12 Ted Yu <[EMAIL PROTECTED]>
>
> > Thanks for digging, Julian.
> >
> > Looks like we need to support BigDecimal in HbaseObjectWritable
> >
> > Actually once a test is written for BigDecimalColumnInterpreter, it would
> > become much easier for anyone to debug this issue.
> >
> > On Wed, Sep 12, 2012 at 9:27 AM, Julian Wissmann
> > <[EMAIL PROTECTED]>wrote:
> >
> > > Hi,
> > >
> > > so I'm slowly getting an overview of the code, here. I haven't really
> > > understood the problem yet, though.
> > >
> > > DataInput and DataOutput cannot handle BigDecimal, which seems to be
> > > somehwere close to the root cause of the problem.
> > > The error is being triggered in HBaseServer on line
> > > 1642         param.readFields(dis);
> > > which calls org.apache.hadoop.io.writable, which implements write and
> > > readFields(), and I assume is being implemented by
> > > HbaseObjectWritable#readFields.
> > > In HbaseObjectWritable#readObject DataInput in then gets checked for
> > being
> > > a primitive data type  and read accordingly.
> > >
> > > Now if I interpret Bytes#valueOf() correctly, it just takes a
> BigDecimal
> > > value and converts _just_ the value to byte[] and not the whole object.
> > So
> > > what readObject finds here, should be interpreted as byte[] and happily
> > > passed on. The first method, that should even care about parsing this
> to
> > > BigDecimal would then be BigDecimalColumnInterpreter#getValue()
> > >
> > > To test this, I decided to overwrite write and readFields, as I inherit
> > > them from Exec, anyway, however I have no understanding of how these
> > > methods work.
> > > I put in a few printlns to get a feeling for it, but turns out it is
> > never
> > > even beling called, at all.
> > >
> > >
> > > 2012/9/10 anil gupta <[EMAIL PROTECTED]>
> > >
> > > > Hi Julian,
> > > >
> > > > I am using only cdh4 libraries. I use the jars present under hadoop
> and
> > > > hbase installed dir. In my last email i gave you some more pointers.
> > Try
> > > to
> > > > follow them and see what happens.
> > > > If then also it doesn't works for you, then i will try to write an
> > > utility
> > > > to test BigDecimalColumnInterpreter on your setup also.
> > > >
> > > > Thanks,
> > > > Anil
> > > >
> > > > On Mon, Sep 10, 2012 at 9:36 AM, Julian Wissmann
> > > > <[EMAIL PROTECTED]>wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I haven't really gotten to working on this, since last wednesday.
> > > > > Checked readFields() and write() today, but don't really see, why I
> > > would
> > > > > need to reimplement those. Admittedly I'm not that into the whole
> > Hbase
> > > > > codebase, yet, so there is a good chance I'm missing something,
> here.

Thanks & Regards,
Anil Gupta
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB