Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - BigDecimalColumnInterpreter


+
Julian Wissmann 2012-09-05, 16:17
+
Ted Yu 2012-09-05, 16:22
+
Julian Wissmann 2012-09-05, 19:07
+
Ted Yu 2012-09-05, 19:22
+
Julian Wissmann 2012-09-05, 19:49
+
Ted Yu 2012-09-05, 20:04
+
Julian Wissmann 2012-09-05, 20:30
+
anil gupta 2012-09-05, 21:04
+
anil gupta 2012-09-05, 21:27
+
Julian Wissmann 2012-09-06, 09:28
+
anil gupta 2012-09-06, 17:19
+
anil gupta 2012-09-06, 17:24
+
Julian Wissmann 2012-09-06, 18:43
+
anil gupta 2012-09-06, 19:22
+
Julian Wissmann 2012-09-10, 16:36
+
anil gupta 2012-09-10, 18:22
+
Julian Wissmann 2012-09-12, 16:27
+
Ted Yu 2012-09-12, 16:45
+
Julian Wissmann 2012-09-12, 19:56
Copy link to this message
-
Re: BigDecimalColumnInterpreter
anil gupta 2012-09-14, 07:56
Hi Julian,

I am still failing to understand how the same
BigDecimalColumnInterpreter(BDCI) is working at my end. I am doing the
interpretation of the bytes in getValue method and i havent faced any
trouble yet. There has to be some difference between our set-up or you have
taken a very different approach using BDCI. I have created a util class
which will create a table in Hbase and then run a aggregation. Here is the
link to class: https://dl.dropbox.com/u/64149128/TestBigDecimalCI.java .
Please make sure that you use the correct libraries at client and server
side. You can run this code directly from the client side. I am using HBase
0.92.1(cdh4.0.0). Let me know the outcome or if you face any error in
running that class. I dont have the cluster access right now. If possible,
tomorrow in morning i will also try to provide the stack trace of method
call of BigDecimalColumnInterpreter.getValue().

@Ted: It would be of great help if you can also give the above mentioned
utility a try on your own set-up and let me know the outcome.

Thanks,
Anil Gupta

On Wed, Sep 12, 2012 at 12:56 PM, Julian Wissmann
<[EMAIL PROTECTED]>wrote:

> Cool!
> I'm sure I'll find some time to digg into it early next week if nobody else
> lusts after it ;-)
>
> Cheers
>
> 2012/9/12 Ted Yu <[EMAIL PROTECTED]>
>
> > Thanks for digging, Julian.
> >
> > Looks like we need to support BigDecimal in HbaseObjectWritable
> >
> > Actually once a test is written for BigDecimalColumnInterpreter, it would
> > become much easier for anyone to debug this issue.
> >
> > On Wed, Sep 12, 2012 at 9:27 AM, Julian Wissmann
> > <[EMAIL PROTECTED]>wrote:
> >
> > > Hi,
> > >
> > > so I'm slowly getting an overview of the code, here. I haven't really
> > > understood the problem yet, though.
> > >
> > > DataInput and DataOutput cannot handle BigDecimal, which seems to be
> > > somehwere close to the root cause of the problem.
> > > The error is being triggered in HBaseServer on line
> > > 1642         param.readFields(dis);
> > > which calls org.apache.hadoop.io.writable, which implements write and
> > > readFields(), and I assume is being implemented by
> > > HbaseObjectWritable#readFields.
> > > In HbaseObjectWritable#readObject DataInput in then gets checked for
> > being
> > > a primitive data type  and read accordingly.
> > >
> > > Now if I interpret Bytes#valueOf() correctly, it just takes a
> BigDecimal
> > > value and converts _just_ the value to byte[] and not the whole object.
> > So
> > > what readObject finds here, should be interpreted as byte[] and happily
> > > passed on. The first method, that should even care about parsing this
> to
> > > BigDecimal would then be BigDecimalColumnInterpreter#getValue()
> > >
> > > To test this, I decided to overwrite write and readFields, as I inherit
> > > them from Exec, anyway, however I have no understanding of how these
> > > methods work.
> > > I put in a few printlns to get a feeling for it, but turns out it is
> > never
> > > even beling called, at all.
> > >
> > >
> > > 2012/9/10 anil gupta <[EMAIL PROTECTED]>
> > >
> > > > Hi Julian,
> > > >
> > > > I am using only cdh4 libraries. I use the jars present under hadoop
> and
> > > > hbase installed dir. In my last email i gave you some more pointers.
> > Try
> > > to
> > > > follow them and see what happens.
> > > > If then also it doesn't works for you, then i will try to write an
> > > utility
> > > > to test BigDecimalColumnInterpreter on your setup also.
> > > >
> > > > Thanks,
> > > > Anil
> > > >
> > > > On Mon, Sep 10, 2012 at 9:36 AM, Julian Wissmann
> > > > <[EMAIL PROTECTED]>wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I haven't really gotten to working on this, since last wednesday.
> > > > > Checked readFields() and write() today, but don't really see, why I
> > > would
> > > > > need to reimplement those. Admittedly I'm not that into the whole
> > Hbase
> > > > > codebase, yet, so there is a good chance I'm missing something,
> here.

Thanks & Regards,
Anil Gupta
+
anil gupta 2012-09-14, 19:27
+
anil gupta 2012-09-14, 22:17
+
Ted Yu 2012-09-15, 14:56
+
anil gupta 2012-09-15, 15:26
+
Julian Wissmann 2012-10-03, 16:26