Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Re: BigInt / longlong


Copy link to this message
-
Re: BigInt / longlong
Tatu Saloranta 2012-03-29, 16:54
On Thu, Mar 29, 2012 at 12:20 AM, Meyer, Dennis <[EMAIL PROTECTED]> wrote:
> Hi,
>
> That's not the best idea as it's wasting a lot of space as encoding eats
> up lots of space (e.g. 1Byte ASCII, 2-3Byte for UTF-8). Especially as AVRO
> uses the MSB for compressing smaller ints, this does not seem very keen
> for mass data.
>
> I'll see if 64Bit unsigned -> 64Bit signed conversion or using the matisse
> of double works better for us.

Wouldn't going via double is every bit as bad an idea as using Strings
(i.e. neither makes much sense to me) -- double operations are rather
costly, and you still lose many more bits on magnitude. So why would
you consider conversions to/from doubles?

But as Scott pointed out, most platforms use simple 2s complement, so
that simple cast should just work (i.e. it's all just matter of
interpretation), as well as basic arithmetics (add, subtract,
non-sign-extending shifts). So as long as you deal with values as
unsigned ints, code should work.

-+ Tatu +-