Hi,

In my application my row values are doubles. I would like my scans to
traverse the rows in order of increasing values.

But if I simply use

double d  = ....
byte[] row = Bytes.toBytes(d);

I will get an ordering which is based on the byte values of doubles, not on
the value of the doubles themselves.

I see also that integer values have the same issue due to the first bit
being the sign bit. So negative values will come after positive values.

Any suggestions?

Thanks,

Jon

On Mon, Nov 5, 2012 at 10:41 AM, Jonathan Bishop <[EMAIL PROTECTED]> wrote:
> Hi,
>
> In my application my row values are doubles. I would like my scans to
> traverse the rows in order of increasing values.
>
> But if I simply use
>
> double d  = ....
> byte[] row = Bytes.toBytes(d);
>
> I will get an ordering which is based on the byte values of doubles, not on
> the value of the doubles themselves.

It's the same ordering as long as you don't use negative values.

>
> I see also that integer values have the same issue due to the first bit
> being the sign bit. So negative values will come after positive values.
>
> Any suggestions?

Don't use row keys that can be negative? :)

Also don't use a single number as a row key, see
http://hbase.apache.org/book.html#rowkey.design

J-D

Hi Jonathan,

If possible try to avoid using Double for rowkey because Double has
precision problems.
Here is more details about problems with double:
1.
http://stackoverflow.com/questions/12699040/double-vs-long-serialization-in-java
2.
http://stackoverflow.com/questions/179427/how-to-resolve-a-java-rounding-double-issue

You might get wrong results due problems in Double. You can use two long or
int to store the decimal value as RowKey.

HTH,
Anil Gupta
On Mon, Nov 5, 2012 at 11:38 AM, Jean-Daniel Cryans <[EMAIL PROTECTED]>wrote:

> On Mon, Nov 5, 2012 at 10:41 AM, Jonathan Bishop <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> >
> > In my application my row values are doubles. I would like my scans to
> > traverse the rows in order of increasing values.
> >
> > But if I simply use
> >
> > double d  = ....
> > byte[] row = Bytes.toBytes(d);
> >
> > I will get an ordering which is based on the byte values of doubles, not
> on
> > the value of the doubles themselves.
>
> It's the same ordering as long as you don't use negative values.
>
> >
> > I see also that integer values have the same issue due to the first bit
> > being the sign bit. So negative values will come after positive values.
> >
> > Any suggestions?
>
> Don't use row keys that can be negative? :)
>
> Also don't use a single number as a row key, see
> http://hbase.apache.org/book.html#rowkey.design
>
> J-D
>

--
Thanks & Regards,
Anil Gupta

Have a look at the lily library. It has code to encode Longs/Doubles into bytes such that resulting bytes sort as expected based on the numbers.

-- Lars

________________________________
 From: Jonathan Bishop <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Sent: Monday, November 5, 2012 10:41 AM
Subject: Using doubles and longs as ordering row values
 
Hi,

In my application my row values are doubles. I would like my scans to
traverse the rows in order of increasing values.

But if I simply use

double d  = ....
byte[] row = Bytes.toBytes(d);

I will get an ordering which is based on the byte values of doubles, not on
the value of the doubles themselves.

I see also that integer values have the same issue due to the first bit
being the sign bit. So negative values will come after positive values.

Any suggestions?

Thanks,

Jon

Here's a project to deal with this issue specifically.  I'm not sure of
it's status:
https://github.com/conikeec/orderly

On Mon, Nov 5, 2012 at 4:01 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> Have a look at the lily library. It has code to encode Longs/Doubles into
> bytes such that resulting bytes sort as expected based on the numbers.
>
> -- Lars
>
>
>
> ________________________________
>  From: Jonathan Bishop <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Sent: Monday, November 5, 2012 10:41 AM
> Subject: Using doubles and longs as ordering row values
>
> Hi,
>
> In my application my row values are doubles. I would like my scans to
> traverse the rows in order of increasing values.
>
> But if I simply use
>
> double d  = ....
> byte[] row = Bytes.toBytes(d);
>
> I will get an ordering which is based on the byte values of doubles, not on
> the value of the doubles themselves.
>
> I see also that integer values have the same issue due to the first bit
> being the sign bit. So negative values will come after positive values.
>
> Any suggestions?
>
> Thanks,
>
> Jon
>

This fork looks a bit more up to date:
https://github.com/ndimiduk/orderly

On Mon, Nov 5, 2012 at 4:26 PM, Dave Latham <[EMAIL PROTECTED]> wrote:

> Here's a project to deal with this issue specifically.  I'm not sure of
> it's status:
> https://github.com/conikeec/orderly
>
>
> On Mon, Nov 5, 2012 at 4:01 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
>> Have a look at the lily library. It has code to encode Longs/Doubles into
>> bytes such that resulting bytes sort as expected based on the numbers.
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>>  From: Jonathan Bishop <[EMAIL PROTECTED]>
>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Sent: Monday, November 5, 2012 10:41 AM
>> Subject: Using doubles and longs as ordering row values
>>
>> Hi,
>>
>> In my application my row values are doubles. I would like my scans to
>> traverse the rows in order of increasing values.
>>
>> But if I simply use
>>
>> double d  = ....
>> byte[] row = Bytes.toBytes(d);
>>
>> I will get an ordering which is based on the byte values of doubles, not
>> on
>> the value of the doubles themselves.
>>
>> I see also that integer values have the same issue due to the first bit
>> being the sign bit. So negative values will come after positive values.
>>
>> Any suggestions?
>>
>> Thanks,
>>
>> Jon
>>
>
>

Thanks Dave,

That looks like what I need.

Jon
On Mon, Nov 5, 2012 at 4:27 PM, Dave Latham <[EMAIL PROTECTED]> wrote:

> This fork looks a bit more up to date:
> https://github.com/ndimiduk/orderly
>
> On Mon, Nov 5, 2012 at 4:26 PM, Dave Latham <[EMAIL PROTECTED]> wrote:
>
> > Here's a project to deal with this issue specifically.  I'm not sure of
> > it's status:
> > https://github.com/conikeec/orderly
> >
> >
> > On Mon, Nov 5, 2012 at 4:01 PM, lars hofhansl <[EMAIL PROTECTED]>
> wrote:
> >
> >> Have a look at the lily library. It has code to encode Longs/Doubles
> into
> >> bytes such that resulting bytes sort as expected based on the numbers.
> >>
> >> -- Lars
> >>
> >>
> >>
> >> ________________________________
> >>  From: Jonathan Bishop <[EMAIL PROTECTED]>
> >> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> >> Sent: Monday, November 5, 2012 10:41 AM
> >> Subject: Using doubles and longs as ordering row values
> >>
> >> Hi,
> >>
> >> In my application my row values are doubles. I would like my scans to
> >> traverse the rows in order of increasing values.
> >>
> >> But if I simply use
> >>
> >> double d  = ....
> >> byte[] row = Bytes.toBytes(d);
> >>
> >> I will get an ordering which is based on the byte values of doubles, not
> >> on
> >> the value of the doubles themselves.
> >>
> >> I see also that integer values have the same issue due to the first bit
> >> being the sign bit. So negative values will come after positive values.
> >>
> >> Any suggestions?
> >>
> >> Thanks,
> >>
> >> Jon
> >>
> >
> >
>

Hello Lars,

Thank you. Where can I find the lily library? I can't find it on github or
Google.

/David
On Fri, Nov 30, 2012 at 3:54 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> As I said, look at the lily library they have solved exactly that problem,
> I've used that before.
>
> It has encodings for Ints/Longs/Floats/Doubles/BigDecimals, to encode them
> such the byte array will sort according to the magnitude of the value which
> includes the sign and the floating point exponent.
>
> It's a very common problem :)
>
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: David Koch <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Cc:
> Sent: Thursday, November 29, 2012 3:00 PM
> Subject: Re: Using doubles and longs as ordering row values
>
> Hello,
>
> I am having a similar issue, only I need to preserve the order of
> qualifiers which are serialized signed longs - rather than row keys. The
> latter is addressed by the orderly library which was mentioned above. Can
> this library be re-used for my purpose? I imagine this is not an exotic
> requirement so I am also interested in knowing how other people have solved
> this problem.
>
> Thank you,
>
> /David
>
>
> On Tue, Nov 6, 2012 at 6:07 AM, Jonathan Bishop <[EMAIL PROTECTED]
> >wrote:
>
> > Thanks Dave,
> >
> > That looks like what I need.
> >
> > Jon
> >
> >
> > On Mon, Nov 5, 2012 at 4:27 PM, Dave Latham <[EMAIL PROTECTED]> wrote:
> >
> > > This fork looks a bit more up to date:
> > > https://github.com/ndimiduk/orderly
> > >
> > > On Mon, Nov 5, 2012 at 4:26 PM, Dave Latham <[EMAIL PROTECTED]>
> wrote:
> > >
> > > > Here's a project to deal with this issue specifically.  I'm not sure
> of
> > > > it's status:
> > > > https://github.com/conikeec/orderly
> > > >
> > > >
> > > > On Mon, Nov 5, 2012 at 4:01 PM, lars hofhansl <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > >> Have a look at the lily library. It has code to encode Longs/Doubles
> > > into
> > > >> bytes such that resulting bytes sort as expected based on the
> numbers.
> > > >>
> > > >> -- Lars
> > > >>
> > > >>
> > > >>
> > > >> ________________________________
> > > >>  From: Jonathan Bishop <[EMAIL PROTECTED]>
> > > >> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> > > >> Sent: Monday, November 5, 2012 10:41 AM
> > > >> Subject: Using doubles and longs as ordering row values
> > > >>
> > > >> Hi,
> > > >>
> > > >> In my application my row values are doubles. I would like my scans
> to
> > > >> traverse the rows in order of increasing values.
> > > >>
> > > >> But if I simply use
> > > >>
> > > >> double d  = ....
> > > >> byte[] row = Bytes.toBytes(d);
> > > >>
> > > >> I will get an ordering which is based on the byte values of doubles,
> > not
> > > >> on
> > > >> the value of the doubles themselves.
> > > >>
> > > >> I see also that integer values have the same issue due to the first
> > bit
> > > >> being the sign bit. So negative values will come after positive
> > values.
> > > >>
> > > >> Any suggestions?
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Jon
> > > >>
> > > >
> > > >
> > >
> >
>
>

On 12/01/2012 10:01 AM, David Koch wrote:
> Hello Lars,
>
> Thank you. Where can I find the lily library? I can't find it on github or
> Google.
http://www.lilyproject.org/lily/index.html
>
> /David
>
>
> On Fri, Nov 30, 2012 at 3:54 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
>> As I said, look at the lily library they have solved exactly that problem,
>> I've used that before.
>>
>> It has encodings for Ints/Longs/Floats/Doubles/BigDecimals, to encode them
>> such the byte array will sort according to the magnitude of the value which
>> includes the sign and the floating point exponent.
>>
>> It's a very common problem :)
>>
>>
>> -- Lars
>>
>>
>>
>> ----- Original Message -----
>> From: David Koch <[EMAIL PROTECTED]>
>> To: [EMAIL PROTECTED]
>> Cc:
>> Sent: Thursday, November 29, 2012 3:00 PM
>> Subject: Re: Using doubles and longs as ordering row values
>>
>> Hello,
>>
>> I am having a similar issue, only I need to preserve the order of
>> qualifiers which are serialized signed longs - rather than row keys. The
>> latter is addressed by the orderly library which was mentioned above. Can
>> this library be re-used for my purpose? I imagine this is not an exotic
>> requirement so I am also interested in knowing how other people have solved
>> this problem.
>>
>> Thank you,
>>
>> /David
>>
>>
>> On Tue, Nov 6, 2012 at 6:07 AM, Jonathan Bishop <[EMAIL PROTECTED]
>>> wrote:
>>> Thanks Dave,
>>>
>>> That looks like what I need.
>>>
>>> Jon
>>>
>>>
>>> On Mon, Nov 5, 2012 at 4:27 PM, Dave Latham <[EMAIL PROTECTED]> wrote:
>>>
>>>> This fork looks a bit more up to date:
>>>> https://github.com/ndimiduk/orderly
>>>>
>>>> On Mon, Nov 5, 2012 at 4:26 PM, Dave Latham <[EMAIL PROTECTED]>
>> wrote:
>>>>> Here's a project to deal with this issue specifically.  I'm not sure
>> of
>>>>> it's status:
>>>>> https://github.com/conikeec/orderly
>>>>>
>>>>>
>>>>> On Mon, Nov 5, 2012 at 4:01 PM, lars hofhansl <[EMAIL PROTECTED]>
>>>> wrote:
>>>>>> Have a look at the lily library. It has code to encode Longs/Doubles
>>>> into
>>>>>> bytes such that resulting bytes sort as expected based on the
>> numbers.
>>>>>> -- Lars
>>>>>>
>>>>>>
>>>>>>
>>>>>> ________________________________
>>>>>>   From: Jonathan Bishop <[EMAIL PROTECTED]>
>>>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>>>>>> Sent: Monday, November 5, 2012 10:41 AM
>>>>>> Subject: Using doubles and longs as ordering row values
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> In my application my row values are doubles. I would like my scans
>> to
>>>>>> traverse the rows in order of increasing values.
>>>>>>
>>>>>> But if I simply use
>>>>>>
>>>>>> double d  = ....
>>>>>> byte[] row = Bytes.toBytes(d);
>>>>>>
>>>>>> I will get an ordering which is based on the byte values of doubles,
>>> not
>>>>>> on
>>>>>> the value of the doubles themselves.
>>>>>>
>>>>>> I see also that integer values have the same issue due to the first
>>> bit
>>>>>> being the sign bit. So negative values will come after positive
>>> values.
>>>>>> Any suggestions?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Jon
>>>>>>
>>>>>
>>
>
> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>
> http://www.uci.cu
> http://www.facebook.com/universidad.uci
> http://www.flickr.com/photos/universidad_uci

--

Marcos Luis Ort�z Valmaseda
about.me/marcosortiz <http://about.me/marcosortiz>
@marcosluis2186 <http://twitter.com/marcosluis2186>

10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci