I understand the need but I don't understand how to generate a reverse
timestamp. Can someone please explain how this is accomplished and how I
can test that its working correctly?

Thanks

HBase maps (row-key, column family name, column, timestamp) to a value.

The KeyValues are also sorted by the same attributes in reverse timestamp order.

The default timestamp is the current time, but you can set any long value (which does not need correlate in any way to the time domain)
as the time stamp and hence define your own ordering within multiple version of the same row key.

See Put: Put(byte[] row, long ts).

Is that what you meant?

-- Lars

________________________________
From: Mark <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Friday, August 19, 2011 6:39 PM
Subject: Reverse timestamp

I understand the need but I don't understand how to generate a reverse timestamp. Can someone please explain how this is accomplished and how I can test that its working correctly?

Thanks

Hi Mark,

Here is an example for generating reverse timestamps. Hope it helps:

http://devblog.streamy.com/2009/04/23/hbase-row-key-design-for-paging-limit-offset-queries/

Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>

<http://in.linkedin.com/in/sonalgoyal>

On Sat, Aug 20, 2011 at 10:56 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> HBase maps (row-key, column family name, column, timestamp) to a value.
>
> The KeyValues are also sorted by the same attributes in reverse timestamp
> order.
>
> The default timestamp is the current time, but you can set any long value
> (which does not need correlate in any way to the time domain)
> as the time stamp and hence define your own ordering within multiple
> version of the same row key.
>
> See Put: Put(byte[] row, long ts).
>
> Is that what you meant?
>
> -- Lars
>
>
>
> ________________________________
> From: Mark <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Friday, August 19, 2011 6:39 PM
> Subject: Reverse timestamp
>
> I understand the need but I don't understand how to generate a reverse
> timestamp. Can someone please explain how this is accomplished and how I can
> test that its working correctly?
>
> Thanks
>

Thanks but I was more referring to reverse ordered timestamps for
composite keys to be used for sorting purposes.

On 8/19/11 10:26 PM, lars hofhansl wrote:
> HBase maps (row-key, column family name, column, timestamp) to a value.
>
> The KeyValues are also sorted by the same attributes in reverse timestamp order.
>
> The default timestamp is the current time, but you can set any long value (which does not need correlate in any way to the time domain)
> as the time stamp and hence define your own ordering within multiple version of the same row key.
>
> See Put: Put(byte[] row, long ts).
>
> Is that what you meant?
>
> -- Lars
>
>
>
> ________________________________
> From: Mark<[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Friday, August 19, 2011 6:39 PM
> Subject: Reverse timestamp
>
> I understand the need but I don't understand how to generate a reverse timestamp. Can someone please explain how this is accomplished and how I can test that its working correctly?
>
> Thanks

I use this bit from the link sent by Sonal:
reverse_order_stamp is an 8 byte, big endian long with a value of (Long.MAX_VALUE - epoch).  This is so the most recent stamp is at the top rather than the bottom.

I use (Long.MAX_VALUE - System.getCurrentTimeMillis()).

-Steve

-----Original Message-----
From: Mark [mailto:[EMAIL PROTECTED]]
Sent: Saturday, August 20, 2011 6:29 AM
To: [EMAIL PROTECTED]
Subject: Re: Reverse timestamp

Thanks but I was more referring to reverse ordered timestamps for
composite keys to be used for sorting purposes.

On 8/19/11 10:26 PM, lars hofhansl wrote:
> HBase maps (row-key, column family name, column, timestamp) to a value.
>
> The KeyValues are also sorted by the same attributes in reverse timestamp order.
>
> The default timestamp is the current time, but you can set any long value (which does not need correlate in any way to the time domain)
> as the time stamp and hence define your own ordering within multiple version of the same row key.
>
> See Put: Put(byte[] row, long ts).
>
> Is that what you meant?
>
> -- Lars
>
>
>
> ________________________________
> From: Mark<[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Friday, August 19, 2011 6:39 PM
> Subject: Reverse timestamp
>
> I understand the need but I don't understand how to generate a reverse timestamp. Can someone please explain how this is accomplished and how I can test that its working correctly?
>
> Thanks

Thanks all but I'm actually using ruby as my client so there is no
Long.MAX_VALUE.

Apparently there is an alternative method...

"These composite row keys are similar to what RDBMSs offer, yet you can
control the sort order for each field separately. You could do, for
example, *a bitwise inversion of the date expressed as a long value (the
Linux epoch)*. This would then sort the rows descending by date"

I just want to be clear on the above statement. A bitwise inversion is
simply flipping the bits correct? So if the current time represented as
bits is

1001110010011111100101011001110

Then the bitwise inversion is

0110001101100000011010100000110

Is this correct?
On 8/20/11 7:10 AM, [EMAIL PROTECTED] wrote:
> I use this bit from the link sent by Sonal:
> reverse_order_stamp is an 8 byte, big endian long with a value of (Long.MAX_VALUE - epoch).  This is so the most recent stamp is at the top rather than the bottom.
>
> I use (Long.MAX_VALUE - System.getCurrentTimeMillis()).
>
> -Steve
>
> -----Original Message-----
> From: Mark [mailto:[EMAIL PROTECTED]]
> Sent: Saturday, August 20, 2011 6:29 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Reverse timestamp
>
> Thanks but I was more referring to reverse ordered timestamps for
> composite keys to be used for sorting purposes.
>
> On 8/19/11 10:26 PM, lars hofhansl wrote:
>> HBase maps (row-key, column family name, column, timestamp) to a value.
>>
>> The KeyValues are also sorted by the same attributes in reverse timestamp order.
>>
>> The default timestamp is the current time, but you can set any long value (which does not need correlate in any way to the time domain)
>> as the time stamp and hence define your own ordering within multiple version of the same row key.
>>
>> See Put: Put(byte[] row, long ts).
>>
>> Is that what you meant?
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>> From: Mark<[EMAIL PROTECTED]>
>> To: [EMAIL PROTECTED]
>> Sent: Friday, August 19, 2011 6:39 PM
>> Subject: Reverse timestamp
>>
>> I understand the need but I don't understand how to generate a reverse timestamp. Can someone please explain how this is accomplished and how I can test that its working correctly?
>>
>> Thanks

That's right if you're using *unsigned* integers. If your integers are
signed (can be negative), then keep the leftmost bit (the highest
significant bit) zero.

Mohamed

On Sat, Aug 20, 2011 at 10:56 AM, Mark <[EMAIL PROTECTED]> wrote:

> Thanks all but I'm actually using ruby as my client so there is no
> Long.MAX_VALUE.
>
> Apparently there is an alternative method...
>
> "These composite row keys are similar to what RDBMSs offer, yet you can
> control the sort order for each field separately. You could do, for example,
> *a bitwise inversion of the date expressed as a long value (the Linux
> epoch)*. This would then sort the rows descending by date"
>
> I just want to be clear on the above statement. A bitwise inversion is
> simply flipping the bits correct? So if the current time represented as bits
> is
>
> 100111001001111110010101100111**0
>
> Then the bitwise inversion is
>
> 011000110110000001101010000011**0
>
> Is this correct?
>
>
>
> On 8/20/11 7:10 AM, [EMAIL PROTECTED] wrote:
>
>> I use this bit from the link sent by Sonal:
>> reverse_order_stamp is an 8 byte, big endian long with a value of
>> (Long.MAX_VALUE - epoch).  This is so the most recent stamp is at the top
>> rather than the bottom.
>>
>> I use (Long.MAX_VALUE - System.getCurrentTimeMillis())**.
>>
>> -Steve
>>
>> -----Original Message-----
>> From: Mark [mailto:static.void.dev@gmail.**com<[EMAIL PROTECTED]>
>> ]
>> Sent: Saturday, August 20, 2011 6:29 AM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Reverse timestamp
>>
>> Thanks but I was more referring to reverse ordered timestamps for
>> composite keys to be used for sorting purposes.
>>
>> On 8/19/11 10:26 PM, lars hofhansl wrote:
>>
>>> HBase maps (row-key, column family name, column, timestamp) to a value.
>>>
>>> The KeyValues are also sorted by the same attributes in reverse timestamp
>>> order.
>>>
>>> The default timestamp is the current time, but you can set any long value
>>> (which does not need correlate in any way to the time domain)
>>> as the time stamp and hence define your own ordering within multiple
>>> version of the same row key.
>>>
>>> See Put: Put(byte[] row, long ts).
>>>
>>> Is that what you meant?
>>>
>>> -- Lars
>>>
>>>
>>>
>>> ______________________________**__
>>> From: Mark<[EMAIL PROTECTED]**>
>>> To: [EMAIL PROTECTED]
>>> Sent: Friday, August 19, 2011 6:39 PM
>>> Subject: Reverse timestamp
>>>
>>> I understand the need but I don't understand how to generate a reverse
>>> timestamp. Can someone please explain how this is accomplished and how I can
>>> test that its working correctly?
>>>
>>> Thanks
>>>
>>

Well, Long.MAX_VALUE is "A constant holding the maximum value a long can have, (2^63)-1".  I don't know much about ruby but I'm guessing that you could use ruby's bignum to hold that value or some value sufficiently large for your timestamp value.

-Steve

-----Original Message-----
From: Mark [mailto:[EMAIL PROTECTED]]
Sent: Saturday, August 20, 2011 7:57 AM
To: [EMAIL PROTECTED]
Subject: Re: Reverse timestamp

Thanks all but I'm actually using ruby as my client so there is no
Long.MAX_VALUE.

Apparently there is an alternative method...

"These composite row keys are similar to what RDBMSs offer, yet you can
control the sort order for each field separately. You could do, for
example, *a bitwise inversion of the date expressed as a long value (the
Linux epoch)*. This would then sort the rows descending by date"

I just want to be clear on the above statement. A bitwise inversion is
simply flipping the bits correct? So if the current time represented as
bits is

1001110010011111100101011001110

Then the bitwise inversion is

0110001101100000011010100000110

Is this correct?
On 8/20/11 7:10 AM, [EMAIL PROTECTED] wrote:
> I use this bit from the link sent by Sonal:
> reverse_order_stamp is an 8 byte, big endian long with a value of (Long.MAX_VALUE - epoch).  This is so the most recent stamp is at the top rather than the bottom.
>
> I use (Long.MAX_VALUE - System.getCurrentTimeMillis()).
>
> -Steve
>
> -----Original Message-----
> From: Mark [mailto:[EMAIL PROTECTED]]
> Sent: Saturday, August 20, 2011 6:29 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Reverse timestamp
>
> Thanks but I was more referring to reverse ordered timestamps for
> composite keys to be used for sorting purposes.
>
> On 8/19/11 10:26 PM, lars hofhansl wrote:
>> HBase maps (row-key, column family name, column, timestamp) to a value.
>>
>> The KeyValues are also sorted by the same attributes in reverse timestamp order.
>>
>> The default timestamp is the current time, but you can set any long value (which does not need correlate in any way to the time domain)
>> as the time stamp and hence define your own ordering within multiple version of the same row key.
>>
>> See Put: Put(byte[] row, long ts).
>>
>> Is that what you meant?
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>> From: Mark<[EMAIL PROTECTED]>
>> To: [EMAIL PROTECTED]
>> Sent: Friday, August 19, 2011 6:39 PM
>> Subject: Reverse timestamp
>>
>> I understand the need but I don't understand how to generate a reverse timestamp. Can someone please explain how this is accomplished and how I can test that its working correctly?
>>
>> Thanks

Shouldn't be an issue since it will always be used for the current
timestamp. Thanks

On 8/20/11 8:06 AM, Mohamed Ibrahim wrote:
> That's right if you're using *unsigned* integers. If your integers are
> signed (can be negative), then keep the leftmost bit (the highest
> significant bit) zero.
>
> Mohamed
>
>
>
> On Sat, Aug 20, 2011 at 10:56 AM, Mark<[EMAIL PROTECTED]>  wrote:
>
>> Thanks all but I'm actually using ruby as my client so there is no
>> Long.MAX_VALUE.
>>
>> Apparently there is an alternative method...
>>
>> "These composite row keys are similar to what RDBMSs offer, yet you can
>> control the sort order for each field separately. You could do, for example,
>> *a bitwise inversion of the date expressed as a long value (the Linux
>> epoch)*. This would then sort the rows descending by date"
>>
>> I just want to be clear on the above statement. A bitwise inversion is
>> simply flipping the bits correct? So if the current time represented as bits
>> is
>>
>> 100111001001111110010101100111**0
>>
>> Then the bitwise inversion is
>>
>> 011000110110000001101010000011**0
>>
>> Is this correct?
>>
>>
>>
>> On 8/20/11 7:10 AM, [EMAIL PROTECTED] wrote:
>>
>>> I use this bit from the link sent by Sonal:
>>> reverse_order_stamp is an 8 byte, big endian long with a value of
>>> (Long.MAX_VALUE - epoch).  This is so the most recent stamp is at the top
>>> rather than the bottom.
>>>
>>> I use (Long.MAX_VALUE - System.getCurrentTimeMillis())**.
>>>
>>> -Steve
>>>
>>> -----Original Message-----
>>> From: Mark [mailto:static.void.dev@gmail.**com<[EMAIL PROTECTED]>
>>> ]
>>> Sent: Saturday, August 20, 2011 6:29 AM
>>> To: [EMAIL PROTECTED]
>>> Subject: Re: Reverse timestamp
>>>
>>> Thanks but I was more referring to reverse ordered timestamps for
>>> composite keys to be used for sorting purposes.
>>>
>>> On 8/19/11 10:26 PM, lars hofhansl wrote:
>>>
>>>> HBase maps (row-key, column family name, column, timestamp) to a value.
>>>>
>>>> The KeyValues are also sorted by the same attributes in reverse timestamp
>>>> order.
>>>>
>>>> The default timestamp is the current time, but you can set any long value
>>>> (which does not need correlate in any way to the time domain)
>>>> as the time stamp and hence define your own ordering within multiple
>>>> version of the same row key.
>>>>
>>>> See Put: Put(byte[] row, long ts).
>>>>
>>>> Is that what you meant?
>>>>
>>>> -- Lars
>>>>
>>>>
>>>>
>>>> ______________________________**__
>>>> From: Mark<[EMAIL PROTECTED]**>
>>>> To: [EMAIL PROTECTED]
>>>> Sent: Friday, August 19, 2011 6:39 PM
>>>> Subject: Reverse timestamp
>>>>
>>>> I understand the need but I don't understand how to generate a reverse
>>>> timestamp. Can someone please explain how this is accomplished and how I can
>>>> test that its working correctly?
>>>>
>>>> Thanks
>>>>

Ah OK... In that case you can take any portion of the key and just invert all the bits.

For example you can have a compound key:
part1|part2, and say part2 is a date (from Date.getTime()) you just invert all the bits and use that as part2.
Inverting the bits is nice, because you need to know nothing about the domain of the key. I.e. it works for strings, longs, ints, etc.
For floating points it's a bit more tricky.
You can look at the HBaseIndex library that comes with Lily for an example.
http://www.lilyproject.org/lily/about/downloads.html, look at the FieldDefinition classes.
-- Lars

________________________________
From: Mark <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Saturday, August 20, 2011 6:29 AM
Subject: Re: Reverse timestamp

Thanks but I was more referring to reverse ordered timestamps for
composite keys to be used for sorting purposes.

On 8/19/11 10:26 PM, lars hofhansl wrote:
> HBase maps (row-key, column family name, column, timestamp) to a value.
>
> The KeyValues are also sorted by the same attributes in reverse timestamp order.
>
> The default timestamp is the current time, but you can set any long value (which does not need correlate in any way to the time domain)
> as the time stamp and hence define your own ordering within multiple version of the same row key.
>
> See Put: Put(byte[] row, long ts).
>
> Is that what you meant?
>
> -- Lars
>
>
>
> ________________________________
> From: Mark<[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Friday, August 19, 2011 6:39 PM
> Subject: Reverse timestamp
>
> I understand the need but I don't understand how to generate a reverse timestamp. Can someone please explain how this is accomplished and how I can test that its working correctly?
>
> Thanks