Hey guys,
in my application the HBase timestamp is used as version in my logic.
I would like to know what is the best way to insert a new record and get its timestamp.

I have come up with two possibilities:

/* I could force timestamp, but it is not a good idea since different servers
 * write into HBase which could lead to crazy behavior */
new Put(row, timestamp);

/* Or I could write into HBase and read it back. But I don't know how much overhead
 * this option causes.*/
@Override
public void put(Put put) throws IOException {
    byte[] row = put.getRow();
    hTableInterface.put(put);
    KeyValue kv = hTableInterface.get(new Get(row)).getColumnLatest(family, qualifier);
    long version = kv.getTimestamp();
}

Is there any better way to do it?

Thanks,
Pablo

What do you mean by using TS as version? Are you determining the ts long value before and then setting it in the Put object? If so, I think you can use a specific cell as a counter (Sequence in Oracle language, or Auto Increment column in MySQL). In that case of course you need the value of the TS so that's not a problem.

On 31 ביול 2012, at 01:13, Pablo Musa <[EMAIL PROTECTED]> wrote:

> Hey guys,
> in my application the HBase timestamp is used as version in my logic.
> I would like to know what is the best way to insert a new record and get its timestamp.
>
> I have come up with two possibilities:
>
> /* I could force timestamp, but it is not a good idea since different servers
> * write into HBase which could lead to crazy behavior */
> new Put(row, timestamp);
>
> /* Or I could write into HBase and read it back. But I don't know how much overhead
> * this option causes.*/
> @Override
> public void put(Put put) throws IOException {
>    byte[] row = put.getRow();
>    hTableInterface.put(put);
>    KeyValue kv = hTableInterface.get(new Get(row)).getColumnLatest(family, qualifier);
>    long version = kv.getTimestamp();
> }
>
> Is there any better way to do it?
>
> Thanks,
> Pablo

On Mon, Jul 30, 2012 at 11:13 PM, Pablo Musa <[EMAIL PROTECTED]> wrote:
> Hey guys,
> in my application the HBase timestamp is used as version in my logic.
> I would like to know what is the best way to insert a new record and get its timestamp.
>
> I have come up with two possibilities:
>
> /* I could force timestamp, but it is not a good idea since different servers
>  * write into HBase which could lead to crazy behavior */
> new Put(row, timestamp);
>
> /* Or I could write into HBase and read it back. But I don't know how much overhead
>  * this option causes.*/
> @Override
> public void put(Put put) throws IOException {
>     byte[] row = put.getRow();
>     hTableInterface.put(put);
>     KeyValue kv = hTableInterface.get(new Get(row)).getColumnLatest(family, qualifier);
>     long version = kv.getTimestamp();
> }
>
> Is there any better way to do it?
>

Not that I know of.

The latter is problematic because another client may have inserted a
version between your put and the get.  The former could work but it
depends on your applications version semantic.  Is it ok if another
client does an update with the same 'version/timestamp'?

St.Ack

> What do you mean by using TS as version?

My application uses versioning between client and server. So I am using the timestamp as a version.

> Are you determining the ts long value before and then setting it in the Put object? If so, I think you can
> use a specific cell as a counter. In that case of course you need the value of the TS so that's not a problem.

The problem is that I would insert an increment in one column for different inserts along the table.
Every column has a timestamp, but just the "latest" timestamp is important to the client. By latest
I mean the latest when the client contacted the server, of course it will change.

> The latter is problematic because another client may have inserted a version between your put and the get.

Actually this is not an issue in my logic. This will not happen at all. Thus I am concerned with the overhead.

> The former could work but it depends on your applications version semantic.

It usually works, but server clocks can be unsynchronized and lead us into a loss of data scenario.
  
> Is it ok if another client does an update with the same 'version/timestamp'?

Not at all. The important thing is that the client has all data until a specific version, which is the
timestamp of the last insert in the last communication.

Thank you very much for the ideas!

-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Stack
Sent: terça-feira, 31 de julho de 2012 18:12
To: [EMAIL PROTECTED]
Subject: Re: Retrieve Put timestamp

On Mon, Jul 30, 2012 at 11:13 PM, Pablo Musa <[EMAIL PROTECTED]> wrote:
> Hey guys,
> in my application the HBase timestamp is used as version in my logic.
> I would like to know what is the best way to insert a new record and get its timestamp.
>
> I have come up with two possibilities:
>
> /* I could force timestamp, but it is not a good idea since different
> servers
>  * write into HBase which could lead to crazy behavior */ new Put(row,
> timestamp);
>
> /* Or I could write into HBase and read it back. But I don't know how
> much overhead
>  * this option causes.*/
> @Override
> public void put(Put put) throws IOException {
>     byte[] row = put.getRow();
>     hTableInterface.put(put);
>     KeyValue kv = hTableInterface.get(new Get(row)).getColumnLatest(family, qualifier);
>     long version = kv.getTimestamp();
> }
>
> Is there any better way to do it?
>

Not that I know of.

The latter is problematic because another client may have inserted a version between your put and the get.  The former could work but it depends on your applications version semantic.  Is it ok if another client does an update with the same 'version/timestamp'?

St.Ack

There is no HBase API for this.
However, this could useful in some scenario, so maybe we could add an API for this.
It's not entirely trivial, though.
________________________________
From: Pablo Musa <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Sent: Monday, July 30, 2012 3:13 PM
Subject: Retrieve Put timestamp

Hey guys,
in my application the HBase timestamp is used as version in my logic.
I would like to know what is the best way to insert a new record and get its timestamp.

I have come up with two possibilities:

/* I could force timestamp, but it is not a good idea since different servers
* write into HBase which could lead to crazy behavior */
new Put(row, timestamp);

/* Or I could write into HBase and read it back. But I don't know how much overhead
* this option causes.*/
@Override
public void put(Put put) throws IOException {
    byte[] row = put.getRow();
    hTableInterface.put(put);
    KeyValue kv = hTableInterface.get(new Get(row)).getColumnLatest(family, qualifier);
    long version = kv.getTimestamp();
}

Is there any better way to do it?

Thanks,
Pablo

We have a similar requirement and here is the solution in our mind:
add a coprocessor, in prePut() get the current ms and set it to put ---
the current implementation get the current ms and set it in put()
return the ms generated to prePut() to client. For now put() does not
return any value. we need to change the behavior of it

Any flaw in this design?
Thanks,

Wei

From:   lars hofhansl <[EMAIL PROTECTED]>
To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
Date:   08/01/2012 12:37 PM
Subject:        Re: Retrieve Put timestamp

There is no HBase API for this.
However, this could useful in some scenario, so maybe we could add an API
for this.
It's not entirely trivial, though.
________________________________
From: Pablo Musa <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Sent: Monday, July 30, 2012 3:13 PM
Subject: Retrieve Put timestamp

Hey guys,
in my application the HBase timestamp is used as version in my logic.
I would like to know what is the best way to insert a new record and get
its timestamp.

I have come up with two possibilities:

/* I could force timestamp, but it is not a good idea since different
servers
* write into HBase which could lead to crazy behavior */
new Put(row, timestamp);

/* Or I could write into HBase and read it back. But I don't know how much
overhead
* this option causes.*/
@Override
public void put(Put put) throws IOException {
    byte[] row = put.getRow();
    hTableInterface.put(put);
    KeyValue kv = hTableInterface.get(new
Get(row)).getColumnLatest(family, qualifier);
    long version = kv.getTimestamp();
}

Is there any better way to do it?

Thanks,
Pablo

On Wed, Aug 1, 2012 at 7:12 PM, Wei Tan <[EMAIL PROTECTED]> wrote:
> We have a similar requirement and here is the solution in our mind:
> add a coprocessor, in prePut() get the current ms and set it to put ---
> the current implementation get the current ms and set it in put()
> return the ms generated to prePut() to client. For now put() does not
> return any value. we need to change the behavior of it
>
> Any flaw in this design?

In 0.96 we have moved to protobufs.  The put/mutate call currently
doesn't return anything:

message MutateResponse {
  optional Result result = 1;

  // used for mutate to indicate processed only
  optional bool processed = 2;
}

Should be easy enough changing it to run timestamps?  Should it do it
always or should we return the request so you have to ask for it?

St.Ack

Currently in Append there is a setter to specify whether to return the result or not. Similar way we can use for Put? Only with specific use cases the return TS might be needed.
May be in a generic way we can return the attributes of the Mutation? So any thing which the client needs back can be added into the attributes [Any byte[] value]
and we can return the same to client [If the flag is turned on] User can add these attributes using pre/post CP hooks.

-Anoop-
________________________________________
From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] on behalf of Stack [[EMAIL PROTECTED]]
Sent: Thursday, August 02, 2012 3:41 AM
To: [EMAIL PROTECTED]
Subject: Re: Retrieve Put timestamp

On Wed, Aug 1, 2012 at 7:12 PM, Wei Tan <[EMAIL PROTECTED]> wrote:
> We have a similar requirement and here is the solution in our mind:
> add a coprocessor, in prePut() get the current ms and set it to put ---
> the current implementation get the current ms and set it in put()
> return the ms generated to prePut() to client. For now put() does not
> return any value. we need to change the behavior of it
>
> Any flaw in this design?

In 0.96 we have moved to protobufs.  The put/mutate call currently
doesn't return anything:

message MutateResponse {
  optional Result result = 1;

  // used for mutate to indicate processed only
  optional bool processed = 2;
}

Should be easy enough changing it to run timestamps?  Should it do it
always or should we return the request so you have to ask for it?

St.Ack

+1.  Anyway all mutations extends OperationsWithAttributes also.

Regards
Ram
> -----Original Message-----
> From: Anoop Sam John [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, August 02, 2012 10:13 AM
> To: [EMAIL PROTECTED]
> Subject: RE: Retrieve Put timestamp
>
> Currently in Append there is a setter to specify whether to return the
> result or not. Similar way we can use for Put? Only with specific use
> cases the return TS might be needed.
> May be in a generic way we can return the attributes of the Mutation?
> So any thing which the client needs back can be added into the
> attributes [Any byte[] value]
> and we can return the same to client [If the flag is turned on] User
> can add these attributes using pre/post CP hooks.
>
> -Anoop-
> ________________________________________
> From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] on behalf of Stack
> [[EMAIL PROTECTED]]
> Sent: Thursday, August 02, 2012 3:41 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Retrieve Put timestamp
>
> On Wed, Aug 1, 2012 at 7:12 PM, Wei Tan <[EMAIL PROTECTED]> wrote:
> > We have a similar requirement and here is the solution in our mind:
> > add a coprocessor, in prePut() get the current ms and set it to put -
> --
> > the current implementation get the current ms and set it in put()
> > return the ms generated to prePut() to client. For now put() does not
> > return any value. we need to change the behavior of it
> >
> > Any flaw in this design?
>
> In 0.96 we have moved to protobufs.  The put/mutate call currently
> doesn't return anything:
>
> message MutateResponse {
>   optional Result result = 1;
>
>   // used for mutate to indicate processed only
>   optional bool processed = 2;
> }
>
> Should be easy enough changing it to run timestamps?  Should it do it
> always or should we return the request so you have to ask for it?
>
> St.Ack

+1.
So far I think timestamp is very useful. I would imagine if we can
configure the return, say in pre/post put, it would be even nicer.
Thanks,
Wei

Wei Tan
Research Staff Member
IBM T. J. Watson Research Center
19 Skyline Dr, Hawthorne, NY  10532
[EMAIL PROTECTED]; 914-784-6752

From:   "Ramkrishna.S.Vasudevan" <[EMAIL PROTECTED]>
To:     <[EMAIL PROTECTED]>,
Date:   08/02/2012 12:54 AM
Subject:        RE: Retrieve Put timestamp

+1.  Anyway all mutations extends OperationsWithAttributes also.

Regards
Ram
> -----Original Message-----
> From: Anoop Sam John [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, August 02, 2012 10:13 AM
> To: [EMAIL PROTECTED]
> Subject: RE: Retrieve Put timestamp
>
> Currently in Append there is a setter to specify whether to return the
> result or not. Similar way we can use for Put? Only with specific use
> cases the return TS might be needed.
> May be in a generic way we can return the attributes of the Mutation?
> So any thing which the client needs back can be added into the
> attributes [Any byte[] value]
> and we can return the same to client [If the flag is turned on] User
> can add these attributes using pre/post CP hooks.
>
> -Anoop-
> ________________________________________
> From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] on behalf of Stack
> [[EMAIL PROTECTED]]
> Sent: Thursday, August 02, 2012 3:41 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Retrieve Put timestamp
>
> On Wed, Aug 1, 2012 at 7:12 PM, Wei Tan <[EMAIL PROTECTED]> wrote:
> > We have a similar requirement and here is the solution in our mind:
> > add a coprocessor, in prePut() get the current ms and set it to put -
> --
> > the current implementation get the current ms and set it in put()
> > return the ms generated to prePut() to client. For now put() does not
> > return any value. we need to change the behavior of it
> >
> > Any flaw in this design?
>
> In 0.96 we have moved to protobufs.  The put/mutate call currently
> doesn't return anything:
>
> message MutateResponse {
>   optional Result result = 1;
>
>   // used for mutate to indicate processed only
>   optional bool processed = 2;
> }
>
> Should be easy enough changing it to run timestamps?  Should it do it
> always or should we return the request so you have to ask for it?
>
> St.Ack

I wonder if there is any follow up on this issue, i.e., a put can return a
timestamp of the record? Thanks!

Best Regards,
Wei

From:   Wei Tan/Watson/IBM
To:     [EMAIL PROTECTED],
Date:   08/02/2012 12:37 PM
Subject:        RE: Retrieve Put timestamp
+1.
So far I think timestamp is very useful. I would imagine if we can
configure the return, say in pre/post put, it would be even nicer.
Thanks,
Wei

Wei Tan
Research Staff Member
IBM T. J. Watson Research Center
19 Skyline Dr, Hawthorne, NY  10532
[EMAIL PROTECTED]; 914-784-6752
From:   "Ramkrishna.S.Vasudevan" <[EMAIL PROTECTED]>
To:     <[EMAIL PROTECTED]>,
Date:   08/02/2012 12:54 AM
Subject:        RE: Retrieve Put timestamp

+1.  Anyway all mutations extends OperationsWithAttributes also.

Regards
Ram
> -----Original Message-----
> From: Anoop Sam John [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, August 02, 2012 10:13 AM
> To: [EMAIL PROTECTED]
> Subject: RE: Retrieve Put timestamp
>
> Currently in Append there is a setter to specify whether to return the
> result or not. Similar way we can use for Put? Only with specific use
> cases the return TS might be needed.
> May be in a generic way we can return the attributes of the Mutation?
> So any thing which the client needs back can be added into the
> attributes [Any byte[] value]
> and we can return the same to client [If the flag is turned on] User
> can add these attributes using pre/post CP hooks.
>
> -Anoop-
> ________________________________________
> From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] on behalf of Stack
> [[EMAIL PROTECTED]]
> Sent: Thursday, August 02, 2012 3:41 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Retrieve Put timestamp
>
> On Wed, Aug 1, 2012 at 7:12 PM, Wei Tan <[EMAIL PROTECTED]> wrote:
> > We have a similar requirement and here is the solution in our mind:
> > add a coprocessor, in prePut() get the current ms and set it to put -
> --
> > the current implementation get the current ms and set it in put()
> > return the ms generated to prePut() to client. For now put() does not
> > return any value. we need to change the behavior of it
> >
> > Any flaw in this design?
>
> In 0.96 we have moved to protobufs.  The put/mutate call currently
> doesn't return anything:
>
> message MutateResponse {
>   optional Result result = 1;
>
>   // used for mutate to indicate processed only
>   optional bool processed = 2;
> }
>
> Should be easy enough changing it to run timestamps?  Should it do it
> always or should we return the request so you have to ask for it?
>
> St.Ack