Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Jean-Daniel: RE: some data replication support in hbase?


Copy link to this message
-
RE: Jean-Daniel: RE: some data replication support in hbase?
Seems like hooking into replication would be a good approach.

There's also a JIRA open about a changes API.  https://issues.apache.org/jira/browse/HBASE-3247

Or you could use Coprocessors which are committed in 0.92 / trunk.  The pre/post hooks can be used as a per-operation trigger mechanism.

> -----Original Message-----
> From: Hiller, Dean (Contractor) [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, December 21, 2010 12:02 PM
> To: [EMAIL PROTECTED]
> Subject: Jean-Daniel: RE: some data replication support in hbase?
>
> Actually yes. Very nice read.  Is there an api that we could hook in triggers
> too?
>
> Ie. This could help us two ways...we could more easily to replication to
> Sybase AND we have this need where we want to just write data into
> clusters and have code that just runs as soon as that happens on those nodes
> closest to that data.  Kind of like map/reduce that runs once data came into
> the cluster right next to the data that came in.
>
> (I have been reading up on Cassandra and it seems to have stuff there, but
> we are leaning towards hbase/hadoop at this point so I am trying to find
> some stuff in hbase for this).  or is Cassandra the only one with triggers at this
> point?
>
> Thanks,
> Dean
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Jean-
> Daniel Cryans
> Sent: Tuesday, December 21, 2010 11:40 AM
> To: [EMAIL PROTECTED]
> Subject: Re: some data replication support in hbase?
>
> Have you read this?
> http://hbase.apache.org/docs/r0.89.20100924/replication.html
>
> It's still experimental but we've been using it here since September with
> success (I also happen to be the one who wrote the feature).
>
> J-D
>
> On Tue, Dec 21, 2010 at 9:14 AM, Hiller, Dean  (Contractor)
> <[EMAIL PROTECTED]> wrote:
> > Are there any hooks in hbase to do data replication?  We have to try
> > to move our 12 hour batch jobs down to 3 hours or so and are looking
> > at moving into a noSQL environment, but currently, customers have
> > replicated data(only a small subset of tables because our data set
> > size is so big).  Are there any good strategies for data replication?
> >
> >
> >
> > It probably doesn't matter but our customers' local db(multiple
> > customers) is Sybase right now(as I think we edicted that a while back
> > to them).  Any ideas here?  All we really care about is that it is
> > eventually consistent with our cluster.
> >
> >
> >
> > I think we may also have issues where the update of two rows should
> > have what hbase had before or after kind of thing.
> >
> >
> >
> > Ideas?
> >
> > Thanks,
> >
> > Dean
> >
> >
> >
> >
> >
> >
> > This message and any attachments are intended only for the use of the
> > addressee and may contain information that is privileged and
> > confidential. If the reader of the message is not the intended
> > recipient or an authorized representative of the intended recipient,
> > you are hereby notified that any dissemination of this communication
> > is strictly prohibited. If you have received this communication in
> > error, please notify us immediately by e-mail and delete the message and
> any attachments from your system.
> >
> >
> This message and any attachments are intended only for the use of the
> addressee and may contain information that is privileged and confidential. If
> the reader of the message is not the intended recipient or an authorized
> representative of the intended recipient, you are hereby notified that any
> dissemination of this communication is strictly prohibited. If you have
> received this communication in error, please notify us immediately by e-mail
> and delete the message and any attachments from your system.