Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Jean-Daniel: RE: some data replication support in hbase?


Copy link to this message
-
RE: Jean-Daniel: RE: some data replication support in hbase?
Seems like hooking into replication would be a good approach.

There's also a JIRA open about a changes API.  https://issues.apache.org/jira/browse/HBASE-3247

Or you could use Coprocessors which are committed in 0.92 / trunk.  The pre/post hooks can be used as a per-operation trigger mechanism.

> -----Original Message-----
> From: Hiller, Dean (Contractor) [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, December 21, 2010 12:02 PM
> To: [EMAIL PROTECTED]
> Subject: Jean-Daniel: RE: some data replication support in hbase?
>
> Actually yes. Very nice read.  Is there an api that we could hook in triggers
> too?
>
> Ie. This could help us two ways...we could more easily to replication to
> Sybase AND we have this need where we want to just write data into
> clusters and have code that just runs as soon as that happens on those nodes
> closest to that data.  Kind of like map/reduce that runs once data came into
> the cluster right next to the data that came in.
>
> (I have been reading up on Cassandra and it seems to have stuff there, but
> we are leaning towards hbase/hadoop at this point so I am trying to find
> some stuff in hbase for this).  or is Cassandra the only one with triggers at this
> point?
>
> Thanks,
> Dean
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Jean-
> Daniel Cryans
> Sent: Tuesday, December 21, 2010 11:40 AM
> To: [EMAIL PROTECTED]
> Subject: Re: some data replication support in hbase?
>
> Have you read this?
> http://hbase.apache.org/docs/r0.89.20100924/replication.html
>
> It's still experimental but we've been using it here since September with
> success (I also happen to be the one who wrote the feature).
>
> J-D
>
> On Tue, Dec 21, 2010 at 9:14 AM, Hiller, Dean  (Contractor)
> <[EMAIL PROTECTED]> wrote:
> > Are there any hooks in hbase to do data replication?  We have to try
> > to move our 12 hour batch jobs down to 3 hours or so and are looking
> > at moving into a noSQL environment, but currently, customers have
> > replicated data(only a small subset of tables because our data set
> > size is so big).  Are there any good strategies for data replication?
> >
> >
> >
> > It probably doesn't matter but our customers' local db(multiple
> > customers) is Sybase right now(as I think we edicted that a while back
> > to them).  Any ideas here?  All we really care about is that it is
> > eventually consistent with our cluster.
> >
> >
> >
> > I think we may also have issues where the update of two rows should
> > have what hbase had before or after kind of thing.
> >
> >
> >
> > Ideas?
> >
> > Thanks,
> >
> > Dean
> >
> >
> >
> >
> >
> >
> > This message and any attachments are intended only for the use of the
> > addressee and may contain information that is privileged and
> > confidential. If the reader of the message is not the intended
> > recipient or an authorized representative of the intended recipient,
> > you are hereby notified that any dissemination of this communication
> > is strictly prohibited. If you have received this communication in
> > error, please notify us immediately by e-mail and delete the message and
> any attachments from your system.
> >
> >
> This message and any attachments are intended only for the use of the
> addressee and may contain information that is privileged and confidential. If
> the reader of the message is not the intended recipient or an authorized
> representative of the intended recipient, you are hereby notified that any
> dissemination of this communication is strictly prohibited. If you have
> received this communication in error, please notify us immediately by e-mail
> and delete the message and any attachments from your system.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB