Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Publishing HLog entries to a bus?


+
Otis Gospodnetic 2013-03-03, 03:08
+
Jean-Daniel Cryans 2013-03-04, 19:34
+
Nick Dimiduk 2013-03-04, 19:39
+
Otis Gospodnetic 2013-03-05, 22:50
+
Gary Helmling 2013-03-05, 23:32
Copy link to this message
-
Re: Publishing HLog entries to a bus?
It could also be done as a new type of replication sink I'd imagine. Either
way it seems we have the plumbing?

On Wednesday, March 6, 2013, Gary Helmling wrote:

> Wouldn't the WALObserver coprocessor interface work for this?  Is there
> anything missing from what it exposes?
>
>
> On Tue, Mar 5, 2013 at 2:50 PM, Otis Gospodnetic <
> [EMAIL PROTECTED] <javascript:;>
> > wrote:
>
> > Hi Nick,
> >
> > I don't know, what would be better/easier?
> > I'd think per RS may be OK, but you'll know better about where HLogs can
> be
> > "caught".
> >
> > Btw. that Databus project just happened to spark the idea, but I guess
> the
> > general version of what I'm thinking is simply: "Can we introduce some
> HLog
> > Listener interfaces/hooks so people can plug in there and do whatever
> they
> > want with HLog entries".  One such impl. can be for putting HLog entries
> on
> > that Databus.
> >
> > Otis
> > --
> > HBASE Performance Monitoring
> > http://sematext.com/spm/hbase-performance-monitoring/
> >
> >
> >
> >
> > On Mon, Mar 4, 2013 at 2:39 PM, Nick Dimiduk <[EMAIL PROTECTED]<javascript:;>>
> wrote:
> >
> > > Hi Otis,
> > >
> > > This is a cool idea. Would you see a single event pipe containing all
> > > edits, per RS pipes, something else entirely?
> > >
> > > -n
> > >
> > > On Sat, Mar 2, 2013 at 7:08 PM, Otis Gospodnetic <
> > > [EMAIL PROTECTED] <javascript:;>
> > > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I was looking at
> > > >
> > > >
> > >
> >
> http://engineering.linkedin.com/data-replication/open-sourcing-databus-linkedins-low-latency-change-data-capture-systemand
> > > > then at
> > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/tree/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserverand
> > > > wondering...
> > > >
> > > > .... would it be possible/interesting/better to "open up" a little
> bit
> > > and
> > > > implement something that puts HLog events on a bus like that Databus
> > and
> > > > thus allow *anyone* to do *anything* with data being written to (or
> > > deleted
> > > > from) HBase?  Replication itself, secondary indexing, etc.
> > > >
> > > > Thanks,
> > > > Otis
> > > > --
> > > > HBASE Performance Monitoring
> > > > http://sematext.com/spm/hbase-performance-monitoring/
> > > >
> > >
> >
>
--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB