Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> reading WAL log file from inside coprocessor


Copy link to this message
-
Re: reading WAL log file from inside coprocessor
I filed HBASE-4844 a while ago but never actually provided a patch.
If you can comment on that jira with the exact use case that'd be helpful.

If you wanted to work on a patch that'd be cooler even :)

-- Lars

________________________________
 From: Jan Van Besien <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Monday, October 29, 2012 2:42 AM
Subject: reading WAL log file from inside coprocessor
 
I am implementing some functionality as a HBase coprocessor which relies on reading WALEdits from the HBase WAL.

I have to requirements:

- get notified whenever a new WALEdit is written to the WAL
- have access to the actual WAL log file such that I can read from it whenever I want

In a way, this is very similar to how HBase replication works.

The first requirement is easy enough, using the WALObserver coprocessor interface. It is for the second requirement however, that I am asking this question.

HBase replication uses the WALActionsListener interface to get notified when the log files are rolled. There doesn't seem to be a way to do that in a WALObserver (coprocessor interface) though.

I thought of adding a WALActionsListener myself from inside the coprocessor code, but I would need to make sure that it is registered before the preLogRolled method is invoked for the first time.

I also looked at HBASE-4132, but I believe this talks about the WALActionsListener and not the WALObserver.
So.. any hints on how to do WAL reading (like replication) from inside a coprocessor?

thanks,
Jan
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB