Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> reading WAL log file from inside coprocessor


Copy link to this message
-
reading WAL log file from inside coprocessor
I am implementing some functionality as a HBase coprocessor which relies
on reading WALEdits from the HBase WAL.

I have to requirements:

- get notified whenever a new WALEdit is written to the WAL
- have access to the actual WAL log file such that I can read from it
whenever I want

In a way, this is very similar to how HBase replication works.

The first requirement is easy enough, using the WALObserver coprocessor
interface. It is for the second requirement however, that I am asking
this question.

HBase replication uses the WALActionsListener interface to get notified
when the log files are rolled. There doesn't seem to be a way to do that
in a WALObserver (coprocessor interface) though.

I thought of adding a WALActionsListener myself from inside the
coprocessor code, but I would need to make sure that it is registered
before the preLogRolled method is invoked for the first time.

I also looked at HBASE-4132, but I believe this talks about the
WALActionsListener and not the WALObserver.
So.. any hints on how to do WAL reading (like replication) from inside a
coprocessor?

thanks,
Jan
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB