Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # user - Accumulo Mutations published to Redis pub/sub via a Constraint


Copy link to this message
-
Accumulo Mutations published to Redis pub/sub via a Constraint
Jamie Stephens 2013-06-27, 23:16
Another quick one:

Here's a Constraint as a Mutation hook that publishes mutations to Redis
pub/sub.

  https://gist.github.com/jsmorph/5881142

Not a production version, and really needs exposed options (as with
Constraints), but the hardcoded version is serviceable, I guess.  I'll see
about adding a generic hook facility for this kind of thing.

Yes, it's dangerous to mess with the write path.  And doing IO there is
particularly worrisome!  Also would be easy to kill Redis instances, but
then again a bunch of Redis instances can be pretty fast.  If the hook does
some filtering before the IO, then maybe things will be better.

With those scary caveats in mind, doing some external event processing on
the other side of the datastore can be interesting.

Example:

  Start up Redis (wherever it needs to be -- see the code), and start
'redis-cli':

     subscribe accumulo

  Then at the Accumulo shell:

    createtable foo
    constraint -a org.apache.accumulo.examples.simple.constraints.RedisPub
    insert b c x d
    insert b c x e
    insert b c x f
    delete b c x

  From redis-cli, you should see

  1) "message"
  2) "accumulo"
  3) "800162090163017800000001640100"
  1) "message"
  2) "accumulo"
  3) "800162090163017800000001650100"
  1) "message"
  2) "accumulo"
  3) "800162090163017800000001660100"
  1) "message"
  2) "accumulo"
  3) "8001620801630178000001000100"
  1) "message"
  2) "accumulo"
  3) "8001620801630178000001000100"

  You can deserialize those mutations with
'RedisPub.deserialize(RedisPub.unhex(...))'.

Again, please let me know of any fixes or suggestions.

--Jamie