Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Blur >> mail # dev >> new queue capability


Copy link to this message
-
Re: new queue capability
On Thu, Feb 27, 2014 at 11:10 AM, Aaron McCurry <[EMAIL PROTECTED]> wrote:

That may be good longer term - I'd be supportive of pulling it from
the shards for now and focus some time fully baking the simpler idea
in the controller.  Or at least disclaiming there be dragons:)
Not sure what you mean by enqueueMutate - I was thinking just taking
the existing QueueReader and plugging it into the Controller (with
some leader election) - obviously calling mutateRow instead of the
current behavior.

Any more than one controller and we have to either expose the
partitioning or protect against dupes, right?
Yeah, from a user perspective, I think I'd ask these questions of the Blur API:
o) What "streams" are available? (e.g. twitter, jms, kafka)
o) Create an instance of a stream (e.g.
client.createStreamTable(type:twitter, name:apache))
- o) Add stream-specific arguments to the table (e.g. twitter search criteria)
o) Add one or more Filter's to a stream table (e.g. the default
twitter stream might index 'mentions' but the user might add a Filter
to drop that column) or drop whole messages.
o) Start/Stop the stream.
o) Get metrics on the stream's activity.

Thanks,