Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> speeding up demux


Copy link to this message
-
Re: speeding up demux
This seems possible, but one thing that would need to be changed is the
directories that demux uses. For example:
demuxProcessing/mrInput
demuxProcessing/mrOutput

These would need to dynamic directories with the timestamp or something else
in them to keep two jobs from interfering with each other.

On Thu, May 26, 2011 at 8:23 PM, Corbin Hoenes <[EMAIL PROTECTED]> wrote:

> Finding demux to be a bit too slow for our needs.  It seems like only 1
> runs at a time; is there some technical reason why we couldn't run a couple
> in parallel?  If so any hints on how difficult it would be to run multiple
> demuxers at a time?
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB