Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> speeding up demux


Copy link to this message
-
Re: speeding up demux
This seems possible, but one thing that would need to be changed is the
directories that demux uses. For example:
demuxProcessing/mrInput
demuxProcessing/mrOutput

These would need to dynamic directories with the timestamp or something else
in them to keep two jobs from interfering with each other.

On Thu, May 26, 2011 at 8:23 PM, Corbin Hoenes <[EMAIL PROTECTED]> wrote:

> Finding demux to be a bit too slow for our needs.  It seems like only 1
> runs at a time; is there some technical reason why we couldn't run a couple
> in parallel?  If so any hints on how difficult it would be to run multiple
> demuxers at a time?
>
>
>