Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Stop chained mapreduce.


+
ilyal levin 2011-09-11, 11:52
Copy link to this message
-
Re: Stop chained mapreduce.
Ilyal,

The MR output files names follow the pattern part-#### and you'll have as
many as reducers your job had.

As you know the output directory, you could do a fs.listStatus() of the
output directory and check all the part-* files.

Hope this helps.

Thanks.

Alejandro

On Sun, Sep 11, 2011 at 4:52 AM, ilyal levin <[EMAIL PROTECTED]> wrote:

> Hi
> I created a chained mapreduce program where each job creates a SequenceFile
> output.
> My stopping condition is simply to check if the last output file (Type -
> SequenceFile) is empty.
>  In order to do that i need to use the SequenceFile.Reader
> and for him to read the data i need the path of the output file. The
> problem is that i don't know the name of the file,
> it usually depends on the number of the reducer. What can i do to solve
> this?
>
> Thanks.
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB