Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Assigning the same partition number to the mapper output


Copy link to this message
-
Re: Assigning the same partition number to the mapper output
Some flexibility is there when it comes to changing the name of the output.
Check out MultipleOutputs

Never used it with a map only job.

Thanks,
Rahul
On Thu, Jun 13, 2013 at 8:33 AM, Maysam Yabandeh <[EMAIL PROTECTED]>wrote:

> Hi,
>
> I was wondering if it is possible in hadoop to assign the same partition
> numbers to the map outputs. I am running a map-only job (with zero
> reducers) and hadoop shuffles the partitions in the output: i.e.
> input/part-m-0000X is processed by task number Y and hence generates
> output/part-m-0000Y (where X != Y).
>
> Thanks
>
> Maysam
>