Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Intermediate files generated.


Copy link to this message
-
Re: Intermediate files generated.
Hemanth Yamijala 2010-07-02, 01:52
Alex,

> I don't think this is what I am looking for. Essential, I wish to run both
> mapper as well as reducer. But at the same time, i wish to make sure that
> the temp files that are used between mappers and reducers are of my choice.
> Here, the choice means that I can specify the files in HDFS that can be used
> as temp files.

Could you explain why you want to do this ?

>
> thanks,
> --PB.
>
> On Fri, Jul 2, 2010 at 12:14 AM, Alex Loddengaard <[EMAIL PROTECTED]> wrote:
>
>> You could use the HDFS API from within your mapper, and run with 0
>> reducers.
>>
>> Alex
>>
>> On Thu, Jul 1, 2010 at 3:07 PM, Pramy Bhats <[EMAIL PROTECTED]
>> >wrote:
>>
>> > Hi,
>> >
>> > I am using hadoop framework for writing MapReduce jobs. I want  to
>> redirect
>> > the output of Map into files of my choice and later use those files as
>> > input
>> > for Reduce phase.
>> >
>> >
>> > Could you please suggest, how to proceed for it ?
>> >
>> > thanks,
>> > --PB.
>> >
>>
>