Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - combine two map tasks


Copy link to this message
-
Re: combine two map tasks
bharath vissapragada 2009-06-28, 12:04
See this .. hope this answers your question .

http://developer.yahoo.com/hadoop/tutorial/module4.html#tips

On Sun, Jun 28, 2009 at 5:28 PM, bonito <[EMAIL PROTECTED]> wrote:

>
> Hello!
> I am a new hadoop user and my question may sound naive..
> However, I would like to ask if there is a way to combine the results of
> two
> mpa tasks that may "run" simultaneously.
> I use the MultipleInput class and thus I have two different mappers.
> I want the result/output of the one map (associated with one input file) to
> be used in the process of the second map (associated with the second input
> file).
> I have thought of storing the map1 output in the hdfs and retrieving it
> using the map2.
> However, I have no clue whether this is possible. I mean...what about
> time-executing issues? map2 has to wait until map1 is completed...
>
> The thought of executing them in a serial manner is not the one I really
> want...
>
> Any suggestion would be appreciated.
> Thank you in advance :)
>
> --
> View this message in context:
> http://www.nabble.com/combine-two-map-tasks-tp24240928p24240928.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>