Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> combine two map tasks

Copy link to this message
Re: combine two map tasks
See this .. hope this answers your question .


On Sun, Jun 28, 2009 at 5:28 PM, bonito <[EMAIL PROTECTED]> wrote:

> Hello!
> I am a new hadoop user and my question may sound naive..
> However, I would like to ask if there is a way to combine the results of
> two
> mpa tasks that may "run" simultaneously.
> I use the MultipleInput class and thus I have two different mappers.
> I want the result/output of the one map (associated with one input file) to
> be used in the process of the second map (associated with the second input
> file).
> I have thought of storing the map1 output in the hdfs and retrieving it
> using the map2.
> However, I have no clue whether this is possible. I mean...what about
> time-executing issues? map2 has to wait until map1 is completed...
> The thought of executing them in a serial manner is not the one I really
> want...
> Any suggestion would be appreciated.
> Thank you in advance :)
> --
> View this message in context:
> http://www.nabble.com/combine-two-map-tasks-tp24240928p24240928.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.