Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> combine two map tasks


Copy link to this message
-
Re: combine two map tasks
See this .. hope this answers your question .

http://developer.yahoo.com/hadoop/tutorial/module4.html#tips

On Sun, Jun 28, 2009 at 5:28 PM, bonito <[EMAIL PROTECTED]> wrote:

>
> Hello!
> I am a new hadoop user and my question may sound naive..
> However, I would like to ask if there is a way to combine the results of
> two
> mpa tasks that may "run" simultaneously.
> I use the MultipleInput class and thus I have two different mappers.
> I want the result/output of the one map (associated with one input file) to
> be used in the process of the second map (associated with the second input
> file).
> I have thought of storing the map1 output in the hdfs and retrieving it
> using the map2.
> However, I have no clue whether this is possible. I mean...what about
> time-executing issues? map2 has to wait until map1 is completed...
>
> The thought of executing them in a serial manner is not the one I really
> want...
>
> Any suggestion would be appreciated.
> Thank you in advance :)
>
> --
> View this message in context:
> http://www.nabble.com/combine-two-map-tasks-tp24240928p24240928.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB