Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Mixing streaming and regular map reduct jobs


Copy link to this message
-
Mixing streaming and regular map reduct jobs
I have a problem where I am using Java and the hadoop APIS to run a map
reduce job on data that can be considered as a set of lines of text.
At the reduce stage I have a collection of lines of text to process in a
convenient order. There are a number of programs written in Python or Perl
which
can handle this data in streaming form and it would be useful as one of the
reduce steps to stream the data to these programs -
I am not sure if this is possible and certainly not sure how it might be
done - does anyone have any bright ideas?

--
Steven M. Lewis PhD
Institute for Systems Biology
Seattle WA
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB