Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro New M/R API with Oozie


Copy link to this message
-
Avro New M/R API with Oozie
What is the proper way to configure a workflow.xml file for oozie when using the new avro map reduce API?   If I am not mistaken, above the dashed line is the old way, and below is the new way.  This is wrong though because oozie does not even try and locate the mapper in new way I am using.   So basically I am asking how to specify the mapper and reducer in oozie with the new Avro m/r API?

Old Way?

<property>
<name>mapred.reducer.class</name>
<value>org.apache.avro.mapred.HadoopReducer</value>
</property>

<property>
<name>mapred.mapper.class</name>
<value>org.apache.avro.mapred.HadoopMapper</value>
</property>
<property>
<name>avro.reducer</name>
<value>org.project.mapreduce.MyReducer</value>
</property>
<property>
<name>avro.mapper</name>
<value>org.project.mapreduce.MyMapper</value>
</property>
-------------------------------------------------------
New Way?

<property>
<name>mapreduce.map.class</name>
<value>org.project.mapreduce.MyMapper</value>
</property>
<property>
<name>mapreduce.reduce.class</name>
<value> org.project.mapreduce.MyReducer</value>
</property>

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB