Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> How pig make hadoop to recognize "pig.reduce.output.dirs" property to set output directory?


Copy link to this message
-
How pig make hadoop to recognize "pig.reduce.output.dirs" property to set output directory?
Hi,

I would like to ask a question on pig property to set the output directory
of MapReduce Job produced by pig. As we know,
"mapreduce.output.fileoutputformat.outputdir" or "mapred.output.dir" is the
property we use to set the output directory.
It seems that pig sets "pig.reduce.output.dirs" property in addition to the
property mentioned above.
They usually have the same value but I find that sometimes they don't (e.g
PigMix L12 - a case of multi stores), they are different. In this case,
hadoop follows "pig.reduce.output.dirs" property.

Could anyone explain how pig make hadoop use "pig.reduce.output.dirs"
property, instead of the usual property
("mapreduce.output.fileoutputformat.outputdir" or "mapred.output.dir"), to
set the output directory?
Please feel free to correct me if there is any false point in my question.

Thank you in advance.
Best Regards,
JI MAHN

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB