Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> Possible issue when using non-bundled hadoop


Copy link to this message
-
Possible issue when using non-bundled hadoop
A pig user brought to my attention that PIG_OPTS are not being picked up
but "pig -Dfoo=bar script.pig" works. I dug a little into it and found that
in the case when an external hadoop (not bundled with pig) is used we rely
on bin/hadoop script for starting up pig. In the process we pass PIG_OPTS
as HADOOP_OPTS to the hadoop script.

bin/hadoop in turn picks up HADOOP_OPTS from
HADOOP_HOME/conf/hadoop-env.sh. It might be possible a user completely
resets HADOOP_OPTS here and PIG_OPTS is cleared.

# Extra Java runtime options.  Empty by default.
export HADOOP_OPTS="-server"

This will be noticeable since Pig 0.9.1
https://issues.apache.org/jira/browse/PIG-2239

Users need to be aware of this and make sure they do not reset HADOOP_OPTS.
Instead this will work
# Extra Java runtime options.  Empty by default.
export HADOOP_OPTS="-server $HADOOP_OPTS"

Thought it would be useful info if this ever comes up.

-Prashant