Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Different behaviour with same Pig Version


Copy link to this message
-
Re: Different behaviour with same Pig Version
forget it ! It came from hadoop-env.sh provided by the package I installed on one cluster :(

Regards
--
Cyril SCETBON

On May 3, 2013, at 11:44 AM, [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]> wrote:

Hi,

I have 2 differents behaviour for the same Pig version (with hadoop 1.1.2) on different servers. If anyone can tell me why with the same versions and the same parameters I don't have the same behaviour. I'm trying to set the builtin import list but it's not used with one hadoop cluster whereas it's used when I use it on a single node cluster :

+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
+ '[' '' == true ']'
+ '[' -n /usr/local/bin/hadoop ']'
+ '[' false == true ']'
+ '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar ']'
++ echo /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ '[' -z /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ '[' -n /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
+ export 'HADOOP_CLASSPATH=/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ HADOOP_CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ export 'HADOOP_OPTS=-Xmx1000m  -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/.. -Djava.net.preferIPv4Stack=true '
+ HADOOP_OPTS='-Xmx1000m  -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/.. -Djava.net.preferIPv4Stack=true '
+ '[' false == true ']'
+ exec /usr/bin/hadoop jar /opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
2013-05-03 07:57:55,140 [main] INFO  org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
2013-05-03 07:57:55,141 [main] INFO  org.apache.pig.Main - Logging error messages to: /home/ubuntu/pig_1367567875107.log
2013-05-03 07:57:57,502 [main] INFO  org.apache.pig.impl.util.Utils - Default bootup file /home/ubuntu/.pigbootup not found
2013-05-03 07:57:58,461 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://hadoop-cass-1:9100
2013-05-03 07:57:59,663 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: hadoop-cass-1:8021
2013-05-03 07:58:01,387 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve CassandraStorage2 using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]

+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
+ '[' '' == true ']'
+ '[' -n /usr/local/bin/hadoop ']'
+ '[' false == true ']'
+ '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar ']'
++ echo /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ '[' -z /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ '[' -n /usr/local/bin/../
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB