Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Trying to get pig 0.11/0.12 working to solve 0.10's issues with python udf


Copy link to this message
-
Trying to get pig 0.11/0.12 working to solve 0.10's issues with python udf
I am trying to use pig 0.11 and pig trunk (currently 0.12) because pig 0.10
seems to be having issues with python udf...

According to this
http://www.mail-archive.com/[EMAIL PROTECTED]/msg05837.html

" after replacing pig.jar and pig-withouthadoop.jar with the
0.11 ones from the svn trunk, they work like a charm."

Well this is clearly not the case for me...

The error I get is:

Pig Stack Trace
---------------
ERROR 2017: Internal error creating job configuration.

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
open iterator for alias ll
 at org.apache.pig.PigServer.openIterator(PigServer.java:841)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696)
 at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
 at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
 at org.apache.pig.Main.run(Main.java:535)
at org.apache.pig.Main.main(Main.java:154)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias ll
at org.apache.pig.PigServer.storeEx(PigServer.java:940)
 at org.apache.pig.PigServer.store(PigServer.java:903)
at org.apache.pig.PigServer.openIterator(PigServer.java:816)
 ... 12 more
Caused by:
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException:
ERROR 2017: Internal error creating job configuration.
 at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:848)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:294)
 at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:177)
at org.apache.pig.PigServer.launchPlan(PigServer.java:1269)
 at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1254)
at org.apache.pig.PigServer.storeEx(PigServer.java:936)
 ... 14 more
Caused by: java.lang.IllegalArgumentException: Can not create a Path from
an empty string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
 at org.apache.hadoop.fs.Path.<init>(Path.java:90)
at org.apache.hadoop.fs.Path.<init>(Path.java:45)
 at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.shipToHDFS(JobControlCompiler.java:1455)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.putJarOnClassPathThroughDistributedCache(JobControlCompiler.java:1432)
 at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:508)
... 19 more
===============================================================================
I am using the following startup script:

export HADOOP_HOME=/usr/lib/hadoop-0.20
export HCAT_HOME=/opt/hcat
export HIVE_HOME=/usr/lib/hive

PIG_CLASSPATH=$HCAT_HOME/share/hcatalog/hcatalog-0.4.0.jar::$HIVE_HOME/conf:$HADOOP_HOME/conf

for file in $HIVE_HOME/lib/*.jar; do
    echo "==> Adding $file"
    PIG_CLASSPATH=$PIG_CLASSPATH:$file
done

export PIG_OPTS=-Dhive.metastore.uris=thrift://
appserver.hadoop.staging.qutics.com:10002
export JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::")
exec bin/pig -Dpig.additional.jars=$PIG_CLASSPATH "$@"

Any clues?
Thank you!
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB