Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Re: Why does Pig not use default resources from the Configuration object?


Copy link to this message
-
Re: Why does Pig not use default resources from the Configuration object?
+User group

Hi Bhooshan,

By default you should be running in MapReduce mode unless specified
otherwise. Are you creating a PigServer object to run your jobs? Can you
provide your code here?

Sent from my iPhone

On Apr 12, 2013, at 6:23 PM, Bhooshan Mogal <[EMAIL PROTECTED]>
wrote:

Apologies for the premature send. I may have some more information. After I
applied the patch and set "pig.use.overriden.hadoop.configs=true", I saw an
NPE (stacktrace below) and a message saying pig was running in exectype
local -

2013-04-13 07:37:13,758 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to hadoop file system at: local
2013-04-13 07:37:13,760 [main] WARN  org.apache.hadoop.conf.Configuration -
mapred.used.genericoptionsparser is deprecated. Instead, use
mapreduce.client.genericoptionsparser.used
2013-04-13 07:37:14,162 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1200: Pig script failed to parse:
<file test.pig, line 1, column 4> pig script failed to validate:
java.lang.NullPointerException
Here is the stacktrace
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error
during parsing. Pig script failed to parse:
<file test.pig, line 1, column 4> pig script failed to validate:
java.lang.NullPointerException
        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1606)
        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1549)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:549)
        at
org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:971)
        at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
        at org.apache.pig.Main.run(Main.java:555)
        at org.apache.pig.Main.main(Main.java:111)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: Failed to parse: Pig script failed to parse:
<file test.pig, line 1, column 4> pig script failed to validate:
java.lang.NullPointerException
        at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)
        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1598)
        ... 14 more
Caused by:
<file test.pig, line 1, column 4> pig script failed to validate:
java.lang.NullPointerException
        at
org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:438)
        at
org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3168)
        at
org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1291)
        at
org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:789)
        at
org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:507)
        at
org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:382)
        at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:177)
        ... 15 more
On Fri, Apr 12, 2013 at 6:16 PM, Bhooshan Mogal <[EMAIL PROTECTED]>wrote:

> Yes, however I did not add core-site.xml, hdfs-site.xml, yarn-site.xml.
> Only my-filesystem-site.xml using both Configuration.addDefaultResource and
> Configuration.addResource.
>
> I see what you are saying though. The patch might require users to take
> care of adding the default config resources as well apart from their own
> resources?
>
>
> On Fri, Apr 12, 2013 at 6:06 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:

Bhooshan
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB