Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Re: Why does Pig not use default resources from the Configuration object?


Copy link to this message
-
Re: Why does Pig not use default resources from the Configuration object?
Prashant Kommireddi 2013-04-13, 04:57
+User group

Hi Bhooshan,

By default you should be running in MapReduce mode unless specified
otherwise. Are you creating a PigServer object to run your jobs? Can you
provide your code here?

Sent from my iPhone

On Apr 12, 2013, at 6:23 PM, Bhooshan Mogal <[EMAIL PROTECTED]>
wrote:

Apologies for the premature send. I may have some more information. After I
applied the patch and set "pig.use.overriden.hadoop.configs=true", I saw an
NPE (stacktrace below) and a message saying pig was running in exectype
local -

2013-04-13 07:37:13,758 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to hadoop file system at: local
2013-04-13 07:37:13,760 [main] WARN  org.apache.hadoop.conf.Configuration -
mapred.used.genericoptionsparser is deprecated. Instead, use
mapreduce.client.genericoptionsparser.used
2013-04-13 07:37:14,162 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1200: Pig script failed to parse:
<file test.pig, line 1, column 4> pig script failed to validate:
java.lang.NullPointerException
Here is the stacktrace
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error
during parsing. Pig script failed to parse:
<file test.pig, line 1, column 4> pig script failed to validate:
java.lang.NullPointerException
        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1606)
        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1549)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:549)
        at
org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:971)
        at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:190)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
        at org.apache.pig.Main.run(Main.java:555)
        at org.apache.pig.Main.main(Main.java:111)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: Failed to parse: Pig script failed to parse:
<file test.pig, line 1, column 4> pig script failed to validate:
java.lang.NullPointerException
        at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)
        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1598)
        ... 14 more
Caused by:
<file test.pig, line 1, column 4> pig script failed to validate:
java.lang.NullPointerException
        at
org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:438)
        at
org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3168)
        at
org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1291)
        at
org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:789)
        at
org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:507)
        at
org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:382)
        at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:177)
        ... 15 more
On Fri, Apr 12, 2013 at 6:16 PM, Bhooshan Mogal <[EMAIL PROTECTED]>wrote:

> Yes, however I did not add core-site.xml, hdfs-site.xml, yarn-site.xml.
> Only my-filesystem-site.xml using both Configuration.addDefaultResource and
> Configuration.addResource.
>
> I see what you are saying though. The patch might require users to take
> care of adding the default config resources as well apart from their own
> resources?
>
>
> On Fri, Apr 12, 2013 at 6:06 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:

Bhooshan
+
Bhooshan Mogal 2013-04-15, 17:35
+
Prashant Kommireddi 2013-04-15, 18:58
+
Bhooshan Mogal 2013-04-15, 23:37
+
Prashant Kommireddi 2013-04-15, 23:43
+
Bhooshan Mogal 2013-04-16, 00:34
+
Prashant Kommireddi 2013-04-16, 00:57
+
Bhooshan Mogal 2013-05-29, 23:55
+
Bhooshan Mogal 2013-06-21, 03:31