Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Parse Nested JSON String in Pig


Copy link to this message
-
Parse Nested JSON String in Pig
Dan Zhu 2013-07-24, 21:12
Hi pig-users,

I have tuples of nested JSON string, I am trying to parse it in Pig.

I tried com.twitter.elephantbird.pig.piggybank.JsonStringToMap() (acctDataJsonized = FOREACH acctData GENERATE com.mozilla.pig.eval.json.JsonMap($0) AS json_map:map[];) but it only parsed the first level rather than nested.

I also tried com.mozilla.pig.eval.json.JsonMap() (acctDataJsonized = FOREACH acctData GENERATE com.twitter.elephantbird.pig.piggybank.JsonStringToMap($0);) but it throws: (is it due to NULL value in the JSON data?)

ERROR 1200: Pig script failed to parse:
<line 7, column 45> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: could not instantiate 'com.mozilla.pig.eval.json.JsonMap' with arguments 'null'

Failed to parse: Pig script failed to parse:
<line 7, column 45> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: could not instantiate 'com.mozilla.pig.eval.json.JsonMap' with arguments 'null'
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)
at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1570)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1543)
at org.apache.pig.PigServer.registerQuery(PigServer.java:545)
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:970)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:495)
at org.apache.pig.Main.main(Main.java:111)
Caused by:
<line 7, column 45> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: could not instantiate 'com.mozilla.pig.eval.json.JsonMap' with arguments 'null'
at org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:980)
at org.apache.pig.parser.LogicalPlanGenerator.func_eval(LogicalPlanGenerator.java:7316)
at org.apache.pig.parser.LogicalPlanGenerator.projectable_expr(LogicalPlanGenerator.java:8857)
at org.apache.pig.parser.LogicalPlanGenerator.var_expr(LogicalPlanGenerator.java:8632)
at org.apache.pig.parser.LogicalPlanGenerator.expr(LogicalPlanGenerator.java:7984)
at org.apache.pig.parser.LogicalPlanGenerator.flatten_generated_item(LogicalPlanGenerator.java:5962)
at org.apache.pig.parser.LogicalPlanGenerator.generate_clause(LogicalPlanGenerator.java:14101)
at org.apache.pig.parser.LogicalPlanGenerator.foreach_plan(LogicalPlanGenerator.java:12493)
at org.apache.pig.parser.LogicalPlanGenerator.foreach_clause(LogicalPlanGenerator.java:12360)
at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1577)
at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:789)
at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:507)
at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:382)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:177)
... 10 more
Caused by: java.lang.RuntimeException: could not instantiate 'com.mozilla.pig.eval.json.JsonMap' with arguments 'null'
at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:554)
at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:560)
at org.apache.pig.impl.PigContext.instantiateFuncFromAlias(PigContext.java:583)
at org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:977)
... 23 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.fasterxml.jackson.databind.ObjectMapper
at com.mozilla.pig.eval.json.JsonMap.<init>(JsonMap.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at java.lang.Class.newInstance0(Class.java:374)
at java.lang.Class.newInstance(Class.java:327)
at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:524)
... 26 more

Any suggestion will be very appreciated.

Thanks & Regards,

Dan
+
Serega Sheypak 2013-07-24, 21:25
+
Harsha 2013-07-24, 21:27