Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Parse Nested JSON String in Pig


Copy link to this message
-
Parse Nested JSON String in Pig
Hi pig-users,

I have tuples of nested JSON string, I am trying to parse it in Pig.

I tried com.twitter.elephantbird.pig.piggybank.JsonStringToMap() (acctDataJsonized = FOREACH acctData GENERATE com.mozilla.pig.eval.json.JsonMap($0) AS json_map:map[];) but it only parsed the first level rather than nested.

I also tried com.mozilla.pig.eval.json.JsonMap() (acctDataJsonized = FOREACH acctData GENERATE com.twitter.elephantbird.pig.piggybank.JsonStringToMap($0);) but it throws: (is it due to NULL value in the JSON data?)

ERROR 1200: Pig script failed to parse:
<line 7, column 45> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: could not instantiate 'com.mozilla.pig.eval.json.JsonMap' with arguments 'null'

Failed to parse: Pig script failed to parse:
<line 7, column 45> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: could not instantiate 'com.mozilla.pig.eval.json.JsonMap' with arguments 'null'
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)
at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1570)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1543)
at org.apache.pig.PigServer.registerQuery(PigServer.java:545)
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:970)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:495)
at org.apache.pig.Main.main(Main.java:111)
Caused by:
<line 7, column 45> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: could not instantiate 'com.mozilla.pig.eval.json.JsonMap' with arguments 'null'
at org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:980)
at org.apache.pig.parser.LogicalPlanGenerator.func_eval(LogicalPlanGenerator.java:7316)
at org.apache.pig.parser.LogicalPlanGenerator.projectable_expr(LogicalPlanGenerator.java:8857)
at org.apache.pig.parser.LogicalPlanGenerator.var_expr(LogicalPlanGenerator.java:8632)
at org.apache.pig.parser.LogicalPlanGenerator.expr(LogicalPlanGenerator.java:7984)
at org.apache.pig.parser.LogicalPlanGenerator.flatten_generated_item(LogicalPlanGenerator.java:5962)
at org.apache.pig.parser.LogicalPlanGenerator.generate_clause(LogicalPlanGenerator.java:14101)
at org.apache.pig.parser.LogicalPlanGenerator.foreach_plan(LogicalPlanGenerator.java:12493)
at org.apache.pig.parser.LogicalPlanGenerator.foreach_clause(LogicalPlanGenerator.java:12360)
at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1577)
at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:789)
at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:507)
at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:382)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:177)
... 10 more
Caused by: java.lang.RuntimeException: could not instantiate 'com.mozilla.pig.eval.json.JsonMap' with arguments 'null'
at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:554)
at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:560)
at org.apache.pig.impl.PigContext.instantiateFuncFromAlias(PigContext.java:583)
at org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:977)
... 23 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.fasterxml.jackson.databind.ObjectMapper
at com.mozilla.pig.eval.json.JsonMap.<init>(JsonMap.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at java.lang.Class.newInstance0(Class.java:374)
at java.lang.Class.newInstance(Class.java:327)
at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:524)
... 26 more

Any suggestion will be very appreciated.

Thanks & Regards,

Dan
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB