Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - how to perform GROUP BY in PIG for this case:


Copy link to this message
-
RE: how to perform GROUP BY in PIG for this case:
yogesh dhari 2012-09-30, 04:58

Thanks Russell :-),

I have build the pig in /opt/pig-0.10.0/
and in
/opt/pig-0.10.0/contrib/Piggybank/java/

and the jar files present there

and registered
to
grunt> register /opt/pig-0.10.0/contrib/piggybank/java/piggybank.jar
grunt> register /opt/pig-0.10.0/build/ivy/lib/Pig/joda-time-1.6.jar

and also defined

grunt> define CustomFormatToISO  org.apache.pig.piggybank.evaluation.datetime.convert.CustomFormatToISO()  ;
grunt> define ISOToMonth  org.apache.pig.piggybank.evaluation.datetime.convert.ISOToMonth();  

Now I performed the query on NYSE_B.

grunt> describe NYSE_B;  
                                                                                    
NYSE_B: {exchange: chararray,symbol: chararray,date: chararray,divi: float}

ans = foreach (group NYSE_B by ISOToMonth(date)) generate group as monthh, MAX(NYSE_A.divi) as max_rt;

and again got the ERROR :-(

2012-09-30 10:25:15,821 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve org.apache.pig.piggybank.evaluation.datetime.convert.ISOToMonth using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]
2012-09-30 10:25:15,822 [main] WARN  org.apache.pig.tools.grunt.Grunt - There is no log file to write to.
2012-09-30 10:25:15,822 [main] ERROR org.apache.pig.tools.grunt.Grunt - Failed to parse: Pig script failed to parse:
<line 12, column 31> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: Cannot instantiate: org.apache.pig.piggybank.evaluation.datetime.convert.ISOToMonth
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:182)
    at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1565)
    at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1538)
    at org.apache.pig.PigServer.registerQuery(PigServer.java:540)
    at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:970)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:490)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by:
<line 12, column 31> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: Cannot instantiate: org.apache.pig.piggybank.evaluation.datetime.convert.ISOToMonth
    at org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:980)
    at org.apache.pig.parser.LogicalPlanGenerator.func_eval(LogicalPlanGenerator.java:7316)
    at org.apache.pig.parser.LogicalPlanGenerator.projectable_expr(LogicalPlanGenerator.java:8857)
    at org.apache.pig.parser.LogicalPlanGenerator.var_expr(LogicalPlanGenerator.java:8632)
    at org.apache.pig.parser.LogicalPlanGenerator.expr(LogicalPlanGenerator.java:7984)
    at org.apache.pig.parser.LogicalPlanGenerator.join_group_by_expr(LogicalPlanGenerator.java:12100)
    at org.apache.pig.parser.LogicalPlanGenerator.join_group_by_clause(LogicalPlanGenerator.java:11921)
    at org.apache.pig.parser.LogicalPlanGenerator.group_item(LogicalPlanGenerator.java:5440)
    at org.apache.pig.parser.LogicalPlanGenerator.group_clause(LogicalPlanGenerator.java:5026)
    at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1313)
    at org.apache.pig.parser.LogicalPlanGenerator.inline_op(LogicalPlanGenerator.java:5739)
    at org.apache.pig.parser.LogicalPlanGenerator.rel(LogicalPlanGenerator.java:5669)
    at org.apache.pig.parser.LogicalPlanGenerator.foreach_clause(LogicalPlanGenerator.java:12350)
    at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1577)
    at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:789)
    at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:507)
    at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:382)
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:175)
    ... 15 more
Please suggest and help

Thanks & regards
Yogesh Kumar.