Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> how to perform GROUP BY in PIG for this case:


Copy link to this message
-
RE: how to perform GROUP BY in PIG for this case:

Thanks Russell :-),

I have build the pig in /opt/pig-0.10.0/
and in
/opt/pig-0.10.0/contrib/Piggybank/java/

and the jar files present there

and registered
to
grunt> register /opt/pig-0.10.0/contrib/piggybank/java/piggybank.jar
grunt> register /opt/pig-0.10.0/build/ivy/lib/Pig/joda-time-1.6.jar

and also defined

grunt> define CustomFormatToISO  org.apache.pig.piggybank.evaluation.datetime.convert.CustomFormatToISO()  ;
grunt> define ISOToMonth  org.apache.pig.piggybank.evaluation.datetime.convert.ISOToMonth();  

Now I performed the query on NYSE_B.

grunt> describe NYSE_B;  
                                                                                    
NYSE_B: {exchange: chararray,symbol: chararray,date: chararray,divi: float}

ans = foreach (group NYSE_B by ISOToMonth(date)) generate group as monthh, MAX(NYSE_A.divi) as max_rt;

and again got the ERROR :-(

2012-09-30 10:25:15,821 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve org.apache.pig.piggybank.evaluation.datetime.convert.ISOToMonth using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]
2012-09-30 10:25:15,822 [main] WARN  org.apache.pig.tools.grunt.Grunt - There is no log file to write to.
2012-09-30 10:25:15,822 [main] ERROR org.apache.pig.tools.grunt.Grunt - Failed to parse: Pig script failed to parse:
<line 12, column 31> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: Cannot instantiate: org.apache.pig.piggybank.evaluation.datetime.convert.ISOToMonth
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:182)
    at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1565)
    at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1538)
    at org.apache.pig.PigServer.registerQuery(PigServer.java:540)
    at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:970)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:490)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by:
<line 12, column 31> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: Cannot instantiate: org.apache.pig.piggybank.evaluation.datetime.convert.ISOToMonth
    at org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:980)
    at org.apache.pig.parser.LogicalPlanGenerator.func_eval(LogicalPlanGenerator.java:7316)
    at org.apache.pig.parser.LogicalPlanGenerator.projectable_expr(LogicalPlanGenerator.java:8857)
    at org.apache.pig.parser.LogicalPlanGenerator.var_expr(LogicalPlanGenerator.java:8632)
    at org.apache.pig.parser.LogicalPlanGenerator.expr(LogicalPlanGenerator.java:7984)
    at org.apache.pig.parser.LogicalPlanGenerator.join_group_by_expr(LogicalPlanGenerator.java:12100)
    at org.apache.pig.parser.LogicalPlanGenerator.join_group_by_clause(LogicalPlanGenerator.java:11921)
    at org.apache.pig.parser.LogicalPlanGenerator.group_item(LogicalPlanGenerator.java:5440)
    at org.apache.pig.parser.LogicalPlanGenerator.group_clause(LogicalPlanGenerator.java:5026)
    at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1313)
    at org.apache.pig.parser.LogicalPlanGenerator.inline_op(LogicalPlanGenerator.java:5739)
    at org.apache.pig.parser.LogicalPlanGenerator.rel(LogicalPlanGenerator.java:5669)
    at org.apache.pig.parser.LogicalPlanGenerator.foreach_clause(LogicalPlanGenerator.java:12350)
    at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1577)
    at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:789)
    at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:507)
    at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:382)
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:175)
    ... 15 more
Please suggest and help

Thanks & regards
Yogesh Kumar.

     
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB