Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Run script with org.apache.pig.builtin macros


Copy link to this message
-
Re: Run script with org.apache.pig.builtin macros
Cheolsoo Park 2013-04-09, 16:15
Hi,

SUM is not a macro but a built-in UDF. Please try:

log = LOAD '/tmp/data' AS (value:long);
total = FOREACH (GROUP log ALL) GENERATE SUM(log.value);
DUMP total;

Please also see: http://pig.apache.org/docs/r0.11.0/func.html#sum

Thanks,
Cheolsoo

On Sat, Apr 6, 2013 at 1:08 PM, raanan nitzan <[EMAIL PROTECTED]>wrote:

> Hi
>
> I'm  trying to run pig script with builtin macros like count/sum/avg ...
>  (org.apache.pig.builtin)
> But on every method I trued I get the same ERROR
>
>  ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: <file /tmp/test.pig,
> line 3> Cannot expand macro 'SUM'. Reason: Macro must be defined before
> expansion
>
> I understood that macro cant be used from grunt (
> http://www.mail-archive.com/[EMAIL PROTECTED]/msg04926.html)
> So how can I use the org.apache.pig.builtin macro's
> I tried to run script using following methods
>
>    - Command line : pig [script file path]
>    - PigRunner :  java MyPigRunner.java [script file path]
>    - PigServer :  java MyPigServer.java [script file path]
>
>
> But for all I get the expension error, what is the way to run pig script
> with builtin macros
>
> I'm using followng pig pacakge : pig-0.10.0+61-1.cdh4.1.3.p0.23.el6.noarch
>
> *Pig script:*
> *log = LOAD '/tmp/data' AS (value:long) ;*
> *total = SUM(log);*
> *DUMP log;*
> *
> *
>
> PigRunner.java:
>
> *public class MyPigRunner {*
> * *
> *
> *
> *static public void main(String [] args) {*
> * *
> * org.apache.pig.PigRunner.run(args, null);*
> * *
> *}*
> *
> *
> *}*
>
> PigServer.java:
> *public class MyPigServer {*
> * *
> *
> *
> *static public void main (String[] args) throws IOException{*
> * *
> * String filename = args[0];*
> * PigServer pig = new PigServer(ExecType.LOCAL); *
> * pig.registerScript(filename);*
> * pig.store("Result", "/tmp/pig.result");*
> * *
> *}*
> *
> *
> *}*
>
>
> *Details at logfile: /usr/lib/pig/bin/pig_1365300225744.log
> 0_31/lib/tools.jar:/usr/lib/hbase/bin/..:/usr/lib/hbase/
> bin/../hbase-0.92.1-cdh4.1.3-security.jar:/usr/lib/hbase/
> bin/../hbase-0.92.1-cdh4.1.3-security-tests.jar:/usr/lib/
> hbase/bin/../hbase.jar:/usr/lib/hbase/bin/../lib/
> activation-1.1.jar:/usr/lib/hbase/bin/../lib/aopalliance-
> 1.0.jar:/usr/lib/hbase/bin/../lib/asm-3.2.jar:/usr/lib/
> hbase/bin/../lib/avro-1.7.1.cloudera.2.jar:/usr/lib/hbase/
> bin/../lib/commons-beanutils-1.7.0.jar:/usr/lib/hbase/bin/.
> ./lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hbase/bin/.
> ./lib/commons-cli-1.2.jar:/usr/lib/hbase/bin/../lib/
> commons-codec-1.4.jar:/usr/lib/hbase/bin/../lib/commons-
> collections-3.2.1.jar:/usr/lib/hbase/bin/../lib/commons-
> configuration-1.6.jar:/usr/lib/hbase/bin/../lib/commons-
> daemon-1.0.3.jar:/usr/lib/hbase/bin/../lib/commons-
> digester-1.8.jar:/usr/lib/hbase/bin/../lib/commons-el-1.
> 0.jar:/usr/lib/hbase/bin/../lib/commons-httpclient-3.1.
> jar:/usr/lib/hbase/bin/../lib/commons-io-2.1.jar:/usr/lib/
> hbase/bin/../lib/commons-lang-2.5.jar:/usr/lib/hbase/bin/../
> lib/commons-logging-1.1.1.jar:/usr/lib/hbase/bin/../lib/
> commons-net-3.1.jar:/usr/lib/hbase/bin/../lib/core-3.1.1.
> jar:/usr/lib/hbase/bin/../lib/gmbal-api-only-3.0.0-b023.jar:
> /usr/lib/hbase/bin/../lib/grizzly-framework-2.1.1.jar:/
> usr/lib/hbase/bin/../lib/grizzly-framework-2.1.1-tests.
> jar:/usr/lib/hbase/bin/../lib/grizzly-http-2.1.1.jar:/usr/
> lib/hbase/bin/../lib/grizzly-http-server-2.1.1.jar:/usr/
> lib/hbase/bin/../lib/grizzly-http-servlet-2.1.1.jar:/usr/
> lib/hbase/bin/../lib/grizzly-rcm-2.1.1.jar:/usr/lib/hbase/
> bin/../lib/guava-11.0.2.jar:/usr/lib/hbase/bin/../lib/
> guice-3.0.jar:/usr/lib/hbase/bin/../lib/guice-servlet-3.0.
> jar:/usr/lib/hbase/bin/../lib/high-scale-lib-1.1.1.jar:/usr/
> lib/hbase/bin/../lib/httpclient-4.0.1.jar:/usr/lib/
> hbase/bin/../lib/httpcore-4.0.1.jar:/usr/lib/hbase/bin/../
> lib/jackson-core-asl-1.8.8.jar:/usr/lib/hbase/bin/../lib/
> jackson-jaxrs-1.8.8.jar:/usr/lib/hbase/bin/../lib/jackson-
> mapper-asl-1.8.8.jar:/usr/lib/hbase/bin/../lib/jackson-xc-1.