Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Run script with org.apache.pig.builtin macros


Copy link to this message
-
Re: Run script with org.apache.pig.builtin macros
Hi,

SUM is not a macro but a built-in UDF. Please try:

log = LOAD '/tmp/data' AS (value:long);
total = FOREACH (GROUP log ALL) GENERATE SUM(log.value);
DUMP total;

Please also see: http://pig.apache.org/docs/r0.11.0/func.html#sum

Thanks,
Cheolsoo

On Sat, Apr 6, 2013 at 1:08 PM, raanan nitzan <[EMAIL PROTECTED]>wrote:

> Hi
>
> I'm  trying to run pig script with builtin macros like count/sum/avg ...
>  (org.apache.pig.builtin)
> But on every method I trued I get the same ERROR
>
>  ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: <file /tmp/test.pig,
> line 3> Cannot expand macro 'SUM'. Reason: Macro must be defined before
> expansion
>
> I understood that macro cant be used from grunt (
> http://www.mail-archive.com/[EMAIL PROTECTED]/msg04926.html)
> So how can I use the org.apache.pig.builtin macro's
> I tried to run script using following methods
>
>    - Command line : pig [script file path]
>    - PigRunner :  java MyPigRunner.java [script file path]
>    - PigServer :  java MyPigServer.java [script file path]
>
>
> But for all I get the expension error, what is the way to run pig script
> with builtin macros
>
> I'm using followng pig pacakge : pig-0.10.0+61-1.cdh4.1.3.p0.23.el6.noarch
>
> *Pig script:*
> *log = LOAD '/tmp/data' AS (value:long) ;*
> *total = SUM(log);*
> *DUMP log;*
> *
> *
>
> PigRunner.java:
>
> *public class MyPigRunner {*
> * *
> *
> *
> *static public void main(String [] args) {*
> * *
> * org.apache.pig.PigRunner.run(args, null);*
> * *
> *}*
> *
> *
> *}*
>
> PigServer.java:
> *public class MyPigServer {*
> * *
> *
> *
> *static public void main (String[] args) throws IOException{*
> * *
> * String filename = args[0];*
> * PigServer pig = new PigServer(ExecType.LOCAL); *
> * pig.registerScript(filename);*
> * pig.store("Result", "/tmp/pig.result");*
> * *
> *}*
> *
> *
> *}*
>
>
> *Details at logfile: /usr/lib/pig/bin/pig_1365300225744.log
> 0_31/lib/tools.jar:/usr/lib/hbase/bin/..:/usr/lib/hbase/
> bin/../hbase-0.92.1-cdh4.1.3-security.jar:/usr/lib/hbase/
> bin/../hbase-0.92.1-cdh4.1.3-security-tests.jar:/usr/lib/
> hbase/bin/../hbase.jar:/usr/lib/hbase/bin/../lib/
> activation-1.1.jar:/usr/lib/hbase/bin/../lib/aopalliance-
> 1.0.jar:/usr/lib/hbase/bin/../lib/asm-3.2.jar:/usr/lib/
> hbase/bin/../lib/avro-1.7.1.cloudera.2.jar:/usr/lib/hbase/
> bin/../lib/commons-beanutils-1.7.0.jar:/usr/lib/hbase/bin/.
> ./lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hbase/bin/.
> ./lib/commons-cli-1.2.jar:/usr/lib/hbase/bin/../lib/
> commons-codec-1.4.jar:/usr/lib/hbase/bin/../lib/commons-
> collections-3.2.1.jar:/usr/lib/hbase/bin/../lib/commons-
> configuration-1.6.jar:/usr/lib/hbase/bin/../lib/commons-
> daemon-1.0.3.jar:/usr/lib/hbase/bin/../lib/commons-
> digester-1.8.jar:/usr/lib/hbase/bin/../lib/commons-el-1.
> 0.jar:/usr/lib/hbase/bin/../lib/commons-httpclient-3.1.
> jar:/usr/lib/hbase/bin/../lib/commons-io-2.1.jar:/usr/lib/
> hbase/bin/../lib/commons-lang-2.5.jar:/usr/lib/hbase/bin/../
> lib/commons-logging-1.1.1.jar:/usr/lib/hbase/bin/../lib/
> commons-net-3.1.jar:/usr/lib/hbase/bin/../lib/core-3.1.1.
> jar:/usr/lib/hbase/bin/../lib/gmbal-api-only-3.0.0-b023.jar:
> /usr/lib/hbase/bin/../lib/grizzly-framework-2.1.1.jar:/
> usr/lib/hbase/bin/../lib/grizzly-framework-2.1.1-tests.
> jar:/usr/lib/hbase/bin/../lib/grizzly-http-2.1.1.jar:/usr/
> lib/hbase/bin/../lib/grizzly-http-server-2.1.1.jar:/usr/
> lib/hbase/bin/../lib/grizzly-http-servlet-2.1.1.jar:/usr/
> lib/hbase/bin/../lib/grizzly-rcm-2.1.1.jar:/usr/lib/hbase/
> bin/../lib/guava-11.0.2.jar:/usr/lib/hbase/bin/../lib/
> guice-3.0.jar:/usr/lib/hbase/bin/../lib/guice-servlet-3.0.
> jar:/usr/lib/hbase/bin/../lib/high-scale-lib-1.1.1.jar:/usr/
> lib/hbase/bin/../lib/httpclient-4.0.1.jar:/usr/lib/
> hbase/bin/../lib/httpcore-4.0.1.jar:/usr/lib/hbase/bin/../
> lib/jackson-core-asl-1.8.8.jar:/usr/lib/hbase/bin/../lib/
> jackson-jaxrs-1.8.8.jar:/usr/lib/hbase/bin/../lib/jackson-
> mapper-asl-1.8.8.jar:/usr/lib/hbase/bin/../lib/jackson-xc-1.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB