Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Re: count total number of tuples in a bag?


Copy link to this message
-
Re: count total number of tuples in a bag?
Ruslan Al-Fakikh 2013-06-25, 09:51
Hi!

What are you trying to do with define c COV('a','b','c') exactly?
Can you try
out = foreach grp generate group, COV(A.$0,A.$1,A.$2);
without the define statement?

Ruslan Al-Fakikh
On Tue, Jun 18, 2013 at 1:17 PM, achile wandji <[EMAIL PROTECTED]>wrote:

> Hi,
> I' trying to compute a correlation with the script above (Pig Latin)
>
> A = load 'input.txt' using PigStorage(':');
> grp = group A all;
> define c COV('a','b','c');
> out = foreach grp generate group, c(A.$0,A.$1,A.$2);
>
> the data sets is
> 1 2 3
> 4 5 6
>
>
> But i get those messages
>
> 2013-06-18 01:19:58,580 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
> File concatenation threshold: 100 optimistic? false
> 2013-06-18 01:19:58,593 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer
> - Choosing to move algebraic foreach to combiner
> 2013-06-18 01:19:58,606 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 2018: Internal error. Unable to introduce the combiner for
> optimization.
>
>
> Someby could help me, please?
>