Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - LIMIT operator doesn't work with variables


Copy link to this message
-
LIMIT operator doesn't work with variables
James Newhaven 2012-04-10, 20:33
Hi,

I am trying to a limit the output size using LIMIT. I want to the limit
size to be 5 percent of the total output size like this:

-- Put all the inids in a bag so we can count them.
G = GROUP F ALL;

-- Count everything in the bag
H =  FOREACH G GENERATE COUNT_STAR(F) AS total;

-- Limit out to 5 percent of the total.
I = LIMIT F H.total * 0.05;

However, when I define the last line with the LIMIT operator, I get the
following error. Does anyone know a way around this?

ERROR 1200: <line 362, column 12>  mismatched input 'H' expecting set null

Failed to parse: <line 362, column 12>  mismatched input 'H' expecting set
null
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222)
Thanks,
James