Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Pig counters and UDF_WARNING_1

Josh Devins 2010-11-10, 15:18
Copy link to this message
Re: Pig counters and UDF_WARNING_1
Thejas M Nair 2010-11-10, 20:56
Try running pig with -warning option (preferably on sample of the data), that will turn off warning aggregation and give more specific warning messages.
When I grep'd for this in the source, I found two udfs using this -

src/org/apache/pig/builtin/REGEX_EXTRACT.java:        warn("RegexExtract : Cannot extract group for input "+input.get(0), PigWarning.UDF_WARNING_1);
src/org/apache/pig/builtin/UPPER.java:                    input.get(0).getClass()+" to String", PigWarning.UDF_WARNING_1);
src/org/apache/pig/builtin/UPPER.java:            warn("Error processing input "+input.get(0), PigWarning.UDF_WARNING_1);

On 11/10/10 7:18 AM, "Josh Devins" <[EMAIL PROTECTED]> wrote:

I just ran a Pig job and for the first time noticed the output at the end of
the job (and of course a matching counter):

Encountered Warning UDF_WARNING_1 108939522 time(s)

What exactly does this count refer to and is there any way to find out what
the actual warning is about? I've checked the job logs and could not see any
details. Considering the size of the counter, it would be good to know what
this refers to.