Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - increment counters in Pig UDF


Copy link to this message
-
Re: increment counters in Pig UDF
Dexin Wang 2010-12-16, 17:07
Perfect. Thanks.
On Dec 16, 2010, at 7:41 AM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> We can do better in 0.8:
>
>
>    PigStatusReporter reporter = PigStatusReporter.getInstance();
>
>    if (reporter != null) {
>
>       reporter.getCounter(myCounterEnum).increment(123L);
>
>    }
>
>
>   getCounter works with both enums and Strings.
>
> -D
>
> On Wed, Dec 15, 2010 at 5:29 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:
>
>> Yes, you can use EvalFunc.warn(Object o, String msg, Enum warningEnum).
>>
>> Daniel
>>
>>
>> Dexin Wang wrote:
>>
>>> Is it possible to increment a counter in Pig UDF (in either
>>> Load/Eval/Store
>>> Func).
>>>
>>> Since we have access to counters using the
>>> org.apache.hadoop.mapred.Reporter:
>>>
>>> http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html#Counters
>>>
>>> the other way to ask this question is how do we get an instance of
>>> Reporter
>>> in UDF? Thanks.
>>>
>>> Dexin
>>>
>>>
>>
>>