Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> increment counters in Pig UDF


Copy link to this message
-
Re: increment counters in Pig UDF
Perfect. Thanks.
On Dec 16, 2010, at 7:41 AM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> We can do better in 0.8:
>
>
>    PigStatusReporter reporter = PigStatusReporter.getInstance();
>
>    if (reporter != null) {
>
>       reporter.getCounter(myCounterEnum).increment(123L);
>
>    }
>
>
>   getCounter works with both enums and Strings.
>
> -D
>
> On Wed, Dec 15, 2010 at 5:29 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:
>
>> Yes, you can use EvalFunc.warn(Object o, String msg, Enum warningEnum).
>>
>> Daniel
>>
>>
>> Dexin Wang wrote:
>>
>>> Is it possible to increment a counter in Pig UDF (in either
>>> Load/Eval/Store
>>> Func).
>>>
>>> Since we have access to counters using the
>>> org.apache.hadoop.mapred.Reporter:
>>>
>>> http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutorial.html#Counters
>>>
>>> the other way to ask this question is how do we get an instance of
>>> Reporter
>>> in UDF? Thanks.
>>>
>>> Dexin
>>>
>>>
>>
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB