Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - how to figure out the range of a split that failed?


Copy link to this message
-
Re: how to figure out the range of a split that failed?
edward choi 2010-07-05, 04:30
Thanks for the response. I went to the web page you told me and several
other pages that I found.
I am still not sure if I got it right.
If I am trying to increment COUNTER_MAP_PROCESS_RECORDS using Hadoop
Streaming, is the example below the way to do it? (assuming that I am using
c++)

example:
cerr << "reporter:counter:counters,linecount,1" << endl;

I am not sure how to use "reporter:counter:<group>,<counter>,<amount>".
I wrote the example by referring to "
https://issues.apache.org/jira/browse/HADOOP-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
"
Am I doing it right?

2010/7/1 Sharad Agarwal <[EMAIL PROTECTED]>

> edward choi wrote:
>
>>
>>
>> Then I noticed that you have to set
>> "mapred.skip.map.auto.incr.proc.count=false" and increment
>> COUNTER_MAP_PROCESSED_RECORDS in your own application. I guess that you can
>> do this in your example, but I don't know how to do it using my way of
>> Hadoop Streaming. Could you enlighten me please?
>>
>>  You can refer
>
> http://hadoop.apache.org/common/docs/r0.20.0/streaming.html#How+do+I+update+counters+in+streaming+applications%3F
> for knowing how to increment counter in streaming.
>
>
> Sharad
>