Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Questions about hadoop-metrics2.properties


Copy link to this message
-
Re: Questions about hadoop-metrics2.properties
Thanks, Luke. Your answer is really helpful.
On Wed, Oct 23, 2013 at 11:51 PM, Luke Lu <[EMAIL PROTECTED]> wrote:

> 1. File and Ganglia are the only bundled sinks, though there are
> socket/json (for chukwa) and graphite sinks patches in the works.
> 2. Hadoop metrics (and metrics2) is mostly designed for system/process
> metrics, which means you'll need to attach jconsole to your map/reduce task
> processes to see your task metrics instrumented via metrics. What you
> actually want is probably custom job counters.
> 3. You don't need any configuration to use JMX to access metrics2, as JMX
> is currently on by default. The configuration in hadoop-metrics2.properties
> is mostly for optional sink configuration and metrics filtering.
>
> __Luke
>
>
>
> On Wed, Oct 23, 2013 at 4:21 PM, Benyi Wang <[EMAIL PROTECTED]> wrote:
>
>> 1. Does hadoop metrics2 only support File and Ganglia sink?
>> 2. Can I expose metrics as JMX, especially for customized metrics? I
>> created some  metrics in my mapreduce job and could successfully output
>> them using a FileSink. But if I use jconsole to access YARN nodemanager, I
>> can only see hadoop metrics e.g Hadoop/NodeManager/NodeManagerMetrices
>> etc.,  not mine with prefix maptask. How to setup to see maptask/reducetask
>> prefix metrics?
>> 3. Is there an example using jmx? I could not find
>>
>> The configuration syntax is:
>>
>>   [prefix].[source|sink|jmx|].[instance].[option]
>>
>>
>> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics2/package-summary.html
>>
>
>