Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: Questions about hadoop-metrics2.properties


Copy link to this message
-
Re: Questions about hadoop-metrics2.properties
Thanks, Luke. Your answer is really helpful.
On Wed, Oct 23, 2013 at 11:51 PM, Luke Lu <[EMAIL PROTECTED]> wrote:

> 1. File and Ganglia are the only bundled sinks, though there are
> socket/json (for chukwa) and graphite sinks patches in the works.
> 2. Hadoop metrics (and metrics2) is mostly designed for system/process
> metrics, which means you'll need to attach jconsole to your map/reduce task
> processes to see your task metrics instrumented via metrics. What you
> actually want is probably custom job counters.
> 3. You don't need any configuration to use JMX to access metrics2, as JMX
> is currently on by default. The configuration in hadoop-metrics2.properties
> is mostly for optional sink configuration and metrics filtering.
>
> __Luke
>
>
>
> On Wed, Oct 23, 2013 at 4:21 PM, Benyi Wang <[EMAIL PROTECTED]> wrote:
>
>> 1. Does hadoop metrics2 only support File and Ganglia sink?
>> 2. Can I expose metrics as JMX, especially for customized metrics? I
>> created some  metrics in my mapreduce job and could successfully output
>> them using a FileSink. But if I use jconsole to access YARN nodemanager, I
>> can only see hadoop metrics e.g Hadoop/NodeManager/NodeManagerMetrices
>> etc.,  not mine with prefix maptask. How to setup to see maptask/reducetask
>> prefix metrics?
>> 3. Is there an example using jmx? I could not find
>>
>> The configuration syntax is:
>>
>>   [prefix].[source|sink|jmx|].[instance].[option]
>>
>>
>> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics2/package-summary.html
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB