Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> How to check if a major_compact is done?


Copy link to this message
-
Re: How to check if a major_compact is done?
Hi, I have fixed the problem. I found the
MBeanServerConnection.getAttribute(ObjectName name, String attribute) is
the one useful here. Thanks to all.
Yun

On Thu, Nov 8, 2012 at 3:58 PM, yun peng <[EMAIL PROTECTED]> wrote:

> Hi, Mikael and Jeremy, thanks for your detailed answers. I have tried the
> MBean code as an attempt. I get certain information out from the printout,
> like follows,
> ...
> javax.management.MBeanAttributeInfo[description=NoDescription,
> name=compactionTimeNumOps, type=java.lang.Integer, read-only, descriptor={}]
> ...
>
> I have a minor question, (sorry about the extra questions, but I am sort
> of newbie to web service interfaces). The descriptor (or the content) of
> MBeanAttributeInfo here is empty, which does not match with the data I can
> see through jconsole, which show compactionTimeNumOps=4 (after issuing two
> major compaction requests). Should I pause the data/content from somewhere
> other than MBeanAttributeInfo.getDescriptor().
>
> Regards,
> Yun
>
>
>
> On Thu, Nov 8, 2012 at 2:36 PM, Mikael Sitruk <[EMAIL PROTECTED]>wrote:
>
>> Yun hi
>>
>> I did something like this in the past (in 0.92.0 version). That trigger
>> major compaction and monitor their completion. I did not go through the
>> JMX
>> (which is not an indicator of major compaction), but via
>> the org.apache.hadoop.hbase.HServerLoad.RegionLoad
>>  class.
>> When an major compaction ends the number of store file will be the number
>> of store which can be checked via: regionLoad.getStorefiles() =>> task.getStores()
>>
>> I will try to find the code back and post it on github.
>>
>>
>> As for MBeans you can do somehting like:
>>
>> url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://" + hostName + ":"
>> +
>> portNum + "/jmxrmi");
>> HashMap<String, String> env = new HashMap<String, String>();
>> JMXConnector jmxc = JMXConnectorFactory.connect(url, env);
>> MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
>> ObjectName rsStat = new ObjectName(
>> "hadoop:service=RegionServer,name=RegionServerStatistics");   //put the
>> mbean you need here.
>> MBeanInfo mbeanInfo = mbsc.getMBeanInfo(rsStat);
>>
>> for (MBeanAttributeInfo mbAttrInfo : mbeanInfo.getAttributes()) {
>> System.out.println(mbAttrInfo);
>> }
>>
>>
>> Regards,
>> Mikael.S
>>
>> On Thu, Nov 8, 2012 at 8:31 PM, yun peng <[EMAIL PROTECTED]> wrote:
>>
>> > Yes, JMX exposes compaction time. My cluster has JMX enabled, and I can
>> > view certain statistics from remote jconsole.
>> >
>> > However, I dont find any reference talking about how to write a Java
>> > program to collect a specific statistics, like compaction time
>> available in
>> > region server status. Does anyone here have experience in writing
>> low-level
>> > code to collect JMX data, instead of using jconsole or other high-level
>> > tools...
>> >
>> > PS: I have found some sample code
>> > here<
>> >
>> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics/package-summary.html
>> > >,
>> > yet the API is deprecated.
>> > Regards,
>> > Yun
>> > On Thu, Nov 8, 2012 at 6:43 AM, Jean-Marc Spaggiari <
>> > [EMAIL PROTECTED]
>> > > wrote:
>> >
>> > > Please someone correct me if I'm wrong, but I think there is some
>> > > information exposed to JMX which give you the duration (and size) of
>> > > the last compaction.
>> > >
>> > > JM
>> > >
>> > > 2012/11/8, PG <[EMAIL PROTECTED]>:
>> > > > Hi, thanks for the comments. One thing is,,,, shouldn't web UI comes
>> > from
>> > > > the hbase API, or can I issue function call to get the progress of
>> > > > compaction?.
>> > > > Hun
>> > > >
>> > > > On Nov 8, 2012, at 1:33 AM, ramkrishna vasudevan
>> > > > <[EMAIL PROTECTED]> wrote:
>> > > >
>> > > >> There is no interface which says that the major compaction is
>> > completed.
>> > > >> But you can see that major compaction is in progress from the web
>> UI.
>> > > >> Sorry if am wrong here.
>> > > >>
>> > > >> Regards
>> > > >> Ram
>> > > >>
>> >
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB