Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> How to check if a major_compact is done?


+
yun peng 2012-11-08, 06:08
+
ramkrishna vasudevan 2012-11-08, 06:33
+
PG 2012-11-08, 08:08
+
Jean-Marc Spaggiari 2012-11-08, 11:43
+
Jerry Lam 2012-11-08, 15:43
+
yun peng 2012-11-08, 18:31
+
Otis Gospodnetic 2012-11-08, 22:03
+
Jeremy Carroll 2012-11-08, 19:22
+
Jeremy Carroll 2012-11-08, 19:19
+
Jeremy Carroll 2012-11-08, 19:21
+
Mikael Sitruk 2012-11-08, 19:36
+
yun peng 2012-11-08, 20:58
Copy link to this message
-
Re: How to check if a major_compact is done?
Hi, I have fixed the problem. I found the
MBeanServerConnection.getAttribute(ObjectName name, String attribute) is
the one useful here. Thanks to all.
Yun

On Thu, Nov 8, 2012 at 3:58 PM, yun peng <[EMAIL PROTECTED]> wrote:

> Hi, Mikael and Jeremy, thanks for your detailed answers. I have tried the
> MBean code as an attempt. I get certain information out from the printout,
> like follows,
> ...
> javax.management.MBeanAttributeInfo[description=NoDescription,
> name=compactionTimeNumOps, type=java.lang.Integer, read-only, descriptor={}]
> ...
>
> I have a minor question, (sorry about the extra questions, but I am sort
> of newbie to web service interfaces). The descriptor (or the content) of
> MBeanAttributeInfo here is empty, which does not match with the data I can
> see through jconsole, which show compactionTimeNumOps=4 (after issuing two
> major compaction requests). Should I pause the data/content from somewhere
> other than MBeanAttributeInfo.getDescriptor().
>
> Regards,
> Yun
>
>
>
> On Thu, Nov 8, 2012 at 2:36 PM, Mikael Sitruk <[EMAIL PROTECTED]>wrote:
>
>> Yun hi
>>
>> I did something like this in the past (in 0.92.0 version). That trigger
>> major compaction and monitor their completion. I did not go through the
>> JMX
>> (which is not an indicator of major compaction), but via
>> the org.apache.hadoop.hbase.HServerLoad.RegionLoad
>>  class.
>> When an major compaction ends the number of store file will be the number
>> of store which can be checked via: regionLoad.getStorefiles() =>> task.getStores()
>>
>> I will try to find the code back and post it on github.
>>
>>
>> As for MBeans you can do somehting like:
>>
>> url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://" + hostName + ":"
>> +
>> portNum + "/jmxrmi");
>> HashMap<String, String> env = new HashMap<String, String>();
>> JMXConnector jmxc = JMXConnectorFactory.connect(url, env);
>> MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
>> ObjectName rsStat = new ObjectName(
>> "hadoop:service=RegionServer,name=RegionServerStatistics");   //put the
>> mbean you need here.
>> MBeanInfo mbeanInfo = mbsc.getMBeanInfo(rsStat);
>>
>> for (MBeanAttributeInfo mbAttrInfo : mbeanInfo.getAttributes()) {
>> System.out.println(mbAttrInfo);
>> }
>>
>>
>> Regards,
>> Mikael.S
>>
>> On Thu, Nov 8, 2012 at 8:31 PM, yun peng <[EMAIL PROTECTED]> wrote:
>>
>> > Yes, JMX exposes compaction time. My cluster has JMX enabled, and I can
>> > view certain statistics from remote jconsole.
>> >
>> > However, I dont find any reference talking about how to write a Java
>> > program to collect a specific statistics, like compaction time
>> available in
>> > region server status. Does anyone here have experience in writing
>> low-level
>> > code to collect JMX data, instead of using jconsole or other high-level
>> > tools...
>> >
>> > PS: I have found some sample code
>> > here<
>> >
>> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics/package-summary.html
>> > >,
>> > yet the API is deprecated.
>> > Regards,
>> > Yun
>> > On Thu, Nov 8, 2012 at 6:43 AM, Jean-Marc Spaggiari <
>> > [EMAIL PROTECTED]
>> > > wrote:
>> >
>> > > Please someone correct me if I'm wrong, but I think there is some
>> > > information exposed to JMX which give you the duration (and size) of
>> > > the last compaction.
>> > >
>> > > JM
>> > >
>> > > 2012/11/8, PG <[EMAIL PROTECTED]>:
>> > > > Hi, thanks for the comments. One thing is,,,, shouldn't web UI comes
>> > from
>> > > > the hbase API, or can I issue function call to get the progress of
>> > > > compaction?.
>> > > > Hun
>> > > >
>> > > > On Nov 8, 2012, at 1:33 AM, ramkrishna vasudevan
>> > > > <[EMAIL PROTECTED]> wrote:
>> > > >
>> > > >> There is no interface which says that the major compaction is
>> > completed.
>> > > >> But you can see that major compaction is in progress from the web
>> UI.
>> > > >> Sorry if am wrong here.
>> > > >>
>> > > >> Regards
>> > > >> Ram
>> > > >>
>> >
+
yun peng 2012-11-17, 01:03
+
Otis Gospodnetic 2012-11-08, 16:27