Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Leap second bug


Copy link to this message
-
Re: Leap second bug
We were also caught by this - we're running CentOS 6. Likewise, once we reset the date/time, HBase was happy again. I wonder why Java processes seemed to be affected more than other processes?

On Jul 2, 2012, at 8:34 AM, Dean Banks wrote:

> We were caught by this issue.  It impacted all of our JVM processes, including the full hbase/zookeeper/HDFS stack.
>
> Before we had determined root cause, we somewhat hastily bounced some of our services (ZK, HM), and had to forcefully kill a few RSes in an attempt to get things back online.
>
> At the end of the day (with root cause addressed), I was thrilled to see that our hbase cluster came back online, split logs for 10 minutes, then passed hbck as soon as all the regions were available.  It is very impressive that hbase is capable of withstanding what amounted to a pretty nasty scenario without data loss.
>
> Nice work, guys.
>
> -Dean
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB