Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Weird Replication exception


+
Asaf Mesika 2013-06-02, 05:20
+
shashwat shriparv 2013-06-02, 14:21
+
Ted Yu 2013-06-02, 14:46
+
Asaf Mesika 2013-06-02, 19:26
Copy link to this message
-
Re: Weird Replication exception
bq. Is 0.94.8 production ready?

I think so. Lars released 0.94.8 Friday evening.

On Sun, Jun 2, 2013 at 12:26 PM, Asaf Mesika <[EMAIL PROTECTED]> wrote:

> I use 0.94.7.
> Is 0.94.8 production ready?
>
> So in summary I have two issues:
> 1. Clocks are out of sync
> 2. I need to upgrade to 0.94.8 to avoid seeing this WARN messages?
>
> On Jun 2, 2013, at 5:46 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > What is the HBase version you're using ?
> >
> > In another thread, I mentioned this:
> >
> > There was a recently integrated JIRA (0.94.8):
> > HBASE-7122 Proper warning message when opening a log file with no entries
> > (idle cluster)
> >
> > Does the HBase you're using contain HBASE-7122 ?
> >
> > Cheers
> >
> > On Sat, Jun 1, 2013 at 10:20 PM, Asaf Mesika <[EMAIL PROTECTED]>
> wrote:
> >
> >> Hi,
> >>
> >> I have a weird error in a cluster I'm checking Replication with.
> >>
> >> I have two clusters set up, each on its own DC (different continents).
> Each
> >> has 1 master, and 3 RS.
> >>
> >> I've done all required setup, started replication and pushed in some
> data
> >> into the master. I had an issue where the slave (peer) cluster went dead
> >> (all RS failed contacting the master), thus replication couldn't work.
> This
> >> happened right before the weekend, so it was out for 3 days.
> >>
> >> Now I'm back in the office - got slave cluster back up (just the RS),
> and I
> >> got some nasty exception in one of the RS of the master cluster:
> >>
> >> 2013-06-02 04:40:45,903 INFO
> >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Getting
> >> 0 rs from peer cluster # c
> >> 2013-06-02 04:40:45,903 INFO
> >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Slave
> >> cluster looks down: c has 0 region servers
> >> 2013-06-02 04:40:46,903 DEBUG
> >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Since
> >> we are unable to replicate, sleeping 1000 times 10
> >> 2013-06-02 04:40:57,019 INFO
> >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Getting
> >> 0 rs from peer cluster # c
> >> 2013-06-02 04:40:57,019 INFO
> >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Slave
> >> cluster looks down: c has 0 region servers
> >> 2013-06-02 04:40:58,019 DEBUG
> >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Since
> >> we are unable to replicate, sleeping 1000 times 10
> >> 2013-06-02 04:41:08,134 INFO
> >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Getting
> >> 1 rs from peer cluster # c
> >> 2013-06-02 04:41:08,134 INFO
> >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> >> Choosing peer a72-246-95-86,60020,1370147274693
> >> 2013-06-02 04:41:08,672 DEBUG
> >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> >> Replicating 1
> >> 2013-06-02 04:41:08,971 INFO
> >>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
> >> Going to report log #172.25.98.74%2C60020%2C1369903540894.1369921571842
> for
> >> position
> >> 49882 in hdfs://
> >>
> >>
> 172.25.98.73:8020/hbase/.logs/172.25.98.74,60020,1369903540894/172.25.98.74%2C60020%2C1369903540894.1369921571842
> >> 2013-06-02 04:41:08,973 DEBUG
> >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> >> Replicated in total: 1
> >> 2013-06-02 04:41:08,973 DEBUG
> >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> Opening
> >> log for replication 172.25.98.74%2C60020%2C1369903540894.1369921571842
> at
> >> 49882
> >> 2013-06-02 04:41:08,977 DEBUG
> >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> >> currentNbOperations:0 and seenEntries:4 and size: 0
> >> 2013-06-02 04:41:08,977 INFO
> >>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager:
> >> Going to report log #172.25.98.74%2C60020%2C1369903540894.1369921571842
> for
> >> position
> >> 69368 in hdfs://
+
Himanshu Vashishtha 2013-06-03, 02:36
+
Asaf Mesika 2013-06-03, 05:18
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB