Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Weird Replication exception


+
Asaf Mesika 2013-06-02, 05:20
+
shashwat shriparv 2013-06-02, 14:21
+
Ted Yu 2013-06-02, 14:46
+
Asaf Mesika 2013-06-02, 19:26
+
Ted Yu 2013-06-02, 19:38
+
Himanshu Vashishtha 2013-06-03, 02:36
Copy link to this message
-
Re: Weird Replication exception
No, this was brand new with 0 length thus the peculiar message of too old
was strange to me.

On Monday, June 3, 2013, Himanshu Vashishtha wrote:

> Hey Asaf,
>
> It looks like you only need 7122. Either upgrade, or you could also patch
> it up.
>
> Syncing up the master and slave cluster is also advised, but that stands
> good in case you are using master-master replication.
>
>
> bq. 172.25.98.74,60020,
> 1369903540894/172.25.98.74
> %2C60020%2C1369903540894.1369925171871
>
> bq.    Meaning I lost data
>
> Did the log whose znode you deleted had any data?
> You can do a cat to see if there is any data on it. You could copy-table
> for that time range, (or a hacky way is to re-create a znode for that log
> under a regionserver noticing the format of its current log znodes, and let
> the replicationSource pick it up in its normal run).
>
> Thanks,
> Himanshu
>
>
>
> On Sun, Jun 2, 2013 at 12:38 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > bq. Is 0.94.8 production ready?
> >
> > I think so. Lars released 0.94.8 Friday evening.
> >
> > On Sun, Jun 2, 2013 at 12:26 PM, Asaf Mesika <[EMAIL PROTECTED]>
> > wrote:
> >
> > > I use 0.94.7.
> > > Is 0.94.8 production ready?
> > >
> > > So in summary I have two issues:
> > > 1. Clocks are out of sync
> > > 2. I need to upgrade to 0.94.8 to avoid seeing this WARN messages?
> > >
> > > On Jun 2, 2013, at 5:46 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> > >
> > > > What is the HBase version you're using ?
> > > >
> > > > In another thread, I mentioned this:
> > > >
> > > > There was a recently integrated JIRA (0.94.8):
> > > > HBASE-7122 Proper warning message when opening a log file with no
> > entries
> > > > (idle cluster)
> > > >
> > > > Does the HBase you're using contain HBASE-7122 ?
> > > >
> > > > Cheers
> > > >
> > > > On Sat, Jun 1, 2013 at 10:20 PM, Asaf Mesika <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> I have a weird error in a cluster I'm checking Replication with.
> > > >>
> > > >> I have two clusters set up, each on its own DC (different
> continents).
> > > Each
> > > >> has 1 master, and 3 RS.
> > > >>
> > > >> I've done all required setup, started replication and pushed in some
> > > data
> > > >> into the master. I had an issue where the slave (peer) cluster went
> > dead
> > > >> (all RS failed contacting the master), thus replication couldn't
> work.
> > > This
> > > >> happened right before the weekend, so it was out for 3 days.
> > > >>
> > > >> Now I'm back in the office - got slave cluster back up (just the
> RS),
> > > and I
> > > >> got some nasty exception in one of the RS of the master cluster:
> > > >>
> > > >> 2013-06-02 04:40:45,903 INFO
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Getting
> > > >> 0 rs from peer cluster # c
> > > >> 2013-06-02 04:40:45,903 INFO
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Slave
> > > >> cluster looks down: c has 0 region servers
> > > >> 2013-06-02 04:40:46,903 DEBUG
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Since
> > > >> we are unable to replicate, sleeping 1000 times 10
> > > >> 2013-06-02 04:40:57,019 INFO
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Getting
> > > >> 0 rs from peer cluster # c
> > > >> 2013-06-02 04:40:57,019 INFO
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Slave
> > > >> cluster looks down: c has 0 region servers
> > > >> 2013-06-02 04:40:58,019 DEBUG
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Since
> > > >> we are unable to replicate, sleeping 1000 times 10
> > > >> 2013-06-02 04:41:08,134 INFO
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Getting
> > > >> 1 rs from peer cluster # c
> > > >> 2013-06-02 04:41:08,134 INFO
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> >