Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Weird Replication exception


Copy link to this message
-
Re: Weird Replication exception
No, this was brand new with 0 length thus the peculiar message of too old
was strange to me.

On Monday, June 3, 2013, Himanshu Vashishtha wrote:

> Hey Asaf,
>
> It looks like you only need 7122. Either upgrade, or you could also patch
> it up.
>
> Syncing up the master and slave cluster is also advised, but that stands
> good in case you are using master-master replication.
>
>
> bq. 172.25.98.74,60020,
> 1369903540894/172.25.98.74
> %2C60020%2C1369903540894.1369925171871
>
> bq.    Meaning I lost data
>
> Did the log whose znode you deleted had any data?
> You can do a cat to see if there is any data on it. You could copy-table
> for that time range, (or a hacky way is to re-create a znode for that log
> under a regionserver noticing the format of its current log znodes, and let
> the replicationSource pick it up in its normal run).
>
> Thanks,
> Himanshu
>
>
>
> On Sun, Jun 2, 2013 at 12:38 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > bq. Is 0.94.8 production ready?
> >
> > I think so. Lars released 0.94.8 Friday evening.
> >
> > On Sun, Jun 2, 2013 at 12:26 PM, Asaf Mesika <[EMAIL PROTECTED]>
> > wrote:
> >
> > > I use 0.94.7.
> > > Is 0.94.8 production ready?
> > >
> > > So in summary I have two issues:
> > > 1. Clocks are out of sync
> > > 2. I need to upgrade to 0.94.8 to avoid seeing this WARN messages?
> > >
> > > On Jun 2, 2013, at 5:46 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> > >
> > > > What is the HBase version you're using ?
> > > >
> > > > In another thread, I mentioned this:
> > > >
> > > > There was a recently integrated JIRA (0.94.8):
> > > > HBASE-7122 Proper warning message when opening a log file with no
> > entries
> > > > (idle cluster)
> > > >
> > > > Does the HBase you're using contain HBASE-7122 ?
> > > >
> > > > Cheers
> > > >
> > > > On Sat, Jun 1, 2013 at 10:20 PM, Asaf Mesika <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> I have a weird error in a cluster I'm checking Replication with.
> > > >>
> > > >> I have two clusters set up, each on its own DC (different
> continents).
> > > Each
> > > >> has 1 master, and 3 RS.
> > > >>
> > > >> I've done all required setup, started replication and pushed in some
> > > data
> > > >> into the master. I had an issue where the slave (peer) cluster went
> > dead
> > > >> (all RS failed contacting the master), thus replication couldn't
> work.
> > > This
> > > >> happened right before the weekend, so it was out for 3 days.
> > > >>
> > > >> Now I'm back in the office - got slave cluster back up (just the
> RS),
> > > and I
> > > >> got some nasty exception in one of the RS of the master cluster:
> > > >>
> > > >> 2013-06-02 04:40:45,903 INFO
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Getting
> > > >> 0 rs from peer cluster # c
> > > >> 2013-06-02 04:40:45,903 INFO
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Slave
> > > >> cluster looks down: c has 0 region servers
> > > >> 2013-06-02 04:40:46,903 DEBUG
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Since
> > > >> we are unable to replicate, sleeping 1000 times 10
> > > >> 2013-06-02 04:40:57,019 INFO
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Getting
> > > >> 0 rs from peer cluster # c
> > > >> 2013-06-02 04:40:57,019 INFO
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Slave
> > > >> cluster looks down: c has 0 region servers
> > > >> 2013-06-02 04:40:58,019 DEBUG
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Since
> > > >> we are unable to replicate, sleeping 1000 times 10
> > > >> 2013-06-02 04:41:08,134 INFO
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > > Getting
> > > >> 1 rs from peer cluster # c
> > > >> 2013-06-02 04:41:08,134 INFO
> > > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> >
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB