Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Replication warning


Copy link to this message
-
Re: Replication warning
There was a recently integrated JIRA (0.94.8):
HBASE-7122 Proper warning message when opening a log file with no entries
(idle cluster)

The stack trace in above JIRA looks very similar to yours.

On Mon, May 27, 2013 at 8:40 AM, Álvaro Recuero <[EMAIL PROTECTED]> wrote:

> Hi All,
>
> From the shell:
>
> ~/hbase-0.94.4$ ./bin/hbase shell
> HBase Shell; enter 'help<RETURN>' for list of supported commands.
> Type "exit<RETURN>" to leave the HBase Shell
> Version 0.94.4, r, Mon May 27 15:29:11 WEST 2013
>
> I have seen today it comes up as soon as there is a new item to be
> replicated, and even happens if an old Hlog is rolled. I am not really sure
> what is going there behind the scenes of Hbase.
>
> Regards,
> Álvaro
>
> On 27 May 2013 14:43, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > The warning came from this loop in ReplicationSource:
> >
> >     // Loop until we close down
> >     while (isActive()) {
> > ...
> >       // Open a reader on it
> >       if (!openReader(sleepMultiplier)) {
> >         // Reset the sleep multiplier, else it'd be reused for the next
> > file
> >         sleepMultiplier = 1;
> >         continue;
> >       }
> >
> >       // If we got a null reader but didn't continue, then sleep and
> > continue
> >       if (this.reader == null) {
> >         if (sleepForRetries("Unable to open a reader", sleepMultiplier))
> {
> >           sleepMultiplier++;
> >         }
> >         continue;
> >       }
> >
> > How often did the following warning come up in the server log ?
> > Which version of HBase are you using ?
> >
> > Cheers
> >
> > On Mon, May 27, 2013 at 5:15 AM, Álvaro Recuero <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Does anyone get this when replication is enabled in HBase? If it is a
> > > warning but does not stop coming up, I wonder what that loop is that
> > > iterating through and if this is really a critical problem.
> > >
> > > cheers,
> > > Álvaro
> > >
> > > 2013-05-27 13:10:40,406 WARN
> > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > Waited
> > > too long for this file, considering dumping
> > > 2013-05-27 13:10:40,406 DEBUG
> > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
> > Unable
> > > to open a reader, sleeping 1000 times 10
> > > 2013-05-27 13:10:50,411 WARN
> > > org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: 1
> > Got:
> > > java.io.EOFException
> > > at java.io.DataInputStream.readFully(DataInputStream.java:197)
> > >  at java.io.DataInputStream.readFully(DataInputStream.java:169)
> > > at
> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> > >  at
> > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
> > > at
> > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
> > >  at
> > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:55)
> > >  at
> > >
> > >
> >
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:177)
> > > at
> org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:721)
> > >  at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.openReader(ReplicationHLogReaderManager.java:66)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:562)
> > >  at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:339)
> > >
> >
>