Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Replication warning


+
Álvaro Recuero 2013-05-27, 12:15
+
Ted Yu 2013-05-27, 13:43
+
Álvaro Recuero 2013-05-27, 15:40
+
Ted Yu 2013-05-28, 03:26
+
Álvaro Recuero 2013-05-28, 12:23
Copy link to this message
-
Re: Replication warning
You can also try upgrading to 0.94.8 - there are other fixes for replication that you will benefit.

Cheers

On May 28, 2013, at 5:23 AM, Álvaro Recuero <[EMAIL PROTECTED]> wrote:

> Hi Ted,
>
> I see, let me check that out then and try to integrate that patch-code into
> my release. For sure it will make a difference just by seeing the lines in
> the patch mentioned. Thank you.
>
> Best,
> Álvaro
>
>
> On 28 May 2013 04:26, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> There was a recently integrated JIRA (0.94.8):
>> HBASE-7122 Proper warning message when opening a log file with no entries
>> (idle cluster)
>>
>> The stack trace in above JIRA looks very similar to yours.
>>
>> On Mon, May 27, 2013 at 8:40 AM, Álvaro Recuero <[EMAIL PROTECTED]>
>> wrote:
>>
>>> Hi All,
>>>
>>> From the shell:
>>>
>>> ~/hbase-0.94.4$ ./bin/hbase shell
>>> HBase Shell; enter 'help<RETURN>' for list of supported commands.
>>> Type "exit<RETURN>" to leave the HBase Shell
>>> Version 0.94.4, r, Mon May 27 15:29:11 WEST 2013
>>>
>>> I have seen today it comes up as soon as there is a new item to be
>>> replicated, and even happens if an old Hlog is rolled. I am not really
>> sure
>>> what is going there behind the scenes of Hbase.
>>>
>>> Regards,
>>> Álvaro
>>>
>>> On 27 May 2013 14:43, Ted Yu <[EMAIL PROTECTED]> wrote:
>>>
>>>> The warning came from this loop in ReplicationSource:
>>>>
>>>>    // Loop until we close down
>>>>    while (isActive()) {
>>>> ...
>>>>      // Open a reader on it
>>>>      if (!openReader(sleepMultiplier)) {
>>>>        // Reset the sleep multiplier, else it'd be reused for the next
>>>> file
>>>>        sleepMultiplier = 1;
>>>>        continue;
>>>>      }
>>>>
>>>>      // If we got a null reader but didn't continue, then sleep and
>>>> continue
>>>>      if (this.reader == null) {
>>>>        if (sleepForRetries("Unable to open a reader",
>> sleepMultiplier))
>>> {
>>>>          sleepMultiplier++;
>>>>        }
>>>>        continue;
>>>>      }
>>>>
>>>> How often did the following warning come up in the server log ?
>>>> Which version of HBase are you using ?
>>>>
>>>> Cheers
>>>>
>>>> On Mon, May 27, 2013 at 5:15 AM, Álvaro Recuero <[EMAIL PROTECTED]>
>>>> wrote:
>>>>
>>>>> Does anyone get this when replication is enabled in HBase? If it is a
>>>>> warning but does not stop coming up, I wonder what that loop is that
>>>>> iterating through and if this is really a critical problem.
>>>>>
>>>>> cheers,
>>>>> Álvaro
>>>>>
>>>>> 2013-05-27 13:10:40,406 WARN
>>>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
>>>> Waited
>>>>> too long for this file, considering dumping
>>>>> 2013-05-27 13:10:40,406 DEBUG
>>>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
>>>> Unable
>>>>> to open a reader, sleeping 1000 times 10
>>>>> 2013-05-27 13:10:50,411 WARN
>>>>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: 1
>>>> Got:
>>>>> java.io.EOFException
>>>>> at java.io.DataInputStream.readFully(DataInputStream.java:197)
>>>>> at java.io.DataInputStream.readFully(DataInputStream.java:169)
>>>>> at
>>> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
>>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1486)
>>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
>>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
>>>>> at
>> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:55)
>>>>> at
>> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:177)
>>>>> at
>>> org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:721)
>>>>> at
>> org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.openReader(ReplicationHLogReaderManager.java:66)
>>>>> at
>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:562)
+
Álvaro Recuero 2013-05-30, 10:27