Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # dev - Review Request: FLUME-1986. doTestInflightCorrupts should not commit transactions


+
Hari Shreedharan 2013-04-11, 01:09
Copy link to this message
-
Re: Review Request: FLUME-1986. doTestInflightCorrupts should not commit transactions
Juhani Connolly 2013-04-12, 09:37

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10415/#review19059
-----------------------------------------------------------
The tests still don't pass in our environment, failing on the assertion
 Assert.assertTrue(!backup || channel.checkpointBackupRestored())

Relevant from the surefire-reports:

  <testcase time="7.258" classname="org.apache.flume.channel.file.TestFileChannelRestart" name="testCorruptInflightTakesWithBackup">
    <failure type="java.lang.AssertionError">java.lang.AssertionError
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertTrue(Assert.java:54)
        at org.apache.flume.channel.file.TestFileChannelRestart.doTestCorruptInflights(TestFileChannelRestart.java:514)
        at org.apache.flume.channel.file.TestFileChannelRestart.testCorruptInflightTakesWithBackup(TestFileChannelRestart.java:431)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
...
</failure>
    <system-out>2013-04-12 18:16:16,498 (main) [INFO - org.apache.flume.channel.file.FileChannel.start(FileChannel.java:281)] Starting FileChannel FileChannel-3d21a8cc-6f46-45c4-8dee-ac5fd530c934 { dataDirs: [/tmp/1365758176497-0/data1,
/tmp/1365758176497-0/data2, /tmp/1365758176497-0/data3] }...
2013-04-12 18:16:16,499 (main) [INFO - org.apache.flume.channel.file.Log.<init>(Log.java:321)] Encryption is not enabled
2013-04-12 18:16:16,499 (main) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:368)] Replay started
2013-04-12 18:16:16,500 (main) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:380)] Found NextFileID 0, from []
2013-04-12 18:16:16,504 (main) [INFO - org.apache.flume.channel.file.EventQueueBackingStoreFile.<init>(EventQueueBackingStoreFile.java:91)] Preallocated /tmp/1365758176497-0/chkpt/checkpoint to 88232 for capacity 10000
2013-04-12 18:16:16,505 (main) [INFO - org.apache.flume.channel.file.EventQueueBackingStoreFileV3.<init>(EventQueueBackingStoreFileV3.java:53)] Starting up with /tmp/1365758176497-0/chkpt/checkpoint and /tmp/1365758176497-0/chkpt/c
heckpoint.meta
2013-04-12 18:16:16,507 (main) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:417)] Last Checkpoint Fri Apr 12 18:16:16 JST 2013, queue depth = 0
2013-04-12 18:16:16,507 (main) [INFO - org.apache.flume.channel.file.Log.doReplay(Log.java:498)] Replaying logs with v2 replay logic
2013-04-12 18:16:16,507 (main) [INFO - org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:244)] Starting replay of []
2013-04-12 18:16:16,507 (main) [INFO - org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:338)] read: 0, put: 0, take: 0, rollback: 0, commit: 0, skip: 0, eventCount:0
2013-04-12 18:16:16,508 (main) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:461)] Rolling /tmp/1365758176497-0/data1
2013-04-12 18:16:16,508 (main) [INFO - org.apache.flume.channel.file.Log.roll(Log.java:922)] Roll start /tmp/1365758176497-0/data1
2013-04-12 18:16:16,508 (main) [INFO - org.apache.flume.channel.file.LogFile$Writer.<init>(LogFile.java:169)] Opened /tmp/1365758176497-0/data1/log-1
2013-04-12 18:16:16,510 (main) [INFO - org.apache.flume.channel.file.Log.roll(Log.java:937)] Roll end
2013-04-12 18:16:16,510 (main) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:461)] Rolling /tmp/1365758176497-0/data2
2013-04-12 18:16:16,511 (main) [INFO - org.apache.flume.channel.file.Log.roll(Log.java:922)] Roll start /tmp/1365758176497-0/data2
2013-04-12 18:16:16,511 (main) [INFO - org.apache.flume.channel.file.LogFile$Writer.<init>(LogFile.java:169)] Opened /tmp/1365758176497-0/data2/log-2
2013-04-12 18:16:16,513 (main) [INFO - org.apache.flume.channel.file.Log.roll(Log.java:937)] Roll end
2013-04-12 18:16:16,513 (main) [INFO - org.apache.flume.channel.file.Log.replay(Log.java:461)] Rolling /tmp/1365758176497-0/data3
2013-04-12 18:16:16,513 (main) [INFO - org.apache.flume.channel.file.Log.roll(Log.java:922)] Roll start /tmp/1365758176497-0/data3
2013-04-12 18:16:16,514 (main) [INFO - org.apache.flume.channel.file.LogFile$Writer.<init>(LogFile.java:169)] Opened /tmp/1365758176497-0/data3/log-3
2013-04-12 18:16:16,516 (main) [INFO - org.apache.flume.channel.file.Log.roll(Log.java:937)] Roll end
2013-04-12 18:16:16,516 (main) [INFO - org.apache.flume.channel.file.EventQueueBackingStoreFile.beginCheckpoint(EventQueueBackingStoreFile.java:215)] Start checkpoint for /tmp/1365758176497-0/chkpt/checkpoint, elements to sync = 0
2013-04-12 18:16:16,520 (main) [INFO - org.apache.flume.channel.file.EventQueueBackingStoreFile.checkpoint(EventQueueBackingStoreFile.java:240)] Updating checkpoint metadata: logWriteOrderID: 1365758273442, queueSize: 0, queueHead: 0
2013-04-12 18:16:16,521 (main) [INFO - org.apache.flume.channel.file.EventQueueBackingStoreFile.startBackupThread(EventQueueBackingStoreFile.java:276)] Attempting to back up checkpoint.
2013-04-12 18:16:16,522 ([channel=FileChannel-3d21a8cc-6f46-45c4-8dee-ac5fd530c934] - CheckpointBackUpThread) [INFO - org.apache.flume.channel.file.Serialization.deleteAllFiles(Serialization.java:100)] Skipping in_use.lock because it is
in excludes set
2013-04-12 18:16:16,522 ([channel=FileChannel-3d21a8cc-6f46-45c4-8dee-ac5fd530c934] - CheckpointBackUpThread) [INFO - org.apache.flume.channel.file.Serialization.deleteAllFiles(Serialization.java:113)] Deleted the following files: .
2013-04-12 18:16:16,524 (main) [INFO - org.apache.flume.channel.file.Log.writeCheckpoint(Log.java:997)] Updated checkpoint for file: /tmp/1365758176497-0/data1/log-1 position: 0 logWriteOrderID:
+
Hari Shreedharan 2013-04-12, 22:50
+
Hari Shreedharan 2013-04-12, 22:49
+
Juhani Connolly 2013-04-15, 02:23