Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Flume is replaying log for hours now


Copy link to this message
-
Re: Flume is replaying log for hours now
Thanks, I read the docs, I know how to turn it on but there's no
explanation what it is.
Should I just set:
useDualCheckpoints=true
backupCheckpointDir=/some/dir

On Thu, Aug 8, 2013 at 4:05 PM, Brock Noland <[EMAIL PROTECTED]> wrote:

> Note the dual and backup checkpoint configs here:
>
> http://flume.apache.org/FlumeUserGuide.html#file-channel
>
>
> On Thu, Aug 8, 2013 at 7:37 AM, Anat Rozenzon <[EMAIL PROTECTED]> wrote:
>
>> I use 3.5G but I can change it to 5G.
>>
>> Not sure I understand what you mean about dual checkpoint, this is my
>> configuration for each of the three channels, should I change it?
>>
>> collector.channels.mc1.type = file
>>
>> collector.channels.mc1.checkpointDir=/home/flume/collector1/channels/mc1/checkpoint
>> collector.channels.mc1.dataDirs=/home/flume/collector1/channels/mc1/data
>> collector.channels.mc1.capacity=100000000
>> collector.channels.mc1.transactionCapacity=10000
>> collector.channels.mc1.use-fast-replay=true
>>
>>
>>
>> On Thu, Aug 8, 2013 at 3:19 PM, Brock Noland <[EMAIL PROTECTED]> wrote:
>>
>>> use-fast-replay would help but you'd need 4-5GB of heap per channel.
>>> With heaps that large you use be using dual checkpointing to avoid this.
>>>
>>> Here is the thread doing the replay:
>>>
>>> "lifecycleSupervisor-1-0" prio=10 tid=0x00007f040472c800 nid=0x1332b runnable [0x00007f03f84ce000]
>>>    java.lang.Thread.State: RUNNABLE
>>>         at org.apache.flume.channel.file.FlumeEventQueue.remove(FlumeEventQueue.java:194)
>>>         - locked <0x00000007256d3dc8> (a org.apache.flume.channel.file.FlumeEventQueue)
>>>         at org.apache.flume.channel.file.ReplayHandler.processCommit(ReplayHandler.java:405)
>>>         at org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:328)
>>>         at org.apache.flume.channel.file.Log.doReplay(Log.java:503)
>>>         at org.apache.flume.channel.file.Log.replay(Log.java:430)
>>>         at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:302)
>>>         - locked <0x00000007256d2e38> (a org.apache.flume.channel.file.FileChannel)
>>>         at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
>>>         - locked <0x00000007256d2e38> (a org.apache.flume.channel.file.FileChannel)
>>>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>         at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>>>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>>>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>>>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>         at java.lang.Thread.run(Thread.java:722)
>>>
>>>
>>>
>>> On Thu, Aug 8, 2013 at 12:52 AM, Anat Rozenzon <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm trying to restart Flume. My setup is:
>>>>
>>>> Acro source => File channel 1 => HDFS sink
>>>>                    => File channel 2 => Another HDFS sink
>>>>                    => File channel 3 => File sink
>>>>
>>>> But it seem to be doing replayLog for hours now, after seeing this
>>>> yesterday, I even tried setting use-fast-replay=true, but it didn't help.
>>>>
>>>> Each file channel capacity is 100000000, is this too high for Flume? I
>>>> started on lower number but then it complained that the channel is getting
>>>> filled so I made it higher.
>>>>
>>>> My log is repeatedly writing such lines:
>>>> 08 Aug 2013 01:36:22,856 INFO  [lifecycleSupervisor-1-1]
>>>> (org.apache.flume.channel.file.ReplayHandler.replayLog:293)  - Read 3240000
>>>> records
>>>> 08 Aug 2013 01:36:41,324 INFO  [lifecycleSupervisor-1-0]
>>>> (org.apache.flume.channel.file.ReplayHandler.replayLog:293)  - Read 3350000