Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - flume-ng data recovery


+
Camp, Roy 2013-01-15, 00:10
Copy link to this message
-
Re: flume-ng data recovery
Brock Noland 2013-01-15, 00:13
Roy,

You deleted the checkpoint files AND the inflights and you got the same error?

Brock

On Mon, Jan 14, 2013 at 4:10 PM, Camp, Roy <[EMAIL PROTECTED]> wrote:
> I somehow got one of my instances into a bad state where I continue to get
> the following error.  I have two data log files with about 2GB worth of data
> that I’d like to save.  Removing the current checkpoint and/or inflight
> files did not work.  Any ideas?
>
>
>
> Thanks,
>
>
>
> Roy
>
>
>
>
>
> Flume-NG 1.3 SNAPSHOT – not the 1.3 stable release
>
>
>
> 2013-01-14 17:03:53,570 (lifecycleSupervisor-1-1) [INFO -
> org.apache.flume.channel.file.Log.replay(Log.java:355)] Replaying logs with
> v2 replay logic
>
> 2013-01-14 17:03:53,574 (lifecycleSupervisor-1-1) [INFO -
> org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:223)]
> Starting replay of [/var/log/flume-ng/collectorfix/data/log-240,
> /var/log/flume-ng/collectorfix/data/log-241]
>
> 2013-01-14 17:03:53,576 (lifecycleSupervisor-1-1) [ERROR -
> org.apache.flume.channel.file.Log.replay(Log.java:373)] Failed to initialize
> Log on [channel=collectorfilefix]
>
> java.lang.IllegalStateException: Checksum of inflights file differs from the
> checksum expected.
>
>     at
> org.apache.flume.channel.file.FlumeEventQueue$InflightEventWrapper.deserialize(FlumeEventQueue.java:487)
>
>     at
> org.apache.flume.channel.file.FlumeEventQueue.deserializeInflightPuts(FlumeEventQueue.java:83)
>
>     at
> org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:226)
>
>     at org.apache.flume.channel.file.Log.replay(Log.java:356)
>
>     at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:258)
>
>     at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
>
>     at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>
>     at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
>
>     at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
>
>     at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
> Source)
>
>     at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
> Source)
>
>     at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
>
>     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
>
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>
>     at java.lang.Thread.run(Unknown Source)
>
> 2013-01-14 17:03:53,579 (lifecycleSupervisor-1-1) [ERROR -
> org.apache.flume.channel.file.FileChannel.start(FileChannel.java:269)]
> Failed to start the file channel [channel=collectorfilefix]
>
> java.lang.IllegalStateException: Checksum of inflights file differs from the
> checksum expected.
>
>     at
> org.apache.flume.channel.file.FlumeEventQueue$InflightEventWrapper.deserialize(FlumeEventQueue.java:487)
>
>     at
> org.apache.flume.channel.file.FlumeEventQueue.deserializeInflightPuts(FlumeEventQueue.java:83)
>
>     at
> org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:226)
>
>     at org.apache.flume.channel.file.Log.replay(Log.java:356)
>
>     at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:258)
>
>     at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
>
>     at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>
>     at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
>
>     at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
>
>     at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
> Source)
>
>     at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
> Source)
>
>     at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
>
>     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown

Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Camp, Roy 2013-01-15, 01:33
+
Brock Noland 2013-01-15, 01:48