Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> NullPointerException : (Channel closed) on writeCheckpoint


+
Jay Vyas 2013-05-11, 21:52
+
Alexander Alten-Lorenz 2013-05-12, 12:57
+
Jay Vyas 2013-05-12, 23:13
+
Jay Vyas 2013-05-13, 15:11
Copy link to this message
-
Re: NullPointerException : (Channel closed) on writeCheckpoint
Hmm that shouldn't cause an NPE. Could you find a JIRA for this?
On Mon, May 13, 2013 at 10:11 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:

> Ahhh I had no batchSize parameter.  So the fix was to add batchSize = 1 to
> my file and now its working.  Anyone wants more details just ping me
>
>
> On Sun, May 12, 2013 at 7:13 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>
>> Hi ! Thanks --- Here's my config
>>
>> a1.sources = r1
>> a1.sinks = myFileSystem
>> a1.channels = c1
>>
>> # Describe/configure the source
>> a1.sources.r1.type = exec
>> a1.sources.r1.command = tail -F /root/xyz/flumeinput.tmp
>> a1.sources.r1.channels = c1
>> a1.sources = r1
>> a1.channels = c1
>> a1.sinks = k1
>> a1.sinks.k1.type = file_roll
>> a1.sinks.k1.channel = c1
>> a1.sinks.k1.sink.directory = /mnt/xyz/flume
>>
>> #Do we need this?
>> agent1.sinks.sink1.hdfs.filePrefix = LogCreateTest
>>
>> # Use a channel which buffers events in memory
>> a1.channels.c1.type = file
>> #a1.channels.c1.capacity = 100
>> #a1.channels.c1.transactionCapacity = 10
>>
>> # Bind the source and sink to the channel
>> a1.sources.r1.channels = c1
>> a1.sinks.myFileSystem.channel = c1
>>
>>
>>
>> On Sun, May 12, 2013 at 8:57 AM, Alexander Alten-Lorenz <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> Could you attach the config too?
>>>
>>> - Alex
>>>
>>> On May 11, 2013, at 11:52 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>>>
>>> > Hi flume :
>>> >
>>> > I've setup a file to file source/sink , just as a test.  Im using the
>>> exec source, with tail -F to follow a simple tmp file that is being catted
>>> to.
>>> >
>>> > However, I've started seeing the following exception in the flume logs:
>>> >
>>> > java.lang.
>>> > IllegalStateException: Channel closed [channel=c1]. Due to
>>> java.lang.NullPointerException: null
>>> >     at
>>> org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:353)
>>> >     at
>>> org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:122)
>>> >     at
>>> org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:183)
>>> >     at
>>> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>>> >     at
>>> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>>> >     at java.lang.Thread.run(Thread.java:662)
>>> > Caused by: java.lang.NullPointerException
>>> >     at org.apache.flume.channel.file.Log.writeCheckpoint(Log.java:895)
>>> >     at org.apache.flume.channel.file.Log.replay(Log.java:406)
>>> >     at
>>> org.apache.flume.channel.file.FileChannel.start(FileChannel.java:303)
>>> >     at
>>> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
>>> >     at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>> >     at
>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>> >     at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> >     at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>> >     at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
>>> >     at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>>> >     at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>> >     at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>> >     ... 1 more
>>> >
>>> > Any thoughts on where this NullPointerException is coming from?  It
>>> appears from scanning the code that maybe it related to a missing folder or
>>> directory.  But I cant find the exact line on the git hub branches.
>>> >
>>> > This is using apache-flume-1.3.1.23-...
>>> >
>>> >
>>> > --
>>> > Jay Vyas
>>> > http://jayunit100.blogspot.com
>>>
>>> --
>>> Alexander Alten-Lorenz
>>> http://mapredit.blogspot.com

Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
+
Jay Vyas 2013-05-13, 15:53
+
Jay Vyas 2013-05-13, 15:58
+
Brock Noland 2013-05-13, 16:16