Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> NullPointerException : (Channel closed) on writeCheckpoint


Copy link to this message
-
Re: NullPointerException : (Channel closed) on writeCheckpoint
Hmm that shouldn't cause an NPE. Could you find a JIRA for this?
On Mon, May 13, 2013 at 10:11 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:

> Ahhh I had no batchSize parameter.  So the fix was to add batchSize = 1 to
> my file and now its working.  Anyone wants more details just ping me
>
>
> On Sun, May 12, 2013 at 7:13 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>
>> Hi ! Thanks --- Here's my config
>>
>> a1.sources = r1
>> a1.sinks = myFileSystem
>> a1.channels = c1
>>
>> # Describe/configure the source
>> a1.sources.r1.type = exec
>> a1.sources.r1.command = tail -F /root/xyz/flumeinput.tmp
>> a1.sources.r1.channels = c1
>> a1.sources = r1
>> a1.channels = c1
>> a1.sinks = k1
>> a1.sinks.k1.type = file_roll
>> a1.sinks.k1.channel = c1
>> a1.sinks.k1.sink.directory = /mnt/xyz/flume
>>
>> #Do we need this?
>> agent1.sinks.sink1.hdfs.filePrefix = LogCreateTest
>>
>> # Use a channel which buffers events in memory
>> a1.channels.c1.type = file
>> #a1.channels.c1.capacity = 100
>> #a1.channels.c1.transactionCapacity = 10
>>
>> # Bind the source and sink to the channel
>> a1.sources.r1.channels = c1
>> a1.sinks.myFileSystem.channel = c1
>>
>>
>>
>> On Sun, May 12, 2013 at 8:57 AM, Alexander Alten-Lorenz <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> Could you attach the config too?
>>>
>>> - Alex
>>>
>>> On May 11, 2013, at 11:52 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>>>
>>> > Hi flume :
>>> >
>>> > I've setup a file to file source/sink , just as a test.  Im using the
>>> exec source, with tail -F to follow a simple tmp file that is being catted
>>> to.
>>> >
>>> > However, I've started seeing the following exception in the flume logs:
>>> >
>>> > java.lang.
>>> > IllegalStateException: Channel closed [channel=c1]. Due to
>>> java.lang.NullPointerException: null
>>> >     at
>>> org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:353)
>>> >     at
>>> org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:122)
>>> >     at
>>> org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:183)
>>> >     at
>>> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>>> >     at
>>> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>>> >     at java.lang.Thread.run(Thread.java:662)
>>> > Caused by: java.lang.NullPointerException
>>> >     at org.apache.flume.channel.file.Log.writeCheckpoint(Log.java:895)
>>> >     at org.apache.flume.channel.file.Log.replay(Log.java:406)
>>> >     at
>>> org.apache.flume.channel.file.FileChannel.start(FileChannel.java:303)
>>> >     at
>>> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
>>> >     at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>> >     at
>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>> >     at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> >     at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>> >     at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
>>> >     at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>>> >     at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>> >     at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>> >     ... 1 more
>>> >
>>> > Any thoughts on where this NullPointerException is coming from?  It
>>> appears from scanning the code that maybe it related to a missing folder or
>>> directory.  But I cant find the exact line on the git hub branches.
>>> >
>>> > This is using apache-flume-1.3.1.23-...
>>> >
>>> >
>>> > --
>>> > Jay Vyas
>>> > http://jayunit100.blogspot.com
>>>
>>> --
>>> Alexander Alten-Lorenz
>>> http://mapredit.blogspot.com

Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB