Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> NullPointerException : (Channel closed) on writeCheckpoint


+
Jay Vyas 2013-05-11, 21:52
+
Alexander Alten-Lorenz 2013-05-12, 12:57
+
Jay Vyas 2013-05-12, 23:13
Copy link to this message
-
Re: NullPointerException : (Channel closed) on writeCheckpoint
Ahhh I had no batchSize parameter.  So the fix was to add batchSize = 1 to
my file and now its working.  Anyone wants more details just ping me
On Sun, May 12, 2013 at 7:13 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:

> Hi ! Thanks --- Here's my config
>
> a1.sources = r1
> a1.sinks = myFileSystem
> a1.channels = c1
>
> # Describe/configure the source
> a1.sources.r1.type = exec
> a1.sources.r1.command = tail -F /root/xyz/flumeinput.tmp
> a1.sources.r1.channels = c1
> a1.sources = r1
> a1.channels = c1
> a1.sinks = k1
> a1.sinks.k1.type = file_roll
> a1.sinks.k1.channel = c1
> a1.sinks.k1.sink.directory = /mnt/xyz/flume
>
> #Do we need this?
> agent1.sinks.sink1.hdfs.filePrefix = LogCreateTest
>
> # Use a channel which buffers events in memory
> a1.channels.c1.type = file
> #a1.channels.c1.capacity = 100
> #a1.channels.c1.transactionCapacity = 10
>
> # Bind the source and sink to the channel
> a1.sources.r1.channels = c1
> a1.sinks.myFileSystem.channel = c1
>
>
>
> On Sun, May 12, 2013 at 8:57 AM, Alexander Alten-Lorenz <
> [EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> Could you attach the config too?
>>
>> - Alex
>>
>> On May 11, 2013, at 11:52 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>>
>> > Hi flume :
>> >
>> > I've setup a file to file source/sink , just as a test.  Im using the
>> exec source, with tail -F to follow a simple tmp file that is being catted
>> to.
>> >
>> > However, I've started seeing the following exception in the flume logs:
>> >
>> > java.lang.
>> > IllegalStateException: Channel closed [channel=c1]. Due to
>> java.lang.NullPointerException: null
>> >     at
>> org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:353)
>> >     at
>> org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:122)
>> >     at
>> org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:183)
>> >     at
>> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>> >     at
>> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>> >     at java.lang.Thread.run(Thread.java:662)
>> > Caused by: java.lang.NullPointerException
>> >     at org.apache.flume.channel.file.Log.writeCheckpoint(Log.java:895)
>> >     at org.apache.flume.channel.file.Log.replay(Log.java:406)
>> >     at
>> org.apache.flume.channel.file.FileChannel.start(FileChannel.java:303)
>> >     at
>> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
>> >     at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>> >     at
>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>> >     at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>> >     at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>> >     at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
>> >     at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>> >     at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>> >     at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>> >     ... 1 more
>> >
>> > Any thoughts on where this NullPointerException is coming from?  It
>> appears from scanning the code that maybe it related to a missing folder or
>> directory.  But I cant find the exact line on the git hub branches.
>> >
>> > This is using apache-flume-1.3.1.23-...
>> >
>> >
>> > --
>> > Jay Vyas
>> > http://jayunit100.blogspot.com
>>
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>
>>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>

--
Jay Vyas
http://jayunit100.blogspot.com
+
Brock Noland 2013-05-13, 15:39
+
Jay Vyas 2013-05-13, 15:53
+
Jay Vyas 2013-05-13, 15:58
+
Brock Noland 2013-05-13, 16:16
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB