Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Getting "Checking file:conf/flume.conf for changes" message in loop


Copy link to this message
-
Getting "Checking file:conf/flume.conf for changes" message in loop
Hi,

I am getting following message in loop. The source file hasn't moved to the destination.
2013-04-29 08:24:41,346 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:73)] Component type: CHANNEL, name: Channel-2 started
2013-04-29 08:24:41,846 (conf-file-poller-0) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:141)] Starting Sink HDFS
2013-04-29 08:24:41,847 (conf-file-poller-0) [INFO - org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.startAllComponents(DefaultLogicalNodeManager.java:152)] Starting Source tail
2013-04-29 08:24:41,847 (lifecycleSupervisor-1-3) [INFO - org.apache.flume.source.ExecSource.start(ExecSource.java:155)] Exec source starting with command:tail -F /home/vkanth/temp/Sample2.txt
2013-04-29 08:24:41,850 (lifecycleSupervisor-1-3) [DEBUG - org.apache.flume.source.ExecSource.start(ExecSource.java:173)] Exec source started
2013-04-29 08:24:41,850 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.register(MonitoredCounterGroup.java:89)] Monitoried counter group for type: SINK, name: HDFS, registered successfully.
2013-04-29 08:24:41,851 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:73)] Component type: SINK, name: HDFS started
2013-04-29 08:24:41,852 (SinkRunner-PollingRunner-DefaultSinkProcessor) [DEBUG - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:143)] Polling sink runner starting
2013-04-29 08:25:11,855(conf-file-poller-0) [DEBUG - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)] Checking file:conf/flume.conf for changes
2013-04-29 08:25:41,861 (conf-file-poller-0) [DEBUG - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)] Checking file:conf/flume.conf for changes
2013-04-29 08:26:11,868(conf-file-poller-0) [DEBUG - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:188)] Checking file:conf/flume.conf for changes
.......
.......
Flume.conf:
agent1.sources = tail
agent1.channels = Channel-2
agent1.sinks = HDFS

agent1.sources.tail.type = exec
agent1.sources.tail.command = tail -F /home/vikas/temp/Sample2.txt
agent1.sources.tail.channels = Channel-2

agent1.sinks.HDFS.channel = Channel-2
agent1.sinks.HDFS.type = hdfs
agent1.sinks.HDFS.hdfs.path = hdfs://dev-pub01.xyz.abc.com:8020/tmp
agent1.sinks.HDFS.hdfs.file.fileType = DataStream

agent1.channels.Channel-2.type = memory
agent1.channels.Channel-2.capacity = 1000
agent1.channels.Channel-2.transactionCapacity=10

Command:
bin/flume-ng agent --conf ./conf/ -f conf/flume.conf -Dflume.root.logger=DEBUG,console -n agent1

Please let me know if I am missing something.

Thanks,
Vikas
+
Jeff Lord 2013-05-01, 01:35
+
Vikas Kanth 2013-05-01, 15:00
+
Alexander Alten-Lorenz 2013-05-02, 09:54
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB