Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Flume service stopped automatically


Copy link to this message
-
Re: Flume service stopped automatically
Hi Brock,

Which log file should I check under /var/log?

Following are the list of files under /var/log modified from yesterdat:

-rw-r----- 1 syslog      adm         1227 2013-05-31 06:25 syslog.2.gz
-rw-r----- 1 mysql       adm           20 2013-05-31 06:25 mysql.log.1.gz
-rw-rw-r-- 1 root        utmp       40704 2013-05-31 23:15 wtmp.1
-rw-r----- 1 syslog      adm        14296 2013-06-01 06:25 syslog.1
-rw-r--r-- 1 root        root           0 2013-06-01 06:25 dpkg.log
drwxr-xr-x 2 root        root        4096 2013-06-01 06:25 apt
-rw-r--r-- 1 root        root           0 2013-06-01 06:25 alternatives.log
-rw-r----- 1 mysql       adm            0 2013-06-01 06:25 mysql.log
drwxr-s--- 2 Debian-exim adm         4096 2013-06-01 06:25 exim4
-rw-rw---- 1 root        utmp           0 2013-06-01 06:25 btmp
-rw-r----- 1 syslog      adm       119666 2013-06-01 18:03 kern.log
-rw-r----- 1 syslog      adm         8845 2013-06-01 21:09 syslog
-rw-r----- 1 syslog      adm      4713281 2013-06-01 21:14 auth.log
-rw-rw-r-- 1 root        utmp        3456 2013-06-01 21:14 wtmp
-rw-r--r-- 1 root        root      293752 2013-06-01 21:14 lastlog

Thanks,
Lenin
On Sat, Jun 1, 2013 at 9:12 PM, Brock Noland <[EMAIL PROTECTED]> wrote:

> Weird... It could have been the linux oom killer? You'd see something in
> /var/log/message if that was the case.
>
>
> On Sat, Jun 1, 2013 at 2:47 AM, Lenin Raj <[EMAIL PROTECTED]> wrote:
>
>> Hello,
>>
>> I have a flume service which pulls twitter data and sinks to HDFS. I
>> started it last night at 8 PM. It was going well. When I checked the status
>> today morning, it seems flume service stopped at around 11:15 PM.
>>
>> I could not find any valid reason in the log file. Flume is doing "Flume
>> node stopping - TwitterAgent" automatically without throwing any error.
>>
>>
>> 31 May 2013 23:13:32,226 INFO  [hdfs-HDFS-call-runner-7]
>> (org.apache.flume.sink.hdfs.BucketWriter.doOpen:208)  - Creating
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23//FlumeData.1370021414624.tmp
>> 31 May 2013 23:14:02,247 INFO  [hdfs-HDFS-roll-timer-0]
>> (org.apache.flume.sink.hdfs.BucketWriter.renameBucket:427)  - Renaming
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23/FlumeData.1370021414624.tmp
>> to
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23/FlumeData.1370021414624
>> 31 May 2013 23:14:26,260 INFO  [hdfs-HDFS-call-runner-1]
>> (org.apache.flume.sink.hdfs.BucketWriter.doOpen:208)  - Creating
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23//FlumeData.1370021414625.tmp
>> 31 May 2013 23:14:56,288 INFO  [hdfs-HDFS-roll-timer-0]
>> (org.apache.flume.sink.hdfs.BucketWriter.renameBucket:427)  - Renaming
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23/FlumeData.1370021414625.tmp
>> to
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23/FlumeData.1370021414625
>> 31 May 2013 23:15:02,317 INFO  [hdfs-HDFS-call-runner-5]
>> (org.apache.flume.sink.hdfs.BucketWriter.doOpen:208)  - Creating
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23//FlumeData.1370021414626.tmp
>> 31 May 2013 23:15:32,340 INFO  [hdfs-HDFS-roll-timer-0]
>> (org.apache.flume.sink.hdfs.BucketWriter.renameBucket:427)  - Renaming
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23/FlumeData.1370021414626.tmp
>> to
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23/FlumeData.1370021414626
>> 31 May 2013 23:15:43,632 INFO  [hdfs-HDFS-call-runner-9]
>> (org.apache.flume.sink.hdfs.BucketWriter.doOpen:208)  - Creating
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23//FlumeData.1370021414627.tmp
>> 31 May 2013 23:15:56,604 INFO  [node-shutdownHook]
>> (org.apache.flume.node.FlumeNode.stop:67)  - Flume node stopping -
>> TwitterAgent
>> 31 May 2013 23:15:56,604 INFO  [node-shutdownHook]
>> (org.apache.flume.lifecycle.LifecycleSupervisor.stop:78)  - Stopping
>> lifecycle supervisor 7
>> 31 May 2013 23:15:56,606 INFO  [node-shutdownHook]
>> (org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stop:215)  -
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB