Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Flume service stopped automatically


Copy link to this message
-
Re: Flume service stopped automatically
Hi Brock,

Which log file should I check under /var/log?

Following are the list of files under /var/log modified from yesterdat:

-rw-r----- 1 syslog      adm         1227 2013-05-31 06:25 syslog.2.gz
-rw-r----- 1 mysql       adm           20 2013-05-31 06:25 mysql.log.1.gz
-rw-rw-r-- 1 root        utmp       40704 2013-05-31 23:15 wtmp.1
-rw-r----- 1 syslog      adm        14296 2013-06-01 06:25 syslog.1
-rw-r--r-- 1 root        root           0 2013-06-01 06:25 dpkg.log
drwxr-xr-x 2 root        root        4096 2013-06-01 06:25 apt
-rw-r--r-- 1 root        root           0 2013-06-01 06:25 alternatives.log
-rw-r----- 1 mysql       adm            0 2013-06-01 06:25 mysql.log
drwxr-s--- 2 Debian-exim adm         4096 2013-06-01 06:25 exim4
-rw-rw---- 1 root        utmp           0 2013-06-01 06:25 btmp
-rw-r----- 1 syslog      adm       119666 2013-06-01 18:03 kern.log
-rw-r----- 1 syslog      adm         8845 2013-06-01 21:09 syslog
-rw-r----- 1 syslog      adm      4713281 2013-06-01 21:14 auth.log
-rw-rw-r-- 1 root        utmp        3456 2013-06-01 21:14 wtmp
-rw-r--r-- 1 root        root      293752 2013-06-01 21:14 lastlog

Thanks,
Lenin
On Sat, Jun 1, 2013 at 9:12 PM, Brock Noland <[EMAIL PROTECTED]> wrote:

> Weird... It could have been the linux oom killer? You'd see something in
> /var/log/message if that was the case.
>
>
> On Sat, Jun 1, 2013 at 2:47 AM, Lenin Raj <[EMAIL PROTECTED]> wrote:
>
>> Hello,
>>
>> I have a flume service which pulls twitter data and sinks to HDFS. I
>> started it last night at 8 PM. It was going well. When I checked the status
>> today morning, it seems flume service stopped at around 11:15 PM.
>>
>> I could not find any valid reason in the log file. Flume is doing "Flume
>> node stopping - TwitterAgent" automatically without throwing any error.
>>
>>
>> 31 May 2013 23:13:32,226 INFO  [hdfs-HDFS-call-runner-7]
>> (org.apache.flume.sink.hdfs.BucketWriter.doOpen:208)  - Creating
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23//FlumeData.1370021414624.tmp
>> 31 May 2013 23:14:02,247 INFO  [hdfs-HDFS-roll-timer-0]
>> (org.apache.flume.sink.hdfs.BucketWriter.renameBucket:427)  - Renaming
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23/FlumeData.1370021414624.tmp
>> to
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23/FlumeData.1370021414624
>> 31 May 2013 23:14:26,260 INFO  [hdfs-HDFS-call-runner-1]
>> (org.apache.flume.sink.hdfs.BucketWriter.doOpen:208)  - Creating
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23//FlumeData.1370021414625.tmp
>> 31 May 2013 23:14:56,288 INFO  [hdfs-HDFS-roll-timer-0]
>> (org.apache.flume.sink.hdfs.BucketWriter.renameBucket:427)  - Renaming
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23/FlumeData.1370021414625.tmp
>> to
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23/FlumeData.1370021414625
>> 31 May 2013 23:15:02,317 INFO  [hdfs-HDFS-call-runner-5]
>> (org.apache.flume.sink.hdfs.BucketWriter.doOpen:208)  - Creating
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23//FlumeData.1370021414626.tmp
>> 31 May 2013 23:15:32,340 INFO  [hdfs-HDFS-roll-timer-0]
>> (org.apache.flume.sink.hdfs.BucketWriter.renameBucket:427)  - Renaming
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23/FlumeData.1370021414626.tmp
>> to
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23/FlumeData.1370021414626
>> 31 May 2013 23:15:43,632 INFO  [hdfs-HDFS-call-runner-9]
>> (org.apache.flume.sink.hdfs.BucketWriter.doOpen:208)  - Creating
>> hdfs://localhost:54310/user/flume/tweets/2013/05/31/23//FlumeData.1370021414627.tmp
>> 31 May 2013 23:15:56,604 INFO  [node-shutdownHook]
>> (org.apache.flume.node.FlumeNode.stop:67)  - Flume node stopping -
>> TwitterAgent
>> 31 May 2013 23:15:56,604 INFO  [node-shutdownHook]
>> (org.apache.flume.lifecycle.LifecycleSupervisor.stop:78)  - Stopping
>> lifecycle supervisor 7
>> 31 May 2013 23:15:56,606 INFO  [node-shutdownHook]
>> (org.apache.flume.node.nodemanager.DefaultLogicalNodeManager.stop:215)  -