Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1


Copy link to this message
-
Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1
there is no master/slave architecture now

its a simple conf file like below <@EOM> and then just start the agent. it
will do needful
you need to start flume like the command below

bin/flume-ng agent --conf-file example.conf --name agent1

agent1.sources = tail
agent1.channels = MemoryChannel-2
agent1.sinks = HDFS
agent1.sources.tail.type = exec
agent1.sources.tail.command = tail -F <file>
agent1.sources.tail.channels = MemoryChannel-2
agent1.sources.tail.interceptors = hostint
agent1.sources.tail.interceptors.hostint.type org.apache.flume.interceptor.HostInterceptor$Builder
agent1.sources.tail.interceptors.hostint.preserveExisting = true
agent1.sources.tail.interceptors.hostint.useIP = true
agent1.sinks.HDFS.channel = MemoryChannel-2
agent1.sinks.HDFS.type = hdfs
agent1.sinks.HDFS.hdfs.path = hdfs://url:8020/user/flume/%{host}
agent1.sinks.HDFS.hdfs.file.Type = SequenceFile
agent1.sinks.HDFS.hdfs.writeFormat = Text
agent1.sinks.HDFS.hdfs.rollInterval = 30
agent1.sinks.HDFS.hdfs.rollCount = 0
agent1.sinks.HDFS.hdfs.rollSize = 5000000000
agent1.channels.MemoryChannel-2.type = memory

more examples at http://flume.apache.org/FlumeUserGuide.html

On Tue, Dec 11, 2012 at 12:44 PM, <[EMAIL PROTECTED]> wrote:

>  Hi,****
>
> I installed flume 1.x higher version today! But is this very much
> different from the earlier. ? Is there no master / node concept here? How
> do I configure the source and sink?****
>
> ** **
>
> Think I got screwed here!****
>
> ** **
>
> *Thanks and Regards,*
>
> *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 *
>
> ** **
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* Monday, December 10, 2012 3:34 PM
>
> *To:* Haldar, Shouvanik
> *Cc:* [EMAIL PROTECTED]
> *Subject:* Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO
> com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1****
>
>  ** **
>
> just a random question, does the user running flume agent have permission
> to read the  event details out of terracotta ? ****
>
> ** **
>
> from the code here
> http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/com.cloudera/flume-core/0.9.4-cdh4.1.0/com/cloudera/flume/watchdog/Watchdog.java
> ****
>
> ** **
>
> line 209 onwards, it looks like the error condition is achieved when you
> have channel set as stdin (I am not sure about this so you may need to look
> at code which part of code you are hitting depending on the flume configs
> you have) ****
>
> ** **
>
> ** **
>
> ** **
>
> On Mon, Dec 10, 2012 at 3:01 PM, <[EMAIL PROTECTED]> wrote:**
> **
>
> Hello,****
>
>  ****
>
> I am sure, the problem is due to Terracotta. Can you please help me? ****
>
>  ****
>
> *Thanks and Regards,*****
>
> *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 *****
>
>  ****
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* Monday, December 10, 2012 2:43 PM
> *To:* Haldar, Shouvanik
> *Cc:* [EMAIL PROTECTED]****
>
>
> *Subject:* Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO
> com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1****
>
>  ****
>
> I just hope you are not hitting this
> https://issues.apache.org/jira/browse/FLUME-796****
>
> I would also like to recommend you to upgraded to flume 1.x asap ****
>
>  ****
>
> if possible can you try changing your flume source to some random file
> with tail -f and try sinking it to some file. That will narrow the problem
> is on flume side or terracotta side ****
>
>  ****
>
> Thanks,****
>
> Nitin ****
>
>  ****
>
> On Mon, Dec 10, 2012 at 1:47 PM, <[EMAIL PROTECTED]> wrote:**
> **
>
> Hi,****
>
>  ****
>
> I am using Flume -> Flume 0.9.4-cdh3u5. ****
>
>  ****
>
> Initially, the logs are ok. But when I start terracotta
> (./start-tc-server.sh) , then only I experience the above phenomena! Is it
> something to do with terracotta.!****
>
>  ****
>
>  ****
>
> 2012-12-10 03:14:08,806 INFO
> com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking

Nitin Pawar