Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1


Copy link to this message
-
Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1
there is no master/slave architecture now

its a simple conf file like below <@EOM> and then just start the agent. it
will do needful
you need to start flume like the command below

bin/flume-ng agent --conf-file example.conf --name agent1

agent1.sources = tail
agent1.channels = MemoryChannel-2
agent1.sinks = HDFS
agent1.sources.tail.type = exec
agent1.sources.tail.command = tail -F <file>
agent1.sources.tail.channels = MemoryChannel-2
agent1.sources.tail.interceptors = hostint
agent1.sources.tail.interceptors.hostint.type org.apache.flume.interceptor.HostInterceptor$Builder
agent1.sources.tail.interceptors.hostint.preserveExisting = true
agent1.sources.tail.interceptors.hostint.useIP = true
agent1.sinks.HDFS.channel = MemoryChannel-2
agent1.sinks.HDFS.type = hdfs
agent1.sinks.HDFS.hdfs.path = hdfs://url:8020/user/flume/%{host}
agent1.sinks.HDFS.hdfs.file.Type = SequenceFile
agent1.sinks.HDFS.hdfs.writeFormat = Text
agent1.sinks.HDFS.hdfs.rollInterval = 30
agent1.sinks.HDFS.hdfs.rollCount = 0
agent1.sinks.HDFS.hdfs.rollSize = 5000000000
agent1.channels.MemoryChannel-2.type = memory

more examples at http://flume.apache.org/FlumeUserGuide.html

On Tue, Dec 11, 2012 at 12:44 PM, <[EMAIL PROTECTED]> wrote:

>  Hi,****
>
> I installed flume 1.x higher version today! But is this very much
> different from the earlier. ? Is there no master / node concept here? How
> do I configure the source and sink?****
>
> ** **
>
> Think I got screwed here!****
>
> ** **
>
> *Thanks and Regards,*
>
> *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 *
>
> ** **
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* Monday, December 10, 2012 3:34 PM
>
> *To:* Haldar, Shouvanik
> *Cc:* [EMAIL PROTECTED]
> *Subject:* Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO
> com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1****
>
>  ** **
>
> just a random question, does the user running flume agent have permission
> to read the  event details out of terracotta ? ****
>
> ** **
>
> from the code here
> http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/com.cloudera/flume-core/0.9.4-cdh4.1.0/com/cloudera/flume/watchdog/Watchdog.java
> ****
>
> ** **
>
> line 209 onwards, it looks like the error condition is achieved when you
> have channel set as stdin (I am not sure about this so you may need to look
> at code which part of code you are hitting depending on the flume configs
> you have) ****
>
> ** **
>
> ** **
>
> ** **
>
> On Mon, Dec 10, 2012 at 3:01 PM, <[EMAIL PROTECTED]> wrote:**
> **
>
> Hello,****
>
>  ****
>
> I am sure, the problem is due to Terracotta. Can you please help me? ****
>
>  ****
>
> *Thanks and Regards,*****
>
> *Shouvanik Haldar | Cloud SME Pool | Mobile:+91-9830017568 *****
>
>  ****
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* Monday, December 10, 2012 2:43 PM
> *To:* Haldar, Shouvanik
> *Cc:* [EMAIL PROTECTED]****
>
>
> *Subject:* Re: Getting weird error -> 2012-12-10 01:01:42,190 INFO
> com.cloudera.flume.watchdog.Watchdog: Subprocess exited with value 1****
>
>  ****
>
> I just hope you are not hitting this
> https://issues.apache.org/jira/browse/FLUME-796****
>
> I would also like to recommend you to upgraded to flume 1.x asap ****
>
>  ****
>
> if possible can you try changing your flume source to some random file
> with tail -f and try sinking it to some file. That will narrow the problem
> is on flume side or terracotta side ****
>
>  ****
>
> Thanks,****
>
> Nitin ****
>
>  ****
>
> On Mon, Dec 10, 2012 at 1:47 PM, <[EMAIL PROTECTED]> wrote:**
> **
>
> Hi,****
>
>  ****
>
> I am using Flume -> Flume 0.9.4-cdh3u5. ****
>
>  ****
>
> Initially, the logs are ok. But when I start terracotta
> (./start-tc-server.sh) , then only I experience the above phenomena! Is it
> something to do with terracotta.!****
>
>  ****
>
>  ****
>
> 2012-12-10 03:14:08,806 INFO
> com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking

Nitin Pawar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB