Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> flume agent not starting


Copy link to this message
-
Re: flume agent not starting
I'd check the hbase sink and serializer config. It's too quiet when it's
misconfigured.
On Wed, Jan 15, 2014 at 6:35 AM, Chhaya Vishwakarma <
[EMAIL PROTECTED]> wrote:

>  Hi,
>
>
>
>
>
> This is my configuration file for pushing data using flume to hbase
>
> host1.sources = src1
>
> host1.sinks = sink1
>
> host1.channels = ch1
>
>
>
> host1.sources.src1.type = exec
>
> host1.sources.src1.command = cat /home/user/Flume/flume-ng/try.log
>
> host1.sources.src1.channels = ch1
>
> host1.sources.src1.restart = true
>
>
>
> host1.sources.src1.interceptors = morphlineinterceptor
>
> host1.sources.src1.interceptors.morphlineinterceptor.type > org.apache.flume.sink.solr.morphline.MorphlineInterceptor$Builder
>
> host1.sources.src1.interceptors.morphlineinterceptor.morphlineFile > /home/user/Flume/flume-ng/morphline.conf
>
>
>
> host1.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>
> host1.sinks.sink1.channel = ch1
>
> host1.sinks.sink1.table = test
>
> host1.sinks.sink1.columnFamily = testing
>
>
>
> host1.sinks.sink1.serializer > org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>
>
>
> host1.channels.ch1.capacity=1000
>
> host1.channels.ch1.transactionCapacity=100
>
>
>
>
>
> host1.channels.ch1.type=memory
>
> When i run command
>
> *bin/flume-ng agent -n host1 -f conf/dateinter.conf
> -Dflume.root.logger=INFO,console -c conf*
>
> flume agent is not starting i can't see anything in Hbase table. Not
> showing any error also
>
> 2014-01-15 17:44:49,413 (lifecycleSupervisor-1-0) [INFO -
> org.apache.flume.node.PollingPropertiesFileConfigurationProvider.start(PollingPropertiesFileConfigurationProvider.java:61)]
> Configuration provider starting
>
> 2014-01-15 17:44:49,434 (conf-file-poller-0) [INFO -
> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:133)]
> Reloading configuration file:conf/dateinter.conf
>
> 2014-01-15 17:44:49,462 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
> Processing:sink1
>
> 2014-01-15 17:44:49,463 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
> Processing:sink1
>
> 2014-01-15 17:44:49,463 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
> Processing:sink1
>
> 2014-01-15 17:44:49,463 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:930)]
> Added sinks: sink1 Agent: host1
>
> 2014-01-15 17:44:49,463 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
> Processing:sink1
>
> 2014-01-15 17:44:49,464 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
> Processing:sink1
>
> 2014-01-15 17:44:49,488 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:140)]
> Post-validation flume configuration contains configuration for agents:
> [host1]
>
> 2014-01-15 17:44:49,489 (conf-file-poller-0) [INFO -
> org.apache.flume.node.AbstractConfigurationProvider.loadChannels(AbstractConfigurationProvider.java:150)]
> Creating channels
>
> 2014-01-15 17:44:49,499 (conf-file-poller-0) [INFO -
> org.apache.flume.channel.DefaultChannelFactory.create(DefaultChannelFactory.java:40)]
> Creating instance of channel ch1 type memory
>
> 2014-01-15 17:44:49,505 (conf-file-poller-0) [INFO -
> org.apache.flume.node.AbstractConfigurationProvider.loadChannels(AbstractConfigurationProvider.java:205)]
> Created channel ch1
>
> 2014-01-15 17:44:49,506 (conf-file-poller-0) [INFO -
> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:39)]
> Creating instance of source src1, type exec

Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB