Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - flume agent not starting


Copy link to this message
-
Re: flume agent not starting
Brock Noland 2014-01-15, 14:00
I'd check the hbase sink and serializer config. It's too quiet when it's
misconfigured.
On Wed, Jan 15, 2014 at 6:35 AM, Chhaya Vishwakarma <
[EMAIL PROTECTED]> wrote:

>  Hi,
>
>
>
>
>
> This is my configuration file for pushing data using flume to hbase
>
> host1.sources = src1
>
> host1.sinks = sink1
>
> host1.channels = ch1
>
>
>
> host1.sources.src1.type = exec
>
> host1.sources.src1.command = cat /home/user/Flume/flume-ng/try.log
>
> host1.sources.src1.channels = ch1
>
> host1.sources.src1.restart = true
>
>
>
> host1.sources.src1.interceptors = morphlineinterceptor
>
> host1.sources.src1.interceptors.morphlineinterceptor.type > org.apache.flume.sink.solr.morphline.MorphlineInterceptor$Builder
>
> host1.sources.src1.interceptors.morphlineinterceptor.morphlineFile > /home/user/Flume/flume-ng/morphline.conf
>
>
>
> host1.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>
> host1.sinks.sink1.channel = ch1
>
> host1.sinks.sink1.table = test
>
> host1.sinks.sink1.columnFamily = testing
>
>
>
> host1.sinks.sink1.serializer > org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>
>
>
> host1.channels.ch1.capacity=1000
>
> host1.channels.ch1.transactionCapacity=100
>
>
>
>
>
> host1.channels.ch1.type=memory
>
> When i run command
>
> *bin/flume-ng agent -n host1 -f conf/dateinter.conf
> -Dflume.root.logger=INFO,console -c conf*
>
> flume agent is not starting i can't see anything in Hbase table. Not
> showing any error also
>
> 2014-01-15 17:44:49,413 (lifecycleSupervisor-1-0) [INFO -
> org.apache.flume.node.PollingPropertiesFileConfigurationProvider.start(PollingPropertiesFileConfigurationProvider.java:61)]
> Configuration provider starting
>
> 2014-01-15 17:44:49,434 (conf-file-poller-0) [INFO -
> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:133)]
> Reloading configuration file:conf/dateinter.conf
>
> 2014-01-15 17:44:49,462 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
> Processing:sink1
>
> 2014-01-15 17:44:49,463 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
> Processing:sink1
>
> 2014-01-15 17:44:49,463 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
> Processing:sink1
>
> 2014-01-15 17:44:49,463 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:930)]
> Added sinks: sink1 Agent: host1
>
> 2014-01-15 17:44:49,463 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
> Processing:sink1
>
> 2014-01-15 17:44:49,464 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:1016)]
> Processing:sink1
>
> 2014-01-15 17:44:49,488 (conf-file-poller-0) [INFO -
> org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:140)]
> Post-validation flume configuration contains configuration for agents:
> [host1]
>
> 2014-01-15 17:44:49,489 (conf-file-poller-0) [INFO -
> org.apache.flume.node.AbstractConfigurationProvider.loadChannels(AbstractConfigurationProvider.java:150)]
> Creating channels
>
> 2014-01-15 17:44:49,499 (conf-file-poller-0) [INFO -
> org.apache.flume.channel.DefaultChannelFactory.create(DefaultChannelFactory.java:40)]
> Creating instance of channel ch1 type memory
>
> 2014-01-15 17:44:49,505 (conf-file-poller-0) [INFO -
> org.apache.flume.node.AbstractConfigurationProvider.loadChannels(AbstractConfigurationProvider.java:205)]
> Created channel ch1
>
> 2014-01-15 17:44:49,506 (conf-file-poller-0) [INFO -
> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:39)]
> Creating instance of source src1, type exec

Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org