Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Hbase Sink


Hi Prabhu,

   Have you exported HADOOP_HOME variable??

Regards,
    Mohammad Tariq
On Thu, Jul 19, 2012 at 6:34 PM, prabhu k <[EMAIL PROTECTED]> wrote:
> Hi Users,
>
> I have followed the below link for move the sample data to hbase sink. i
> then executed the below command, I am getting following error output.
> pasted the flume.log log file.
>
> Please suggest and help on this issue.
>
> http://cloudfront.blogspot.in/2012/06/how-to-move-data-into-hbase-table-using.html
>
> command
> ==========> bin/flume-ng agent -n hbase-agent -c conf/ -f conf/hbase-agent.conf
>
>
> hbase-agent.conf
> =================> hbase-agent.sources = tail
> hbase-agent.sinks = sink1
> hbase-agent.channels = ch1
> hbase-agent.sources.tail.type = exec
> hbase-agent.sources.tail.command = tail -F /usr/local/demo.txt
> hbase-agent.sources.tail.channels = ch1
> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
> hbase-agent.sinks.sink1.channel = ch1
> hbase-agent.sinks.sink1.table = demo
> hbase-agent.sinks.sink1.columnFamily = cf
> hbase-agent.sinks.sink1.serializer > org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
> hbase-agent.sinks.sink1.serializer.keyType = timestamp
> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
> hbase-agent.sinks.sink1.serializer.suffix = timestamp
> hbase-agent.channels.ch1.type=memory
>
>
> flume.log
> ================> /flume/flume-1.2.0-incubating-SNAPSHOT# more flume.log
> 2012-07-19 18:24:58,007 INFO lifecycle.LifecycleSupervisor: Starting
> lifecycle supervisor 1
> 2012-07-19 18:24:58,008 INFO node.FlumeNode: Flume node starting -
> hbase-agent
> 2012-07-19 18:24:58,011 INFO nodemanager.DefaultLogicalNodeManager: Node
> manager starting
> 2012-07-19 18:24:58,011 INFO lifecycle.LifecycleSupervisor: Starting
> lifecycle supervisor 10
> 2012-07-19 18:24:58,011 INFO properties.PropertiesFileConfigurationProvider:
> Configuration provider starting
> 2012-07-19 18:24:58,013 INFO properties.PropertiesFileConfigurationProvider:
> Reloading configuration file:conf/hbase-agent.conf
> 2012-07-19 18:24:58,019 INFO conf.FlumeConfiguration: Processing:sink1
> 2012-07-19 18:24:58,020 INFO conf.FlumeConfiguration: Processing:sink1
> 2012-07-19 18:24:58,020 INFO conf.FlumeConfiguration: Processing:sink1
> 2012-07-19 18:24:58,020 INFO conf.FlumeConfiguration: Added sinks: sink1
> Agent: hbase-agent
> 2012-07-19 18:24:58,020 INFO conf.FlumeConfiguration: Processing:sink1
> 2012-07-19 18:24:58,020 INFO conf.FlumeConfiguration: Processing:sink1
> 2012-07-19 18:24:58,020 INFO conf.FlumeConfiguration: Processing:sink1
> 2012-07-19 18:24:58,020 INFO conf.FlumeConfiguration: Processing:sink1
> 2012-07-19 18:24:58,021 INFO conf.FlumeConfiguration: Processing:sink1
> 2012-07-19 18:24:58,021 INFO conf.FlumeConfiguration: Processing:sink1
> 2012-07-19 18:24:58,034 INFO conf.FlumeConfiguration: Post-validation flume
> configuration contains configuration  for agents: [hbase-agent]
> 2012-07-19 18:24:58,034 INFO properties.PropertiesFileConfigurationProvider:
> Creating channels
> 2012-07-19 18:24:58,038 INFO properties.PropertiesFileConfigurationProvider:
> created channel ch1
> 2012-07-19 18:24:58,046 INFO sink.DefaultSinkFactory: Creating instance of
> sink sink1 typeorg.apache.flume.sink.hbase.HBaseSink
> 2012-07-19 18:24:58,051 ERROR
> properties.PropertiesFileConfigurationProvider: Failed to start agent
> because dependencies were not found in classpath. Error follows.
> java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
>         at org.apache.flume.sink.hbase.HBaseSink.<init>(HBaseSink.java:94)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:513)