Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Including Hbase dependencies
iain wright
2012-10-05, 18:51
Mike Percy
2012-10-07, 04:34
iain wright
2012-10-08, 06:15
iain wright
2012-10-08, 06:57
+
iain wright 2012-10-05, 18:51
+
Mike Percy 2012-10-07, 04:34
+
iain wright 2012-10-08, 06:15
Copy link to this message
-
Re: Including Hbase dependencies
Hari Shreedharan 2012-10-08, 06:42
s/org.apache.flume.sink.hbase.SimpleHBaseEventSerializer/org.apache.flume.sink.hbase.SimpleHbaseEventSerializer. I am sorry for not following the same naming convention.

I also suggest using the AsyncHBaseSink. The AsyncHBaseSink provides way better performance. The only reason HBaseSink itself is not deprecated is that it uses the "official" HBase client API. But as of now, the AsyncHBaseSink is recommended over HBaseSink.

Thanks,
Hari
--
Hari Shreedharan
On Sunday, October 7, 2012 at 11:15 PM, iain wright wrote:

> Thanks again Mike -
>
> HADOOP_HOME & HBASE_HOME in the env script did the trick.
>
> Now we're getting stuck on the class for the serializer not being found, which i see present, I'm guessing its just another PATH im missing:
> [ERROR - org.apache.flume.sink.hbase.HBaseSink.configure(HBaseSink.java:176)] Could not instantiate event serializer.
> java.lang.ClassNotFoundException: org.apache.flume.sink.hbase.SimpleHBaseEventSerializer
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
>     at java.lang.Class.forName0(Native Method)
>     at java.lang.Class.forName(Class.java:186)
>     at org.apache.flume.sink.hbase.HBaseSink.configure(HBaseSink.java:170)
>     at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
>     at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSinks(PropertiesFileConfigurationProvider.java:373)
>     at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:223)
>     at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
>     at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
>     at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>     at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>     at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>     at java.lang.Thread.run(Thread.java:722)
>
> sink exists:
>
> $/app/flume/flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase]$ ls -lah
> total 58
> drwxrwxrwx  2 root  wheel    11B Oct  3 16:36 .
> drwxrwxrwx  3 root  wheel     3B Oct  3 16:36 ..
> -rwxrwxrwx  1 root  wheel    15k Oct  3 16:36 AsyncHBaseSink.java
> -rwxrwxrwx  1 root  wheel   2.6k Oct  3 16:36 AsyncHbaseEventSerializer.java
> -rwxrwxrwx  1 root  wheel   9.1k Oct  3 16:36 HBaseSink.java
> -rwxrwxrwx  1 root  wheel   1.8k Oct  3 16:36 HBaseSinkConfigurationConstants.java
> -rwxrwxrwx  1 root  wheel   2.2k Oct  3 16:36 HbaseEventSerializer.java
> -rwxrwxrwx  1 root  wheel   5.9k Oct  3 16:36 RegexHbaseEventSerializer.java
> -rwxrwxrwx  1 root  wheel   4.8k Oct  3 16:36 SimpleAsyncHbaseEventSerializer.java
> -rwxrwxrwx  1 root  wheel   4.6k Oct  3 16:36 SimpleHbaseEventSerializer.java
> -rwxrwxrwx  1 root  wheel   1.8k Oct  3 16:36 SimpleRowKeyGenerator.java
>
>
> Startup used:
>  $/app/apache-flume-1.3.0-SNAPSHOT/bin/flume-ng agent -n agent1 -c ./conf -f conf/brian.properties  -Dflume.root.logger=INFO,console
 (http://www.labctsi.org/)> This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from disclosure under applicable law. If you are not the intended recipient, do not disclose or disseminate the message to anyone except the intended recipient. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender by return email, and delete all copies of this message.
 (http://www.labctsi.org/)> > > This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from disclosure under applicable law. If you are not the intended recipient, do not disclose or disseminate the message to anyone except the intended recipient. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender by return email, and delete all copies of this message.
+
iain wright 2012-10-08, 06:57