Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> FLumeNG -HDFS errors


Copy link to this message
-
Re: FLumeNG -HDFS errors
HI,

Your Hadoop libraries are not in classpath, from where you run Flume.

The following should work: Please change the paths to your respective
installations,

1)

C:\Users>set CLASSPATH=.;D:\Source\my-flume\my-flume;D:\Source\apache-
flume-1.3.0-SNAPSHOT-dist\apache-flume-1.3.0-SNAPSHOT\lib\*;D:\Source\hadoop-1.0
.3\hadoop-1.0.3\*;D:\Source\hadoop-1.0.3\hadoop-1.0.3\lib\*

2)

C:\Users>java org.apache.flume.node.Application -f D:\Source\my-flume\
my-flume\flume-conf.properties -n host1
Best Regards,
On Fri, Oct 12, 2012 at 3:41 PM, Rewti Ingle
<[EMAIL PROTECTED]>wrote:

>   Hi, ** ****
>
> ** **
>
> I have installed FlumeNG [1.2] and Hadoop [1.0.3].****
>
> ** **
>
> I am getting following error  while running flume agent with HDFS sink.***
> *
>
> ** **
>
> 2012-10-11 16:55:57,640 WARN hdfs.HDFSEventSink: HDFS IO error****
>
> java.io.IOException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException:
> org.apache.hadoop.hdfs.DistributedFileSystem****
>
>                 at
> org.apache.flume.sink.hdfs.BucketWriter.doOpen(BucketWriter.java:202)****
>
>                 at
> org.apache.flume.sink.hdfs.BucketWriter.access$000(BucketWriter.java:48)**
> **
>
>                 at
> org.apache.flume.sink.hdfs.BucketWriter$1.run(BucketWriter.java:155)****
>
>                 at
> org.apache.flume.sink.hdfs.BucketWriter$1.run(BucketWriter.java:152)****
>
>                 at
> org.apache.flume.sink.hdfs.BucketWriter.runPrivileged(BucketWriter.java:125)
> ****
>
>                 at
> org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:152)****
>
>                 at
> org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:307)****
>
>                 at
> org.apache.flume.sink.hdfs.HDFSEventSink$1.call(HDFSEventSink.java:717)***
> *
>
>                 at
> org.apache.flume.sink.hdfs.HDFSEventSink$1.call(HDFSEventSink.java:714)***
> *
>
>                 at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)****
>
>                 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ****
>
>                 at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> ****
>
>                 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> ****
>
>                 at java.lang.Thread.run(Thread.java:662)****
>
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.apache.hadoop.hdfs.DistributedFileSystem****
>
>                 at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1128)****
>
>                 at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1729)****
>
>                 at
> org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:74)****
>
>                 at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1768)***
> *
>
>                 at
> org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1750)****
>
>                 at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:234)
> ****
>
>                 at org.apache.hadoop.fs.Path.getFileSystem(Path.java:189)*
> ***
>
>                 at
> org.apache.flume.sink.hdfs.BucketWriter.doOpen(BucketWriter.java:186)****
>
>                 ... 13 more****
>
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hdfs.DistributedFileSystem****
>
>                 at java.net.URLClassLoader$1.run(URLClassLoader.java:202)*
> ***
>
>                 at java.security.AccessController.doPrivileged(Native
> Method)****
>
>                 at
> java.net.URLClassLoader.findClass(URLClassLoader.java:190)****
>
>                 at java.lang.ClassLoader.loadClass(ClassLoader.java:306)**
> **
>
>                 at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)****
>
>                 at java.lang.ClassLoader.loadClass(ClassLoader.java:247)**
> **
>
>                 at java.lang.Class.forName0(Native Method)****
>
>                 at java.lang.Class.forName(Class.java:247)****
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB