Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> FileChannel on Windows


+
Paul Chavez 2013-04-10, 18:55
Copy link to this message
-
Re: FileChannel on Windows
Hi,

You are missing the hadoop-core.jar.

Brock
On Wed, Apr 10, 2013 at 1:55 PM, Paul Chavez <
[EMAIL PROTECTED]> wrote:

> **
> Hello,
>
> I've been using flume on Windows to send logs to an avro source for awhile
> now, using the AvroClient class. I'm now trying to get a full agent running
> to hopefully leverage the spoolDir and httpSource functionality. Part of my
> requirements is a persistent channel so I'm attempting to get a config
> running the utilizes the fileChannel.
>
> I followed the instructions at
> http://mapredit.blogspot.com/2012/07/run-flume-13x-on-windows.html to
> build flume, same as I did for the AvroClient. However it seems I need some
> HDFS libraries. Can anyone point me to instructions for building the JAR(s)
> I need to get this working?
>
> Currently I'm running into the following exception:
> 10 Apr 2013 11:50:29,317 INFO  [lifecycleSupervisor-1-0]
> (org.apache.flume.channel.file.FileChannel.start:285)  - Starting
> FileChannel fc_Default { dataDirs: [c:\flume_data\log] }...
> 10 Apr 2013 11:50:29,317 ERROR [lifecycleSupervisor-1-0]
> (org.apache.flume.channel.file.FileChannel.start:314)  - Failed to start
> the file channel [channel=fc_Default]
> java.lang.NoClassDefFoundError: org/apache/hadoop/io/Writable
>  at java.lang.ClassLoader.defineClass1(Native Method)
>  at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
>  at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
>  at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
>  at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
>  at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
>  at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>  at org.apache.flume.channel.file.Log$Builder.build(Log.java:212)
>  at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:302)
>  at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>  at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>  at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>  at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
>  at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>  at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>  at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>  at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.io.Writable
>  at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>  ... 24 more
> 10 Apr 2013 11:50:29,317 ERROR [lifecycleSupervisor-1-0]
> (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:238)  -
> Unable to start FileChannel fc_Default { dataDirs: [c:\flume_data\log] } -
> Exception follows.
> java.lang.NoClassDefFoundError: org/apache/hadoop/io/Writable
>  at java.lang.ClassLoader.defineClass1(Native Method)
>  at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)

Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
+
Hari Shreedharan 2013-04-10, 19:08
+
Israel Ekpo 2013-04-10, 20:22
+
Israel Ekpo 2013-04-10, 19:06
+
Paul Chavez 2013-04-10, 20:25
+
Roshan Naik 2013-04-10, 20:38
+
Hari Shreedharan 2013-04-10, 20:41
+
Brock Noland 2013-04-10, 20:44
+
Hari Shreedharan 2013-04-10, 20:58
+
Israel Ekpo 2013-04-10, 21:20
+
Brock Noland 2013-04-10, 21:32
+
Israel Ekpo 2013-04-10, 21:45
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB