Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> FileChannel on Windows


+
Paul Chavez 2013-04-10, 18:55
+
Brock Noland 2013-04-10, 19:07
+
Hari Shreedharan 2013-04-10, 19:08
+
Israel Ekpo 2013-04-10, 20:22
+
Israel Ekpo 2013-04-10, 19:06
+
Paul Chavez 2013-04-10, 20:25
Copy link to this message
-
Re: FileChannel on Windows
Hari,
  i recall some discussion around the desire to get rid of this dependency.
i guess it means removing that piece of code that supports the format
upgrade ? are there any plans to do that ?
-roshan
On Wed, Apr 10, 2013 at 1:25 PM, Paul Chavez <
[EMAIL PROTECTED]> wrote:

> **
> I did not realize I could use pre-built binaries on Windows. Thank you, I
> used this to grab the hadoop-common.jar from our existing Hadoop cluster
> and there are no startup errors now.
>
> Thanks to you, Brock and Hari.
>
> -Paul
>
>  ------------------------------
> *From:* Israel Ekpo [mailto:[EMAIL PROTECTED]]
> *Sent:* Wednesday, April 10, 2013 12:07 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: FileChannel on Windows
>
> Paul,
>
> If Flume 1.3.1 is what you are looking for, you don't have to build it
> from source.
>
> You can just download it directly from the site. Its already released.
>
> Paul Chavez <[EMAIL PROTECTED]> wrote:
>
> Hello,
>
> I've been using flume on Windows to send logs to an avro source for awhile
> now, using the AvroClient class. I'm now trying to get a full agent running
> to hopefully leverage the spoolDir and httpSource functionality. Part of my
> requirements is a persistent channel so I'm attempting to get a config
> running the utilizes the fileChannel.
>
> I followed the instructions at
> http://mapredit.blogspot.com/2012/07/run-flume-13x-on-windows.html to
> build flume, same as I did for the AvroClient. However it seems I need some
> HDFS libraries. Can anyone point me to instructions for building the JAR(s)
> I need to get this working?
>
> Currently I'm running into the following exception:
> 10 Apr 2013 11:50:29,317 INFO  [lifecycleSupervisor-1-0]
> (org.apache.flume.channel.file.FileChannel.start:285)  - Starting
> FileChannel fc_Default { dataDirs: [c:\flume_data\log] }...
> 10 Apr 2013 11:50:29,317 ERROR [lifecycleSupervisor-1-0]
> (org.apache.flume.channel.file.FileChannel.start:314)  - Failed to start
> the file channel [channel=fc_Default]
> java.lang.NoClassDefFoundError: org/apache/hadoop/io/Writable
>  at java.lang.ClassLoader.defineClass1(Native Method)
>  at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
>  at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
>  at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
>  at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
>  at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
>  at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>  at org.apache.flume.channel.file.Log$Builder.build(Log.java:212)
>  at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:302)
>  at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>  at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>  at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>  at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
>  at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>  at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>  at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>  at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.io.Writable
+
Hari Shreedharan 2013-04-10, 20:41
+
Brock Noland 2013-04-10, 20:44
+
Hari Shreedharan 2013-04-10, 20:58
+
Israel Ekpo 2013-04-10, 21:20
+
Brock Noland 2013-04-10, 21:32
+
Israel Ekpo 2013-04-10, 21:45