Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> FileChannel on Windows


Copy link to this message
-
Re: FileChannel on Windows
http://issues.apache.org/jira/browse/FLUME-1285
On Wed, Apr 10, 2013 at 4:20 PM, Israel Ekpo <[EMAIL PROTECTED]> wrote:

> Brock, do you have the JIRA issue where the patches were uploaded?
>
> If I get some spare time this weekend I can take a shot at it.
>
>
>
> On 10 April 2013 16:44, Brock Noland <[EMAIL PROTECTED]> wrote:
>
>> I think we just need to copy 4-5 classes over from hadoop. Someone
>> uploaded a patch I just haven't had time to work with it.
>>
>>
>> On Wed, Apr 10, 2013 at 3:41 PM, Hari Shreedharan <
>> [EMAIL PROTECTED]> wrote:
>>
>>>  Yes. We will need to remove the format upgrade code and the old format
>>> code from the file channel as well. I think it should be possible to do it,
>>> but I suspect it is just that no one had the time to look at this as yet.
>>>
>>> --
>>> Hari Shreedharan
>>>
>>> On Wednesday, April 10, 2013 at 1:38 PM, Roshan Naik wrote:
>>>
>>> Hari,
>>>   i recall some discussion around the desire to get rid of this
>>> dependency. i guess it means removing that piece of code that supports the
>>> format upgrade ? are there any plans to do that ?
>>>  -roshan
>>>
>>>
>>> On Wed, Apr 10, 2013 at 1:25 PM, Paul Chavez <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>> **
>>> I did not realize I could use pre-built binaries on Windows. Thank you,
>>> I used this to grab the hadoop-common.jar from our existing Hadoop cluster
>>> and there are no startup errors now.
>>>
>>> Thanks to you, Brock and Hari.
>>>
>>> -Paul
>>>
>>>  ------------------------------
>>> *From:* Israel Ekpo [mailto:[EMAIL PROTECTED]]
>>> *Sent:* Wednesday, April 10, 2013 12:07 PM
>>> *To:* [EMAIL PROTECTED]
>>> *Subject:* Re: FileChannel on Windows
>>>
>>> Paul,
>>>
>>> If Flume 1.3.1 is what you are looking for, you don't have to build it
>>> from source.
>>>
>>> You can just download it directly from the site. Its already released.
>>>
>>> Paul Chavez <[EMAIL PROTECTED]> wrote:
>>>
>>> Hello,
>>>
>>> I've been using flume on Windows to send logs to an avro source for
>>> awhile now, using the AvroClient class. I'm now trying to get a full agent
>>> running to hopefully leverage the spoolDir and httpSource functionality.
>>> Part of my requirements is a persistent channel so I'm attempting to get a
>>> config running the utilizes the fileChannel.
>>>
>>> I followed the instructions at
>>> http://mapredit.blogspot.com/2012/07/run-flume-13x-on-windows.html to
>>> build flume, same as I did for the AvroClient. However it seems I need some
>>> HDFS libraries. Can anyone point me to instructions for building the JAR(s)
>>> I need to get this working?
>>>
>>> Currently I'm running into the following exception:
>>> 10 Apr 2013 11:50:29,317 INFO  [lifecycleSupervisor-1-0]
>>> (org.apache.flume.channel.file.FileChannel.start:285)  - Starting
>>> FileChannel fc_Default { dataDirs: [c:\flume_data\log] }...
>>> 10 Apr 2013 11:50:29,317 ERROR [lifecycleSupervisor-1-0]
>>> (org.apache.flume.channel.file.FileChannel.start:314)  - Failed to start
>>> the file channel [channel=fc_Default]
>>> java.lang.NoClassDefFoundError: org/apache/hadoop/io/Writable
>>>  at java.lang.ClassLoader.defineClass1(Native Method)
>>>  at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
>>>  at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
>>>  at
>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
>>>  at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
>>>  at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
>>>  at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
>>>  at java.security.AccessController.doPrivileged(Native Method)
>>>  at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>>  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>>  at org.apache.flume.channel.file.Log$Builder.build(Log.java:212)
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB