Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> How to load zip file into hdfs sink using flume-ng


+
Priyanka Jain 2013-02-08, 16:36
+
Nitin Pawar 2013-02-08, 17:03
Copy link to this message
-
Re: How to load zip file into hdfs sink using flume-ng
Actually it might be tricky to use the directory spooling source to read a
compressed archive. It's possible, but you would definitely need to write
your own deserializer.

Flume is an event-oriented streaming system, it's not really optimized to
be a plain file transfer mechanism like FTP.

Regards,
Mike
On Fri, Feb 8, 2013 at 9:03 AM, Nitin Pawar <[EMAIL PROTECTED]> wrote:

> priyanka,
> you can do that flume-ng-1.3 using directory spooling
>
> just be aware what kind of channel you use
>
>
> On Fri, Feb 8, 2013 at 10:06 PM, Priyanka Jain <[EMAIL PROTECTED]
> > wrote:
>
>> Hi,,,
>> I want to know is it possible to load the zip file from remote location
>> to the hdfs sink in flume-ng-1.2
>>
>> Thanks in advance,
>> Priyanka
>>
>
>
>
> --
> Nitin Pawar
>