Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> How to load zip file into hdfs sink using flume-ng


Copy link to this message
-
Re: How to load zip file into hdfs sink using flume-ng
Actually it might be tricky to use the directory spooling source to read a
compressed archive. It's possible, but you would definitely need to write
your own deserializer.

Flume is an event-oriented streaming system, it's not really optimized to
be a plain file transfer mechanism like FTP.

Regards,
Mike
On Fri, Feb 8, 2013 at 9:03 AM, Nitin Pawar <[EMAIL PROTECTED]> wrote:

> priyanka,
> you can do that flume-ng-1.3 using directory spooling
>
> just be aware what kind of channel you use
>
>
> On Fri, Feb 8, 2013 at 10:06 PM, Priyanka Jain <[EMAIL PROTECTED]
> > wrote:
>
>> Hi,,,
>> I want to know is it possible to load the zip file from remote location
>> to the hdfs sink in flume-ng-1.2
>>
>> Thanks in advance,
>> Priyanka
>>
>
>
>
> --
> Nitin Pawar
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB