Hi!

I have, what I would imagine, is a pretty common use case -- a bunch of S3
buckets with access logging enabled. S3 is nice in a sense that the logging
for the bucket all goes back to a bucket of your choice, but of course, I really
want this data in HDFS so I can analyze it.

Is there a flume source that can help me here? I couldn't find one.

Thanks,
Roman.

Roman,
I haven't seen an implementation floating around, but I would love to get
support for an S3 Source into Flume!

Regards,
Mike

On Thu, Aug 23, 2012 at 5:06 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:

> Hi!
>
> I have, what I would imagine, is a pretty common use case -- a bunch of S3
> buckets with access logging enabled. S3 is nice in a sense that the logging
> for the bucket all goes back to a bucket of your choice, but of course, I
> really
> want this data in HDFS so I can analyze it.
>
> Is there a flume source that can help me here? I couldn't find one.
>
> Thanks,
> Roman.
>

There is a related discussion at

http://mail-archives.apache.org/mod_mbox/flume-user/201208.mbox/%[EMAIL PROTECTED]%3E
Thanks,
Mubarak
On Aug 23, 2012, at 5:44 PM, Mike Percy wrote:

> Roman,
> I haven't seen an implementation floating around, but I would love to get support for an S3 Source into Flume!
>
> Regards,
> Mike
>
> On Thu, Aug 23, 2012 at 5:06 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> Hi!
>
> I have, what I would imagine, is a pretty common use case -- a bunch of S3
> buckets with access logging enabled. S3 is nice in a sense that the logging
> for the bucket all goes back to a bucket of your choice, but of course, I really
> want this data in HDFS so I can analyze it.
>
> Is there a flume source that can help me here? I couldn't find one.
>
> Thanks,
> Roman.
>