You should be looking at Flume.
Its made for this
On Tue, Mar 19, 2013 at 9:03 PM, Christian Schneider <
[EMAIL PROTECTED]> wrote:
> I found out that these logs are stored directly at the TaskNodes.
> We need to have them stored over a long time (some months or better a
> year). What is a good way of doing that?
> With my current knowledge I would write a cron job that picks up all the
> files every few minutes.
> But I guess thats not the best approach...
> Best Regards,