Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Best way to collect Hadoop logs across cluster

Mark Kerzner 2013-04-19, 04:23
Roman Shaposhnik 2013-04-19, 04:44
Mark Kerzner 2013-04-26, 00:08
Copy link to this message
Re: Best way to collect Hadoop logs across cluster
When you destroy an EC2 instance, the correct behavior is to erase all
Why don't you create a service to collect the logs directly to a S3 bucket
in real-time or in a batch of 5 mins?
2013/4/18 Mark Kerzner <[EMAIL PROTECTED]>

> Hi,
> my clusters are on EC2, and they disappear after the cluster's instances
> are destroyed. What is the best practice to collect the logs for later
> storage?
> EC2 does exactly that with their EMR, how do they do it?
> Thank you,
> Mark

Marcos Ortiz Valmaseda,
*Data-Driven Product Manager* at PDVSA
*Blog*: http://dataddict.wordpress.com/
*LinkedIn: *http://www.linkedin.com/in/marcosluis2186
*Twitter*: @marcosluis2186 <http://twitter.com/marcosluis2186>
Mark Kerzner 2013-04-19, 05:01
Marcos Luis Ortiz Valmase... 2013-04-19, 05:55