Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # dev >> Closing .chukwa files in collector at regular fixed offset interval to avoid time interval drift


Copy link to this message
-
Re: Closing .chukwa files in collector at regular fixed offset interval to avoid time interval drift
Not quite sure I understand the use case, but if it seems useful,
sure, open a JIRA.

--Ari

On Tue, May 31, 2011 at 11:33 AM, Shweta Shah <[EMAIL PROTECTED]> wrote:
> Hi,
>
> We have a system built on top of Chukwa that introduces the need to have all
> sequence files for a given rotateInterval closed and available at a
> predictable time.  Currently, we are experiencing some drift in the times
> that the sequence files are closed due to the way the TimerTask is scheduled
> in the SeqFileWriter class.  We would like to submit a solution that will
> allow people to configure the time all collectors should close their files
> for processing in a given interval, while still supporting the default
> functionality.  We have written the code and are currently testing the
> functionality, but would like to know if anyone has any feedback on this
> development before submitting a JIRA ticket.
>
> - Shweta
>

--
Ari Rabkin [EMAIL PROTECTED]
UC Berkeley Computer Science Department
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB