Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa, mail # dev - Closing .chukwa files in collector at regular fixed offset interval to avoid time interval drift


Copy link to this message
-
Re: Closing .chukwa files in collector at regular fixed offset interval to avoid time interval drift
Ariel Rabkin 2011-05-31, 21:01
Not quite sure I understand the use case, but if it seems useful,
sure, open a JIRA.

--Ari

On Tue, May 31, 2011 at 11:33 AM, Shweta Shah <[EMAIL PROTECTED]> wrote:
> Hi,
>
> We have a system built on top of Chukwa that introduces the need to have all
> sequence files for a given rotateInterval closed and available at a
> predictable time.  Currently, we are experiencing some drift in the times
> that the sequence files are closed due to the way the TimerTask is scheduled
> in the SeqFileWriter class.  We would like to submit a solution that will
> allow people to configure the time all collectors should close their files
> for processing in a given interval, while still supporting the default
> functionality.  We have written the code and are currently testing the
> functionality, but would like to know if anyone has any feedback on this
> development before submitting a JIRA ticket.
>
> - Shweta
>

--
Ari Rabkin [EMAIL PROTECTED]
UC Berkeley Computer Science Department