Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # dev >> Closing .chukwa files in collector at regular fixed offset interval to avoid time interval drift


+
Shweta Shah 2011-05-31, 18:33
+
Ariel Rabkin 2011-05-31, 21:01
Copy link to this message
-
Re: Closing .chukwa files in collector at regular fixed offset interval to avoid time interval drift
Also.

This sounds like something fairly site-specific. It would be really
good to have unit tests so we don't break this
feature in the future.

--Ari

On Tue, May 31, 2011 at 2:01 PM, Ariel Rabkin <[EMAIL PROTECTED]> wrote:
> Not quite sure I understand the use case, but if it seems useful,
> sure, open a JIRA.
>
> --Ari
>
> On Tue, May 31, 2011 at 11:33 AM, Shweta Shah <[EMAIL PROTECTED]> wrote:
>> Hi,
>>
>> We have a system built on top of Chukwa that introduces the need to have all
>> sequence files for a given rotateInterval closed and available at a
>> predictable time.  Currently, we are experiencing some drift in the times
>> that the sequence files are closed due to the way the TimerTask is scheduled
>> in the SeqFileWriter class.  We would like to submit a solution that will
>> allow people to configure the time all collectors should close their files
>> for processing in a given interval, while still supporting the default
>> functionality.  We have written the code and are currently testing the
>> functionality, but would like to know if anyone has any feedback on this
>> development before submitting a JIRA ticket.
>>
>> - Shweta
>>
>
>
>
> --
> Ari Rabkin [EMAIL PROTECTED]
> UC Berkeley Computer Science Department
>

--
Ari Rabkin [EMAIL PROTECTED]
UC Berkeley Computer Science Department
+
Eric Yang 2011-05-31, 21:12
+
Bill Graham 2011-06-01, 00:16
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB