Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # dev - Review Request: FLUME-2014: Race condition when using local timestamp with BucketPath


+
Mike Percy 2013-04-22, 08:02
+
Hari Shreedharan 2013-04-22, 19:00
Copy link to this message
-
Re: Review Request: FLUME-2014: Race condition when using local timestamp with BucketPath
Mike Percy 2013-04-22, 21:27

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10699/
-----------------------------------------------------------

(Updated April 22, 2013, 9:27 p.m.)
Review request for Flume.
Changes
-------

Good catch Hari! Fixed.

We need unit tests for the timeZone stuff, I filed FLUME-2018 to track that.
Description
-------

This patch changes the behavior of the BucketPath.escapeString() function to only ever call clock.currentTimeMillis() once during an invocation of the method. This prevents a race condition that can cause unexpected results in the interpolated paths.
This addresses bug FLUME-2014.
    https://issues.apache.org/jira/browse/FLUME-2014
Diffs (updated)
-----

  flume-ng-core/src/main/java/org/apache/flume/formatter/output/BucketPath.java 971c75c
  flume-ng-core/src/test/java/org/apache/flume/formatter/output/TestBucketPath.java 9cfefc0

Diff: https://reviews.apache.org/r/10699/diff/
Testing
-------

Added unit test for the previous condition. All tests pass.

Here is some output from running the new unit test without the patch:

Running org.apache.flume.formatter.output.TestBucketPath
Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.87 sec <<< FAILURE!

Results :

Failed tests:   testDateRace(org.apache.flume.formatter.output.TestBucketPath): Race condition detected expected:<02:[5]0> but was:<02:[0]0>

Tests run: 6, Failures: 1, Errors: 0, Skipped: 0
Thanks,

Mike Percy

+
Hari Shreedharan 2013-04-22, 21:46