Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Ordering of messages in flume-ng


+
Chris Schneider 2014-02-12, 05:53
+
Christopher Shannon 2014-02-12, 06:02
Copy link to this message
-
Re: Ordering of messages in flume-ng
Brock Noland 2014-02-12, 16:20
Hi,

In the cast of no failures with a single source, single channel and single
sink you will see ordering. However, I believe when there is a failure file
channel will change ordering on rollback.

If strict ordering is required it's advisable to assign sequence numbers
upstream and then re-order the data with either a MR job or Impala query
once they land in MapReduce.

Brock
On Wed, Feb 12, 2014 at 12:02 AM, Christopher Shannon <[EMAIL PROTECTED]

Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org