Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Memory Channel


+
Mohit Anchlia 2013-01-16, 17:46
+
Hari Shreedharan 2013-01-16, 17:55
+
Mohit Anchlia 2013-01-16, 18:21
+
Brock Noland 2013-01-16, 18:26
+
Mohit Anchlia 2013-01-16, 18:47
Copy link to this message
-
Re: Memory Channel
Nitin Pawar 2013-01-16, 18:55
you can configure it as you nee
number of events
rollover by time
and other ways as well
On Thu, Jan 17, 2013 at 12:17 AM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:

> Right. I was asking about sync to "sink". My sink is hdfs so does flume
> sync to hdfs on every write operation?
>
>
> On Wed, Jan 16, 2013 at 10:26 AM, Brock Noland <[EMAIL PROTECTED]> wrote:
>
>> Memory Channel does not write to disk and as such never syncs to disk.
>> File Channel does sync to disk for each batch put on or taken off the
>> channel.
>>
>> On Wed, Jan 16, 2013 at 10:21 AM, Mohit Anchlia <[EMAIL PROTECTED]>
>> wrote:
>> > Thanks! What I am really trying to understand is when does flume sync
>> to the
>> > sink. I am not using batch events.
>> >
>> >
>> > On Wed, Jan 16, 2013 at 9:55 AM, Hari Shreedharan
>> > <[EMAIL PROTECTED]> wrote:
>> >>
>> >> It means that the channel can store that many events. If it is full,
>> then
>> >> the put() calls (on the source side) will start throwing
>> ChannelException.
>> >> The put call will block only for keep-alive number of seconds, after
>> which
>> >> it will throw.
>> >>
>> >>
>> >> Hari
>> >>
>> >> --
>> >> Hari Shreedharan
>> >>
>> >> On Wednesday, January 16, 2013 at 9:46 AM, Mohit Anchlia wrote:
>> >>
>> >> Could someone help me understand capacity attribute of memoryChannel?
>> Does
>> >> it mean that memoryChannel flushes to sink only when this capacity is
>> >> reached or does it mean that it's the max events stored in memory and
>> call
>> >> blocks until everything else gets freed?
>> >>
>> >>
>> >> http://flume.apache.org/FlumeUserGuide.html#memory-channel
>> >>
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> Apache MRUnit - Unit testing MapReduce -
>> http://incubator.apache.org/mrunit/
>>
>
>
--
Nitin Pawar
+
Brock Noland 2013-01-16, 19:00
+
Mohit Anchlia 2013-01-16, 19:03
+
Roshan Naik 2013-01-17, 01:13
+
Juhani Connolly 2013-01-17, 02:00