Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Memory Channel


+
Mohit Anchlia 2013-01-16, 17:46
+
Hari Shreedharan 2013-01-16, 17:55
+
Mohit Anchlia 2013-01-16, 18:21
+
Brock Noland 2013-01-16, 18:26
Copy link to this message
-
Re: Memory Channel
Right. I was asking about sync to "sink". My sink is hdfs so does flume
sync to hdfs on every write operation?

On Wed, Jan 16, 2013 at 10:26 AM, Brock Noland <[EMAIL PROTECTED]> wrote:

> Memory Channel does not write to disk and as such never syncs to disk.
> File Channel does sync to disk for each batch put on or taken off the
> channel.
>
> On Wed, Jan 16, 2013 at 10:21 AM, Mohit Anchlia <[EMAIL PROTECTED]>
> wrote:
> > Thanks! What I am really trying to understand is when does flume sync to
> the
> > sink. I am not using batch events.
> >
> >
> > On Wed, Jan 16, 2013 at 9:55 AM, Hari Shreedharan
> > <[EMAIL PROTECTED]> wrote:
> >>
> >> It means that the channel can store that many events. If it is full,
> then
> >> the put() calls (on the source side) will start throwing
> ChannelException.
> >> The put call will block only for keep-alive number of seconds, after
> which
> >> it will throw.
> >>
> >>
> >> Hari
> >>
> >> --
> >> Hari Shreedharan
> >>
> >> On Wednesday, January 16, 2013 at 9:46 AM, Mohit Anchlia wrote:
> >>
> >> Could someone help me understand capacity attribute of memoryChannel?
> Does
> >> it mean that memoryChannel flushes to sink only when this capacity is
> >> reached or does it mean that it's the max events stored in memory and
> call
> >> blocks until everything else gets freed?
> >>
> >>
> >> http://flume.apache.org/FlumeUserGuide.html#memory-channel
> >>
> >>
> >>
> >
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/
>
+
Nitin Pawar 2013-01-16, 18:55
+
Brock Noland 2013-01-16, 19:00
+
Mohit Anchlia 2013-01-16, 19:03
+
Roshan Naik 2013-01-17, 01:13
+
Juhani Connolly 2013-01-17, 02:00