Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - how to print the channel capacity


+
liuyongbo 2013-05-15, 05:37
+
Nitin Pawar 2013-05-15, 08:20
Copy link to this message
-
Re: how to print the channel capacity
Nitin Pawar 2013-05-15, 08:46
sorry pressed enter too soon

as for your question: how many events a flume agent can hold?
sorry but I don't think there is any direct answer to that.... .I may be
very well wrong there as I am myself pretty new with flume

there was a JIRA for the capacity of file channels FLUME-1571
On Wed, May 15, 2013 at 1:50 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote:

> for maximum performance on your data flow two things which will matter
> most are: the channel and the transaction batch size.
> when you say losing data, are you using memory channel? or file channel?
>
> Flume can batch events. The batch size is the maximum number of events
> that a sink or client will attempt to take from a channel in a single
> transaction.
>
>  What is the channel type
> do you have a slow sink so the # events written out are less than # event
> incoming to channels so over time it piles up
>
> others may point out more things.
> Also your flume conf and if you are seeing any errors on flume then that
> will help people to find out the problem
>
>
> On Wed, May 15, 2013 at 11:07 AM, liuyongbo <[EMAIL PROTECTED]> wrote:
>
>>  Hi:****
>>
>>          I’m using flume to pass log data to mongodb, but I find that
>> some data lose when the pressure is in high level, so I want to know the
>> max request that flume can hold and need to print the capacity.but I can
>> not find the proper way to do this instead of change the source code. Any
>> ideas?****
>>
>>          thanks****
>>
>
>
>
> --
> Nitin Pawar
>

--
Nitin Pawar
+
Nitin Pawar 2013-05-15, 08:49
+
liuyongbo 2013-05-15, 11:39
+
Nitin Pawar 2013-05-15, 11:48
+
Paul Chavez 2013-05-15, 16:36
+
Matt Wise 2013-05-15, 17:10
+
Matt Wise 2013-05-15, 18:05
+
liuyongbo 2013-05-16, 08:06