Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - how to print the channel capacity


+
liuyongbo 2013-05-15, 05:37
Copy link to this message
-
Re: how to print the channel capacity
Nitin Pawar 2013-05-15, 08:20
for maximum performance on your data flow two things which will matter most
are: the channel and the transaction batch size.
when you say losing data, are you using memory channel? or file channel?

Flume can batch events. The batch size is the maximum number of events that
a sink or client will attempt to take from a channel in a single
transaction.

 What is the channel type
do you have a slow sink so the # events written out are less than # event
incoming to channels so over time it piles up

others may point out more things.
Also your flume conf and if you are seeing any errors on flume then that
will help people to find out the problem
On Wed, May 15, 2013 at 11:07 AM, liuyongbo <[EMAIL PROTECTED]> wrote:

>  Hi:****
>
>          I’m using flume to pass log data to mongodb, but I find that some
> data lose when the pressure is in high level, so I want to know the max
> request that flume can hold and need to print the capacity.but I can not
> find the proper way to do this instead of change the source code. Any ideas?
> ****
>
>          thanks****
>

--
Nitin Pawar
+
Nitin Pawar 2013-05-15, 08:46
+
Nitin Pawar 2013-05-15, 08:49
+
liuyongbo 2013-05-15, 11:39
+
Nitin Pawar 2013-05-15, 11:48
+
Paul Chavez 2013-05-15, 16:36
+
Matt Wise 2013-05-15, 17:10
+
Matt Wise 2013-05-15, 18:05
+
liuyongbo 2013-05-16, 08:06