Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Max events per RPC call from across sink  to Avro source


+
Madhu Gmail 2013-04-16, 21:16
+
Madhu Gmail 2013-04-16, 21:45
+
Chris Neal 2013-04-16, 21:45
Copy link to this message
-
Re: Max events per RPC call from across sink to Avro source
I found the issue with my configuration.

I had batchSize  instead of batch-size

It was the property name for batch size.

Only for Avro sink,  it should be batch-size.  But every where else it is  batchSize.

Madhu  Munagala
(214)679-2872

On Apr 16, 2013, at 4:45 PM, Chris Neal <[EMAIL PROTECTED]> wrote:

> I believe that comment is indicating the default value, not the max.  Perhaps your config is not correct?  
>
>
> On Tue, Apr 16, 2013 at 4:16 PM, Madhu Gmail <[EMAIL PROTECTED]> wrote:
>> Hello,
>>
>> I have flume agent configuration with sink batch size as 900.
>>
>> Also I have another  flume agent working as collector.
>>
>> Even though I have 900 as batch-size on agent sink,  I can only see 100 event batch when the sink gets the data from file channel and sends it to collector.
>>
>> I also see below comment in the code where it says max is 100 per RPC call
>>
>> <td><tt>batch-size</tt></td>
>> * <td>The maximum number of events to send per RPC.</td>
>> * <td>events (int)</td>
>> * <td>100</td>
>>
>> Madhu  Munagala
>> (214)679-2872
>
+
Alexander Alten-Lorenz 2013-04-17, 05:44
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB