Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Max events per RPC call from across sink  to Avro source


Copy link to this message
-
Re: Max events per RPC call from across sink to Avro source
> Only for Avro sink,  it should be batch-size.  But every where else it is  batchSize.
Hmm, indeed. I filed a jira in your behalf (https://issues.apache.org/jira/browse/FLUME-2006) , we're happy to fix this small issue.

- Alex

On Apr 17, 2013, at 1:17 AM, Madhu Gmail <[EMAIL PROTECTED]> wrote:

> I found the issue with my configuration.
>
> I had batchSize  instead of batch-size
>
> It was the property name for batch size.
>
> Only for Avro sink,  it should be batch-size.  But every where else it is  batchSize.
>
>
>
> Madhu  Munagala
> (214)679-2872
>
> On Apr 16, 2013, at 4:45 PM, Chris Neal <[EMAIL PROTECTED]> wrote:
>
>> I believe that comment is indicating the default value, not the max.  Perhaps your config is not correct?  
>>
>>
>> On Tue, Apr 16, 2013 at 4:16 PM, Madhu Gmail <[EMAIL PROTECTED]> wrote:
>> Hello,
>>
>> I have flume agent configuration with sink batch size as 900.
>>
>> Also I have another  flume agent working as collector.
>>
>> Even though I have 900 as batch-size on agent sink,  I can only see 100 event batch when the sink gets the data from file channel and sends it to collector.
>>
>> I also see below comment in the code where it says max is 100 per RPC call
>>
>> <td><tt>batch-size</tt></td>
>> * <td>The maximum number of events to send per RPC.</td>
>> * <td>events (int)</td>
>> * <td>100</td>
>>
>> Madhu  Munagala
>> (214)679-2872
>>

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB