Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Need for UDP / Multicast Source


Copy link to this message
-
Re: Need for UDP / Multicast Source
Hi Andrew,

You may try lowering transactionCapacity here.
The transactionCapacity should be set to the value of the largest batch
size that will be used to store or remove events from that channel. You
currently have it equal to the capacity of the channel. So essentially the
channel *could be* filled with one transaction depending on how you are
batching with your client.

Also it may be useful to turn up jmx monitoring and watch the channel
counters using jconsole. This way you can see exactly how many events are
placed in the channel.

To do this you will need to Set the following Java system properties
located at,
/etc/flume-ng/conf/flume-env.sh.

com.sun.management.jmxremote
com.sun.management.jmxremote.port=8081
com.sun.management.jmxremote.authenticate=false
com.sun.management.jmxremote.ssl=false

You should than be able to connect with
jconsole hostname:8081

-Jeff
On Thu, Jan 17, 2013 at 9:36 AM, Andrew Otto <[EMAIL PROTECTED]> wrote:

> > I took those two border sequence numbers and extracted all of the log
> lines generated by cp1044 on the nearby host (not using Flume).  I should
> be able to compare the number of lines here with the number of lines in the
> 12 files I extracted from HDFS and Flume.  If they are the same, then Flume
> and UDPSource is working!
>
> Oh, I meant to link to a Gist with my current flume.conf and the commands
> I executed to investigate this.  Here it is:
>
> https://gist.github.com/4557178
>
>
>
>