Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Constant Traffic on port 35872


Copy link to this message
-
Re: Constant Traffic on port 35872
I doubt it's the Thrift RPC layer. It's most likely the app.
On Wed, Jan 16, 2013 at 5:53 PM, James Stewart <[EMAIL PROTECTED]
> wrote:

> I thought it was only ‘heartbeats’ that were supposed to go via port
> 35872, so I reduced flume.config.heartbeat.period on the nodes to 60 sec.
> According to the master it’s only seeing heartbeats every 60 sec now, and
> yet I still get constantly spammed with data on port 35872 from every node.
> ****
>
> ** **
>
> It does look like metric collection or config reporting of some kind, like
> it’s reporting the configuration of the sources/sinks and even data about
> the JVM:****
>
> ** **
>
> ............rt.starttime....Thu Jan 17 11:47:07 EST 2013...
> rt.vmname...!Java HotSpot(TM) 64-Bit Server
> VM....name...(pn-opsynxsr0202.aus.optiver.com.jvm-Info....rt.vmversion....16.3-b01....rt.vmvendor....Sun
> Microsystems Inc.****
>
> ..........max.....@
> ......mem.other.used................mem.other.committed.....s......mem.heap.max...........
> ****
>
> ** **
>
> But it’s just the same data over and over again every second. This traffic
> is travelling across a WAN and with a lot of nodes it’s a significant
> enough amount of data to be a problem.****
>
> ** **
>
> I don’t know much about Java, but could this be something to do with
> Thrift?****
>
> ** **
>
> ** **
>
> *From:* Mike Percy [mailto:[EMAIL PROTECTED]]
> *Sent:* Thursday, 17 January 2013 12:19 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: Constant Traffic on port 35872****
>
> ** **
>
> I know next to nothing about Flume OG but if I had to guess I'd say it's
> either a heartbeat or metrics collection. Why do you want it to stop?****
>
> ** **
>
> On Wed, Jan 16, 2013 at 5:06 PM, James Stewart <
> [EMAIL PROTECTED]> wrote:****
>
> Hello all,****
>
>  ****
>
> I’m using flume 0.9.4 – before anybody mentions it, we aren’t in a
> position to upgrade at the moment due to custom decorators + sinks.****
>
>  ****
>
> I’m seeing constant traffic from my various flume nodes back to my master
> on port 35872. Even after increasing my timeout period to 60 sec and
> disabling all custom sources/sinks/decorators, I am still constantly
> receiving packets from all of my nodes back to my master. I have included a
> dump of the tcp packets below – I receive this same traffic from every node
> every 1-2 sec.****
>
>  ****
>
> Does anybody have any idea what is generating this traffic and how I can
> stop it?****
>
>  ****
>
> Many thanks,****
>
>  ****
>
> James****
>
>  ****
>
>  ****
>
> V..?..-MLogicalNodeManager.my_logical_node_apps.sourceConfig....null...1LogicalNodeManager.my_logical_node_apps.physicalnode....my_physical_node.my_domain.com...:LogicalNodeManager.my_physical_node.my_domain.com.sinkConfig....null...-LogicalNodeManager.my_logical_node_apps.hostname....my_physical_node.my_domain.com....AgentWALAckManager.name....AgentWALAckManager...8LogicalNodeManager.my_physical_node.my_domain.com.nodename....my_physical_node.my_domain.com...<LogicalNodeManager.my_physical_node.my_domain.com.sourceConfig....null....LogicalNodeManager.name....LogicalNodeManager...)LogicalNodeManager.my_logical_node_apps.name....my_logical_node_apps.../LogicalNodeManager.my_logical_node_apps.sinkConfig....null...*LogicalNodeManager.my_logical_node_apps.state....IDLE...8LogicalNodeManager.my_physical_node.my_domain.com.hostname....my_physical_node.my_domain.com...-LogicalNodeManager.my_logical_node_apps.nodename....my_logical_node_apps...9LogicalNodeManager.my_physical_node.my_domain.com.sink.name....NullSink....name....pn-my_physical_node.my_domain.com....LogicalNodeManager.my_logical_node_apps.sink.name....NullSink....LogicalNodeManager.my_logical_node_apps.sink.type....NullSink...7LogicalNodeManager.my_physical_node.my_domain.com.version....Thu
> Jan 01 10:00:00 EST
> 1970...9LogicalNodeManager.my_physical_node.my_domain.com.sink.type....NullSink...5LogicalNodeManager.my_physical_node.my_domain.com.state....IDLE...,LogicalNodeManager.my_logical_node_apps.version....Thu
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB