Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> avro client fails to connect to flume


Copy link to this message
-
Re: avro client fails to connect to flume
Please send your logs also. It will help us understand if there is an
error during startup.

Hari

On Thu, Nov 1, 2012 at 6:44 PM, Siva Gudavalli
<[EMAIL PROTECTED]> wrote:
> hi Roshan,
>
> here is the config....
>
> agent.sources = avro-AppSrv-source
> agent.sinks = hdfs-Cluster-sink
> agent.channels = mem-channel
>
> # set channel for sources, sinks
> agent.sources.avro-AppSrv-source.channels = mem-channel
> agent.sources.hdfs-Cluster-sink.channel = mem-channel
>
> # set source properties
> agent.sources.avro-AppSrv-source.type = avro
> agent.sources.avro-AppSrv-source.bind = localhost
> agent.sources.avro-AppSrv-source.port = 41414
>
> # set channel properties
> agent.channels.mem-channel.type = memory
> agent.channels.mem-channel.capacity = 10000
> agent.channels.mem-channel.transactionCapacity = 10000
>
> # set sink properties
> agent.sinks.hdfs-Cluster-sink.type = hdfs
> agent.sinks.hdfs-Cluster-sink.hdfs.path = hdfs://localhost:54310/flume
>
> ________________________________________
> From: Roshan Naik [[EMAIL PROTECTED]]
> Sent: Friday, November 02, 2012 4:09 AM
> To: [EMAIL PROTECTED]
> Subject: Re: avro client fails to connect to flume
>
> Could you provide your flume configs ?
> -roshan
>
> On Thu, Nov 1, 2012 at 8:13 AM, Siva Gudavalli <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
> hi,
>
> i am tryint to use avro client to source my log file to avro-source.
>
> flume-source -> avro source
> flume-channel -> memory
> flume-sink -> hdfs
>
> all of the services are up; but when i am calling avro client its failing connection to flume.
>
> please help!
>
> environment:
>
> hadoop 0.20.2
> flume 1.2
>
> logs from agent:
>
> 2012-11-01 20:03:49,977 INFO source.AvroSource: Avro source avro-AppSrv-source started.
> 2012-11-01 20:04:01,685 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370> => /127.0.0.1:41414<http://127.0.0.1:41414>] OPEN
> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370> => /127.0.0.1:41414<http://127.0.0.1:41414>] BOUND: /127.0.0.1:41414<http://127.0.0.1:41414>
> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370> => /127.0.0.1:41414<http://127.0.0.1:41414>] CONNECTED: /127.0.0.1:43370<http://127.0.0.1:43370>
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370> :> /127.0.0.1:41414<http://127.0.0.1:41414>] DISCONNECTED
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370> :> /127.0.0.1:41414<http://127.0.0.1:41414>] UNBOUND
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370> :> /127.0.0.1:41414<http://127.0.0.1:41414>] CLOSED
>
> logs from avro client:
>
> Info: Sourcing environment configuration script /usr/local/flume/conf/flume-env.sh
> Info: Including Hadoop libraries found via (/usr/local/hadoop/bin/hadoop) for HDFS access
> + exec /usr/lib/jvm/java-7-openjdk-i386/bin/java -Xms1024m -Xmx2048m -Dcom.sun.management.jmxremote -Dflume.root.logger=DEBUG,console -cp '/usr/local/flume/conf:/usr/local/flume/lib/*:/usr/local/flume/conf' -Djava.library.path=:/usr/local/hadoop/bin/../lib/native/Linux-i386-32 org.apache.flume.client.avro.AvroCLIClient -H localhost -p 41414 -F /usr/local/hbase/logs/hbase-hduser-master-172.20.204.119.log
> 2012-11-01 20:25:50,176 (main) [DEBUG - org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:420)] Batch size string = 5
> 2012-11-01 20:25:50,284 (main) [ERROR - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:76)] Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost, port: 41414 }: RPC connection error
>         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
>         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB