Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> avro client fails to connect to flume


+
Siva Gudavalli 2012-11-01, 15:13
+
Roshan Naik 2012-11-01, 22:39
+
Siva Gudavalli 2012-11-02, 01:44
Copy link to this message
-
Re: avro client fails to connect to flume
Please send your logs also. It will help us understand if there is an
error during startup.

Hari

On Thu, Nov 1, 2012 at 6:44 PM, Siva Gudavalli
<[EMAIL PROTECTED]> wrote:
> hi Roshan,
>
> here is the config....
>
> agent.sources = avro-AppSrv-source
> agent.sinks = hdfs-Cluster-sink
> agent.channels = mem-channel
>
> # set channel for sources, sinks
> agent.sources.avro-AppSrv-source.channels = mem-channel
> agent.sources.hdfs-Cluster-sink.channel = mem-channel
>
> # set source properties
> agent.sources.avro-AppSrv-source.type = avro
> agent.sources.avro-AppSrv-source.bind = localhost
> agent.sources.avro-AppSrv-source.port = 41414
>
> # set channel properties
> agent.channels.mem-channel.type = memory
> agent.channels.mem-channel.capacity = 10000
> agent.channels.mem-channel.transactionCapacity = 10000
>
> # set sink properties
> agent.sinks.hdfs-Cluster-sink.type = hdfs
> agent.sinks.hdfs-Cluster-sink.hdfs.path = hdfs://localhost:54310/flume
>
> ________________________________________
> From: Roshan Naik [[EMAIL PROTECTED]]
> Sent: Friday, November 02, 2012 4:09 AM
> To: [EMAIL PROTECTED]
> Subject: Re: avro client fails to connect to flume
>
> Could you provide your flume configs ?
> -roshan
>
> On Thu, Nov 1, 2012 at 8:13 AM, Siva Gudavalli <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
> hi,
>
> i am tryint to use avro client to source my log file to avro-source.
>
> flume-source -> avro source
> flume-channel -> memory
> flume-sink -> hdfs
>
> all of the services are up; but when i am calling avro client its failing connection to flume.
>
> please help!
>
> environment:
>
> hadoop 0.20.2
> flume 1.2
>
> logs from agent:
>
> 2012-11-01 20:03:49,977 INFO source.AvroSource: Avro source avro-AppSrv-source started.
> 2012-11-01 20:04:01,685 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370> => /127.0.0.1:41414<http://127.0.0.1:41414>] OPEN
> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370> => /127.0.0.1:41414<http://127.0.0.1:41414>] BOUND: /127.0.0.1:41414<http://127.0.0.1:41414>
> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370> => /127.0.0.1:41414<http://127.0.0.1:41414>] CONNECTED: /127.0.0.1:43370<http://127.0.0.1:43370>
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370> :> /127.0.0.1:41414<http://127.0.0.1:41414>] DISCONNECTED
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370> :> /127.0.0.1:41414<http://127.0.0.1:41414>] UNBOUND
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /127.0.0.1:43370<http://127.0.0.1:43370> :> /127.0.0.1:41414<http://127.0.0.1:41414>] CLOSED
>
> logs from avro client:
>
> Info: Sourcing environment configuration script /usr/local/flume/conf/flume-env.sh
> Info: Including Hadoop libraries found via (/usr/local/hadoop/bin/hadoop) for HDFS access
> + exec /usr/lib/jvm/java-7-openjdk-i386/bin/java -Xms1024m -Xmx2048m -Dcom.sun.management.jmxremote -Dflume.root.logger=DEBUG,console -cp '/usr/local/flume/conf:/usr/local/flume/lib/*:/usr/local/flume/conf' -Djava.library.path=:/usr/local/hadoop/bin/../lib/native/Linux-i386-32 org.apache.flume.client.avro.AvroCLIClient -H localhost -p 41414 -F /usr/local/hbase/logs/hbase-hduser-master-172.20.204.119.log
> 2012-11-01 20:25:50,176 (main) [DEBUG - org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:420)] Batch size string = 5
> 2012-11-01 20:25:50,284 (main) [ERROR - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:76)] Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost, port: 41414 }: RPC connection error
>         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
>         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
+
Siva Gudavalli 2012-11-02, 04:16
+
Alexander Lorenz 2012-11-07, 12:58