Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> avro client fails to connect to flume


Copy link to this message
-
Re: avro client fails to connect to flume
Could you provide your flume configs ?
-roshan

On Thu, Nov 1, 2012 at 8:13 AM, Siva Gudavalli <[EMAIL PROTECTED]
> wrote:

> hi,
>
> i am tryint to use avro client to source my log file to avro-source.
>
> flume-source -> avro source
> flume-channel -> memory
> flume-sink -> hdfs
>
> all of the services are up; but when i am calling avro client its failing
> connection to flume.
>
> please help!
>
> environment:
>
> hadoop 0.20.2
> flume 1.2
>
> logs from agent:
>
> 2012-11-01 20:03:49,977 INFO source.AvroSource: Avro source
> avro-AppSrv-source started.
> 2012-11-01 20:04:01,685 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 => /127.0.0.1:41414] OPEN
> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 => /127.0.0.1:41414] BOUND: /127.0.0.1:41414
> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 => /127.0.0.1:41414] CONNECTED: /127.0.0.1:43370
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 :> /127.0.0.1:41414] DISCONNECTED
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 :> /127.0.0.1:41414] UNBOUND
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 :> /127.0.0.1:41414] CLOSED
>
> logs from avro client:
>
> Info: Sourcing environment configuration script
> /usr/local/flume/conf/flume-env.sh
> Info: Including Hadoop libraries found via (/usr/local/hadoop/bin/hadoop)
> for HDFS access
> + exec /usr/lib/jvm/java-7-openjdk-i386/bin/java -Xms1024m -Xmx2048m
> -Dcom.sun.management.jmxremote -Dflume.root.logger=DEBUG,console -cp
> '/usr/local/flume/conf:/usr/local/flume/lib/*:/usr/local/flume/conf'
> -Djava.library.path=:/usr/local/hadoop/bin/../lib/native/Linux-i386-32
> org.apache.flume.client.avro.AvroCLIClient -H localhost -p 41414 -F
> /usr/local/hbase/logs/hbase-hduser-master-172.20.204.119.log
> 2012-11-01 20:25:50,176 (main) [DEBUG -
> org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:420)]
> Batch size string = 5
> 2012-11-01 20:25:50,284 (main) [ERROR -
> org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:76)]
> Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost,
> port: 41414 }: RPC connection error
>         at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
>         at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
>         at
> org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
>         at
> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
>         at
> org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:168)
>         at
> org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:68)
> Caused by: java.io.IOException: Error connecting to localhost/
> 127.0.0.1:41414
>         at
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:249)
>         at
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:198)
>         at
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:147)
>         at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
>         ... 5 more
> Caused by: java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:401)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:370)
>         at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:292)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB