-
Re: avro client fails to connect to flumeRoshan Naik 2012-11-01, 22:39
Could you provide your flume configs ?
-roshan
On Thu, Nov 1, 2012 at 8:13 AM, Siva Gudavalli <[EMAIL PROTECTED]
> wrote:
> hi,
>
> i am tryint to use avro client to source my log file to avro-source.
>
> flume-source -> avro source
> flume-channel -> memory
> flume-sink -> hdfs
>
> all of the services are up; but when i am calling avro client its failing
> connection to flume.
>
> please help!
>
> environment:
>
> hadoop 0.20.2
> flume 1.2
>
> logs from agent:
>
> 2012-11-01 20:03:49,977 INFO source.AvroSource: Avro source
> avro-AppSrv-source started.
> 2012-11-01 20:04:01,685 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 => /127.0.0.1:41414] OPEN
> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 => /127.0.0.1:41414] BOUND: /127.0.0.1:41414
> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 => /127.0.0.1:41414] CONNECTED: /127.0.0.1:43370
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 :> /127.0.0.1:41414] DISCONNECTED
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 :> /127.0.0.1:41414] UNBOUND
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 :> /127.0.0.1:41414] CLOSED
>
> logs from avro client:
>
> Info: Sourcing environment configuration script
> /usr/local/flume/conf/flume-env.sh
> Info: Including Hadoop libraries found via (/usr/local/hadoop/bin/hadoop)
> for HDFS access
> + exec /usr/lib/jvm/java-7-openjdk-i386/bin/java -Xms1024m -Xmx2048m
> -Dcom.sun.management.jmxremote -Dflume.root.logger=DEBUG,console -cp
> '/usr/local/flume/conf:/usr/local/flume/lib/*:/usr/local/flume/conf'
> -Djava.library.path=:/usr/local/hadoop/bin/../lib/native/Linux-i386-32
> org.apache.flume.client.avro.AvroCLIClient -H localhost -p 41414 -F
> /usr/local/hbase/logs/hbase-hduser-master-172.20.204.119.log
> 2012-11-01 20:25:50,176 (main) [DEBUG -
> org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:420)]
> Batch size string = 5
> 2012-11-01 20:25:50,284 (main) [ERROR -
> org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:76)]
> Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost,
> port: 41414 }: RPC connection error
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
> at
> org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
> at
> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
> at
> org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:168)
> at
> org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:68)
> Caused by: java.io.IOException: Error connecting to localhost/
> 127.0.0.1:41414
> at
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:249)
> at
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:198)
> at
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:147)
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
> ... 5 more
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:401)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:370)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:292)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
-roshan
On Thu, Nov 1, 2012 at 8:13 AM, Siva Gudavalli <[EMAIL PROTECTED]
> wrote:
> hi,
>
> i am tryint to use avro client to source my log file to avro-source.
>
> flume-source -> avro source
> flume-channel -> memory
> flume-sink -> hdfs
>
> all of the services are up; but when i am calling avro client its failing
> connection to flume.
>
> please help!
>
> environment:
>
> hadoop 0.20.2
> flume 1.2
>
> logs from agent:
>
> 2012-11-01 20:03:49,977 INFO source.AvroSource: Avro source
> avro-AppSrv-source started.
> 2012-11-01 20:04:01,685 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 => /127.0.0.1:41414] OPEN
> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 => /127.0.0.1:41414] BOUND: /127.0.0.1:41414
> 2012-11-01 20:04:01,689 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 => /127.0.0.1:41414] CONNECTED: /127.0.0.1:43370
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 :> /127.0.0.1:41414] DISCONNECTED
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 :> /127.0.0.1:41414] UNBOUND
> 2012-11-01 20:04:02,562 INFO ipc.NettyServer: [id: 0x00e71be2, /
> 127.0.0.1:43370 :> /127.0.0.1:41414] CLOSED
>
> logs from avro client:
>
> Info: Sourcing environment configuration script
> /usr/local/flume/conf/flume-env.sh
> Info: Including Hadoop libraries found via (/usr/local/hadoop/bin/hadoop)
> for HDFS access
> + exec /usr/lib/jvm/java-7-openjdk-i386/bin/java -Xms1024m -Xmx2048m
> -Dcom.sun.management.jmxremote -Dflume.root.logger=DEBUG,console -cp
> '/usr/local/flume/conf:/usr/local/flume/lib/*:/usr/local/flume/conf'
> -Djava.library.path=:/usr/local/hadoop/bin/../lib/native/Linux-i386-32
> org.apache.flume.client.avro.AvroCLIClient -H localhost -p 41414 -F
> /usr/local/hbase/logs/hbase-hduser-master-172.20.204.119.log
> 2012-11-01 20:25:50,176 (main) [DEBUG -
> org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:420)]
> Batch size string = 5
> 2012-11-01 20:25:50,284 (main) [ERROR -
> org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:76)]
> Unable to open connection to Flume. Exception follows.
> org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost,
> port: 41414 }: RPC connection error
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
> at
> org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
> at
> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
> at
> org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:168)
> at
> org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:68)
> Caused by: java.io.IOException: Error connecting to localhost/
> 127.0.0.1:41414
> at
> org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:249)
> at
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:198)
> at
> org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:147)
> at
> org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
> ... 5 more
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:401)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:370)
> at
> org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:292)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
