Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Help determining exit code from flume avro-client


Copy link to this message
-
Re: Help determining exit code from flume avro-client
It'd be great if you could submit the patch upstream to Flume! :-)

--
Hari Shreedharan
On Tuesday, October 30, 2012 at 4:06 PM, Paul Chavez wrote:

> Thank you, I was unsure if that was desired behavior or not. I updated the class as a POC to add exit codes for each of those catch blocks and it works great.
>  
> Thanks,
> Paul
>  
>
> From: Hari Shreedharan [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, October 30, 2012 11:47 AM
> To: [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])
> Subject: Re: Help determining exit code from flume avro-client
>
> Hi Paul.
>
> Thanks for reporting this. It does look like Flume does not return non-zero error code even if there was an exception. I filed https://issues.apache.org/jira/browse/FLUME-1670 to address this.  
>
> Thanks,
> Hari
>
> --  
> Hari Shreedharan
>
>
> On Tuesday, October 30, 2012 at 11:25 AM, Paul Chavez wrote:
>
> > I am working on getting the flume avro-client functionality working on Windows, and am currently stuck on how to determine if the file was sent successfully.
> >  
> > I used the blog post at http://mapredit.blogspot.com/2012/07/run-flume-13x-on-windows.html to guide me and have built the 1.4.0 snapshot successfully. I can manually send files along to flume using the ''org.apache.flume.client.avro.AvroCLIClient" class.
> >  
> > The full command I am using is invoked via batch file with the following single line:
> > "C:\Program Files (x86)\Java\jdk1.6.0_37\bin\java.exe" -Xmx20m -Dlog4j.configuration=file:///C:\apache-flume-1.4.0-SNAPSHOT\conf\log4j.properties -cp "C:\apache-flume-1.4.0-SNAPSHOT\lib\*" org.apache.flume.client.avro.AvroCLIClient -H 10.20.2.8 -p 6240 -F%1
> >  
> > My intention is to invoke from Powershell like so:
> > .\flume.bat C:\IISLogs\W3SVC100\u_ex121029.log
> >  
> > This is working well and I am able to successfully send IIS and our app logs to flume and then onto HDFS without issues. Now I am trying to automate the process, and need to know if the operation failed for some reason. If I simply turn off the flume source I can trigger an error which is dumped to console like so (using flume.root.logger=DEBUG,console in the log4j file):
> >  
> >  
> > 2012-10-30 11:14:22,185 (main) [ERROR - org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:76)] Unable t
> > o open connection to Flume. Exception follows.
> > org.apache.flume.FlumeException: NettyAvroRpcClient { host: x.x.x.x, port: 6240 }: RPC connection error
> >         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:117)
> >         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:93)
> >         at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:507)
> >         at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:169)
> >         at org.apache.flume.client.avro.AvroCLIClient.run(AvroCLIClient.java:168)
> >         at org.apache.flume.client.avro.AvroCLIClient.main(AvroCLIClient.java:68)
> > Caused by: java.io.IOException: Error connecting to /x.x.x.x:6240
> >         at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)
> >         at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
> >         at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
> >         at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:106)
> >         ... 5 more
> > Caused by: java.net.ConnectException: Connection refused: no further information
> >         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> >         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
> >         at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:
> > 401)
> >         at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipeli
> > neSink.java:370)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB