Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Flume-NG Python avro Client


Copy link to this message
-
Re: Flume-NG Python avro Client
Hi,

This is a known issue in Avro. Please refer https://issues.apache.org/jira/browse/AVRO-1111
Thanks.
-Mubarak
On Jul 23, 2012, at 4:15 AM, [EMAIL PROTECTED] wrote:

> Hi,
> I am trying to build a flume client in python that send message to Avro source but always got 'java.lang.OutOfMemoryError: Java heap space'  error. I tried same thing with java it works properly
> I have attached my source and error log, any help/suggestions exports out there ?
>
> ---------- Source
> import sys
> import httplib
> import avro.ipc as ipc
> import avro.protocol as protocol
>
> server_addr = ('192.168.104.2', 3452)
> PROTOCOL = protocol.parse(open("flume.avpr").read())
>
> def sendData():
>     client = ipc.HTTPTransceiver(server_addr[0], server_addr[1])
>     requestor = ipc.Requestor(PROTOCOL, client)
>        
>     event = dict()
>     event['headers'] = {'name': 'abc', 'address': 'zyx'}
>     event['body'] = bytes('hello')
>
>     params = dict()
>     params['event'] = event  
>     print("Result : " + requestor.request('append', params))
>
>     client.close()
>
>
> if __name__ == '__main__':
>     sendData()
>
>
> ----------- error msg
> 2012-07-23 10:57:54,903 WARN ipc.NettyServer: Unexpected exception from downstream.
> java.lang.OutOfMemoryError: Java heap space
>     at java.util.ArrayList.<init>(ArrayList.java:112)
>     at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:154)
>     at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:131)
>     at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:282)
>     at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
>     at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
>     at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
>     at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
>     at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
>     at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>     at java.lang.Thread.run(Thread.java:662)
>
>