Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Flume-NG Python avro Client


+
dinesh.simkhada@... 2012-07-23, 11:15
Copy link to this message
-
Re: Flume-NG Python avro Client
Hi,

This is a known issue in Avro. Please refer https://issues.apache.org/jira/browse/AVRO-1111
Thanks.
-Mubarak
On Jul 23, 2012, at 4:15 AM, [EMAIL PROTECTED] wrote:

> Hi,
> I am trying to build a flume client in python that send message to Avro source but always got 'java.lang.OutOfMemoryError: Java heap space'  error. I tried same thing with java it works properly
> I have attached my source and error log, any help/suggestions exports out there ?
>
> ---------- Source
> import sys
> import httplib
> import avro.ipc as ipc
> import avro.protocol as protocol
>
> server_addr = ('192.168.104.2', 3452)
> PROTOCOL = protocol.parse(open("flume.avpr").read())
>
> def sendData():
>     client = ipc.HTTPTransceiver(server_addr[0], server_addr[1])
>     requestor = ipc.Requestor(PROTOCOL, client)
>        
>     event = dict()
>     event['headers'] = {'name': 'abc', 'address': 'zyx'}
>     event['body'] = bytes('hello')
>
>     params = dict()
>     params['event'] = event  
>     print("Result : " + requestor.request('append', params))
>
>     client.close()
>
>
> if __name__ == '__main__':
>     sendData()
>
>
> ----------- error msg
> 2012-07-23 10:57:54,903 WARN ipc.NettyServer: Unexpected exception from downstream.
> java.lang.OutOfMemoryError: Java heap space
>     at java.util.ArrayList.<init>(ArrayList.java:112)
>     at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:154)
>     at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:131)
>     at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:282)
>     at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
>     at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
>     at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
>     at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
>     at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
>     at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>     at java.lang.Thread.run(Thread.java:662)
>
>

+
dinesh.simkhada@... 2012-07-23, 12:00
+
Brock Noland 2012-07-24, 00:31
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB