Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Avro RPC: Python to Java isn't working for me...


+
Stefan Krawczyk 2013-05-22, 03:18
+
Atin Sood 2013-05-22, 11:11
+
Stefan Krawczyk 2013-05-22, 15:55
+
Atin Sood 2013-05-22, 23:22
+
Stefan Krawczyk 2013-05-22, 23:36
Copy link to this message
-
Re: Avro RPC: Python to Java isn't working for me...
Stefan

Were you ever able to make any progress on this ?

Just curious :)

--
Atin Sood
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Wednesday, May 22, 2013 at 7:36 PM, Stefan Krawczyk wrote:

> Yes agreed, that would be a good approach to take, however before I go try to figure out how to write a NettyTransceiver in python, I'd like to get some community input on so that:
>  - I understand why there wasn't one built in the first place, and thus answer is it actually possible to build that in python
>  - I'm not duplicating effort if there is somebody already working on it
>
> So any NettyServer & NettyTransceiver experts in the house?
>
> :)
>
> Cheers,
>
> Stefan
>
>
>
> On Wed, May 22, 2013 at 4:22 PM, Atin Sood <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > Sure Stefan
> >
> > I am a newbie to both python and avro and have limited experience in networking.
> >
> > But if I am not wrong the problem is at python client end as I am guessing you must be using s.th (http://s.th) like
> >
> > client code - attach to the server and send a message
> >     client = ipc.HTTPTransceiver(server_addr[0], server_addr[1])
> >     requestor = ipc.Requestor(PROTOCOL, client)
> >
> >
> > So even though you can go ahead and switch to NettyServer in java you will be limited to use httpserver because your python client uses http client
> >
> > https://github.com/phunt/avro-rpc-quickstart/blob/master/src/main/java/example/Main.java
> >
> > I guess the best way to get around this will be to look at source code of avro python client ipc.py file
> >
> > and add a new implementation besides the one that comes out of the box.
> >
> > class HTTPTransceiver(object): """ A simple HTTP-based transceiver implementation. Useful for clients but not for servers """
> >
> >
> > That said, again I am new to the whole thing so I might be totally wrong :)
> >
> > --
> > Atin Sood
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> >
> >
> > On Wednesday, May 22, 2013 at 11:55 AM, Stefan Krawczyk wrote:
> >
> > > Hi Atin,
> > >
> > > Thanks for the response. Yes I understand I could use HTTPServer on the java side and things would work. However I'm after a solution where I can still have the java side use the NettyServer.
> > >
> > > Cheers,
> > >
> > > Stefan
> > >
> > >
> > > On Wed, May 22, 2013 at 4:11 AM, Atin Sood <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > > > You can try looking into something that I wrote as an example
> > > >
> > > > https://github.com/atinsood/HESDataAnalyticsFinalProject/tree/master/javaXPython
> > > >
> > > > https://github.com/atinsood/HESDataAnalyticsFinalProject#javaxpython
> > > >
> > > > --
> > > > Atin Sood
> > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> > > >
> > > >
> > > > On Tuesday, May 21, 2013 at 11:18 PM, Stefan Krawczyk wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I am trying to use Avro RPC and have a python client talk to a java server, using the avro-rpc-quickstart (https://github.com/phunt/avro-rpc-quickstart) on github as a base (I made sure the avro version being pulled in was 1.7.4). However when I get my python client to talk to the java server I see this error:
> > > > >
> > > > > 2013-05-20 19:38:32,512 (pool-5-thread-2) [WARN - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.exceptionCaught(NettyServer.java:201)] Unexpected exception from downstream.
> > > > > org.apache.avro.AvroRuntimeException: Excessively large list allocation request detected: 539959368 items! Connection closed.
> > > > > at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:167)
> > > > > at org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:139)
> > > > > at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:286)
> > > > > at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208)
+
Stefan Krawczyk 2013-05-29, 04:12
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB