Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Python/Java/Ruby RPC interop issues


Copy link to this message
-
Python/Java/Ruby RPC interop issues
Hello!

New to the Avro world. I have been evaluating Flume and testing various
capabilities. I have run into an issue with the Avro RPC piece. So far, I
have found that the Flume Avro RPC java client works with a Flume Avro
Source as expected (I believe these implementations use Netty). However,
when I tried using a Ruby RPC client to communicate with the Flume Avro
Source, there was a failure and the source reported:

org.apache.avro.AvroRuntimeException: Excessively large list allocation
request detected: 539959368 items! Connection closed.

I then tried this with a Python RPC client and had a similar issue. I
suspected client/server Avro interop issues, so then verified that a Python
Avro RPC client and Python Avro RPC server communicate correctly (using
https://github.com/phunt/avro-rpc-quickstart). Also verified that Ruby Avro
RPC client and Ruby Avro RPC server communicate correctly. However, using
the Python Avro RPC client with Ruby Avro RPC server fails (client hangs),
and Ruby client with Python server also fails (again, client hangs).

I know I'm not providing exhaustive info here, but I'm surprised to find
out-of-the-box interop problems. Do folks regularly use these
client/servers in various mixes (Java/Ruby/Python)? Are there particular
gotchas or configurations that are not obvious?

Thank you,
Steve