Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Speed improvements for reflection-based serialization


+
Leo Romanoff 2013-03-21, 07:39
Copy link to this message
-
Re: Speed improvements for reflection-based serialization
Leo,

Please submit your patches, I'd love to see them.  Create an issue in
Jira and attach your changes there.

https://issues.apache.org/jira/browse/AVRO

Thanks,

Doug

On Thu, Mar 21, 2013 at 12:39 AM, Leo Romanoff <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I've played a bit with Avro serialization based on reflection, i.e. with the classes from org.apache.avro.generic package.
> It works fine in this mode, but is rather slow compared to such frameworks like protostuff or kryo.
> Quick look at the source code has shown that a lot of reflection-based operations and class lookups are not cached. And such operations are usually pretty expensive when executed by a JVM.
>
> So I changed some of org.apache.avro.generic classes, introduced caching and a few other optimizations. Now it seems to perform much better.
> I could submit my patch for a review, if anyone is interested in such improvements. I'm new to Avro, but I got the impression that Voldermont and may be a few other BigData projects are using it? May be they are interested, though I don't know if they use this reflection-based serialization.
>
> Best Regards,
>   Leo
+
Doug Cutting 2013-03-21, 16:31