Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - Incorrect Json schema generated for @java_class


+
avronewbie 2013-02-28, 03:18
+
Doug Cutting 2013-02-28, 17:19
+
Alexandre Normand 2013-02-28, 17:25
+
Doug Cutting 2013-02-28, 18:04
+
Alexandre Normand 2013-02-28, 18:36
+
Doug Cutting 2013-02-28, 21:58
Copy link to this message
-
Re: Incorrect Json schema generated for @java_class
Alexandre Normand 2013-02-28, 22:08
Cool. Thanks. I had a patch in progress for this that I haven't completed but I'll resume work. I'll create a ticket now so it's being tracked.  

--  
Alex
On Thursday, February 28, 2013 at 1:58 PM, Doug Cutting wrote:

> If you cannot use ReflectDatumReader and ReflectDatumWriter then
> perhaps we could move the readString() and writeString()
> implementations from these to SpecificData. One can imagine also
> wishing to, e.g., specify the collection class used for arrays and
> maps via the java-class schema property with specific data. The
> reflect package is more about automatically inferring schemas through
> introspection. That's not the case here, rather this is just
> permitting more control over what classes are used to implement one's
> schemas, which seems reasonably within the scope of specific.
>  
> Doug
>  
> On Thu, Feb 28, 2013 at 10:36 AM, Alexandre Normand
> <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > I agree with the SpecificCompiler#getStringType() change. I actually have an
> > early draft patch that has this change. As for using the ReflectDatum
> > Reader/Writer, I guess that might be sufficient for some use cases but,
> > unfortunately, it isn't for mine ;). We're using kiji
> > (https://github.com/kijiproject) and I don't have control over the
> > Reader/Writer implementation being used.
> >  
> > I don't want to dirty the Generic/Specific reader/writers but it might be
> > some desirable feature to support JDK classes like the
> > BigDecimal/BigInteger?
> >  
> > --
> > Alex
> >  
> > On Thursday, February 28, 2013 at 10:04 AM, Doug Cutting wrote:
> >  
> > You could get the specific compiler to emit the desired type by
> > modifying the method SpecificCompiler#getStringType() to take the
> > Schema as a parameter and, if it has a java-class attribute, return
> > that. Then things should work so long as you use ReflectDatumReader
> > and ReflectDatumWriter. Would that suffice?
> >  
> > Doug
> >  
> > On Thu, Feb 28, 2013 at 9:25 AM, Alexandre Normand
> > <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> >  
> > Hey Doug,
> > I was also wondering about something related. In reflection, we can use the
> > java-class property to define the java class for stringable types. Could we
> > find a way to bring some of this to the Specific generated classes so that a
> > stringable type (I'm specifically thinking of BigDecimal and BigInteger)
> > could be exposed as the java-class type instead of Strings? I started
> > looking into this and there seem to be two possibilities, none of which I'm
> > terribly excited about:
> >  
> > 1. Use some reflection to invoke the string constructor for the
> > java-class…but that seems dirty as the Specific API probably shouldn't do
> > reflection, right?
> > 2. Handle a very specific list of java-class types (such as BigDecimal and
> > BigInteger) and have code that deals specifically with these in
> > Reader/Writer.
> >  
> > Thoughts?
> > --
> > Alex
> >  
> > On Thursday, February 28, 2013 at 9:19 AM, Doug Cutting wrote:
> >  
> > Annotation names in IDL are copied directly to schema properties with
> > the same names.
> >  
> > The bug is in the IDL documentation, which uses the annotation
> > @java_class in an example, which is meaningless. The Java
> > implementation requires the "java-class" schema property, so the
> > annotation used should be @java-class.
> >  
> > I filed an issue in Jira for this.
> >  
> > https://issues.apache.org/jira/browse/AVRO-1264
> >  
> > Doug
> >  
> > On Wed, Feb 27, 2013 at 7:18 PM, avronewbie <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> >  
> > For the IDL field:
> >  
> > @java_class("java.util.ArrayList") array<int> arrayOfInts;
> >  
> > the JSON schema generated by the avro-maven-plugin is
> >  
> > {\"name\":\"arrayOfInts\",\"type\":{\"type\":\"array\",\"items\":\"int\",\"java_class\":\"java.util.ArrayList\"}}
> >  
> > Shouldn't it be generated in JSON as java-class (with hypen instead of
+
avronewbie 2013-02-28, 20:10
+
Doug Cutting 2013-02-28, 21:51