Hi,

In java, I'd like to make a deep copy of an avro record.

Looking at the code that the avro compiler generates it looks like the way
to do this for records of type GraphNodeData
would be
GraphNodeData copy = GraphNodeData.newBuilder(existing_value).build();

where GraphNodeData is the name of my avro record.

Unfortunately, this generates a compile time error because
the method GraphNodeData.build  is decorated with "@Override" but it
apparently does not override any method.
Deleting "@Override" makes the code work. Unfortunately, I would have to do
this every time I regenerated my avro classes from the schema.

Am I doing something wrong or should I file a bug?

Thanks
J

There is deepcopy method on genericdata I think

I used it to create deep copies of the avro data

Sent from my iPhone

On Mar 11, 2012, at 3:40 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:

> Hi,
>
> In java, I'd like to make a deep copy of an avro record.
>
> Looking at the code that the avro compiler generates it looks like the way to do this for records of type GraphNodeData
> would be
> GraphNodeData copy = GraphNodeData.newBuilder(existing_value).build();
>
> where GraphNodeData is the name of my avro record.
>
> Unfortunately, this generates a compile time error because  
> the method GraphNodeData.build  is decorated with "@Override" but it apparently does not override any method.
> Deleting "@Override" makes the code work. Unfortunately, I would have to do this every time I regenerated my avro classes from the schema.
>
> Am I doing something wrong or should I file a bug?
>
> Thanks
> J

Yes. How do I instantiate an instance of GenericData though? It looks like
its constructors are protected.

Thanks
J

On Sun, Mar 11, 2012 at 1:52 PM, <[EMAIL PROTECTED]> wrote:

> There is deepcopy method on genericdata I think
>
> I used it to create deep copies of the avro data
>
> Sent from my iPhone
>
> On Mar 11, 2012, at 3:40 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > In java, I'd like to make a deep copy of an avro record.
> >
> > Looking at the code that the avro compiler generates it looks like the
> way to do this for records of type GraphNodeData
> > would be
> > GraphNodeData copy = GraphNodeData.newBuilder(existing_value).build();
> >
> > where GraphNodeData is the name of my avro record.
> >
> > Unfortunately, this generates a compile time error because
> > the method GraphNodeData.build  is decorated with "@Override" but it
> apparently does not override any method.
> > Deleting "@Override" makes the code work. Unfortunately, I would have to
> do this every time I regenerated my avro classes from the schema.
> >
> > Am I doing something wrong or should I file a bug?
> >
> > Thanks
> > J
>

GenericData is a singleton.  You can use the static method
GenericData#get() to obtain the singleton instance:

http://avro.apache.org/docs/current/api/java/org/apache/avro/generic/GenericData.html#get()

-James
On Sun, Mar 11, 2012 at 5:22 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:

> Yes. How do I instantiate an instance of GenericData though? It looks like
> its constructors are protected.
>
> Thanks
> J
>
>
> On Sun, Mar 11, 2012 at 1:52 PM, <[EMAIL PROTECTED]> wrote:
>
>> There is deepcopy method on genericdata I think
>>
>> I used it to create deep copies of the avro data
>>
>> Sent from my iPhone
>>
>> On Mar 11, 2012, at 3:40 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>>
>> > Hi,
>> >
>> > In java, I'd like to make a deep copy of an avro record.
>> >
>> > Looking at the code that the avro compiler generates it looks like the
>> way to do this for records of type GraphNodeData
>> > would be
>> > GraphNodeData copy = GraphNodeData.newBuilder(existing_value).build();
>> >
>> > where GraphNodeData is the name of my avro record.
>> >
>> > Unfortunately, this generates a compile time error because
>> > the method GraphNodeData.build  is decorated with "@Override" but it
>> apparently does not override any method.
>> > Deleting "@Override" makes the code work. Unfortunately, I would have
>> to do this every time I regenerated my avro classes from the schema.
>> >
>> > Am I doing something wrong or should I file a bug?
>> >
>> > Thanks
>> > J
>>
>
>

Thanks.

J

On Sun, Mar 11, 2012 at 6:20 PM, James Baldassari <[EMAIL PROTECTED]>wrote:

> GenericData is a singleton.  You can use the static method
> GenericData#get() to obtain the singleton instance:
>
>
> http://avro.apache.org/docs/current/api/java/org/apache/avro/generic/GenericData.html#get()
>
> -James
>
>
> On Sun, Mar 11, 2012 at 5:22 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>
>> Yes. How do I instantiate an instance of GenericData though? It looks
>> like its constructors are protected.
>>
>> Thanks
>> J
>>
>>
>> On Sun, Mar 11, 2012 at 1:52 PM, <[EMAIL PROTECTED]> wrote:
>>
>>> There is deepcopy method on genericdata I think
>>>
>>> I used it to create deep copies of the avro data
>>>
>>> Sent from my iPhone
>>>
>>> On Mar 11, 2012, at 3:40 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>>>
>>> > Hi,
>>> >
>>> > In java, I'd like to make a deep copy of an avro record.
>>> >
>>> > Looking at the code that the avro compiler generates it looks like the
>>> way to do this for records of type GraphNodeData
>>> > would be
>>> > GraphNodeData copy = GraphNodeData.newBuilder(existing_value).build();
>>> >
>>> > where GraphNodeData is the name of my avro record.
>>> >
>>> > Unfortunately, this generates a compile time error because
>>> > the method GraphNodeData.build  is decorated with "@Override" but it
>>> apparently does not override any method.
>>> > Deleting "@Override" makes the code work. Unfortunately, I would have
>>> to do this every time I regenerated my avro classes from the schema.
>>> >
>>> > Am I doing something wrong or should I file a bug?
>>> >
>>> > Thanks
>>> > J
>>>
>>
>>
>

GenericData.deepCopy gives me an instance of GenericData.Record. How do I
convert this to an instance of my specific data (e.g GraphNodeData).

It looks like the issue with the @override statement for the build might
have something to do with the fact that the specification for @Override
changed in jdk1.6 (see this post)
http://stackoverflow.com/questions/2335655/why-is-javac-failing-on-override-annotation.
 However, I'm using java version 1.6.0_27 so I'm not sure why its not
working for me.

To clarify, the statement
GraphNodeData copy = GraphNodeData.newBuilder(this.data).build()
Compiles fine, but generates a runtime warning
The method build() of type GraphNodeData.Builder must override a superclass
method

J

On Sun, Mar 11, 2012 at 7:37 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:

> Thanks.
>
> J
>
>
> On Sun, Mar 11, 2012 at 6:20 PM, James Baldassari <[EMAIL PROTECTED]>wrote:
>
>> GenericData is a singleton.  You can use the static method
>> GenericData#get() to obtain the singleton instance:
>>
>>
>> http://avro.apache.org/docs/current/api/java/org/apache/avro/generic/GenericData.html#get()
>>
>> -James
>>
>>
>> On Sun, Mar 11, 2012 at 5:22 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>>
>>> Yes. How do I instantiate an instance of GenericData though? It looks
>>> like its constructors are protected.
>>>
>>> Thanks
>>> J
>>>
>>>
>>> On Sun, Mar 11, 2012 at 1:52 PM, <[EMAIL PROTECTED]> wrote:
>>>
>>>> There is deepcopy method on genericdata I think
>>>>
>>>> I used it to create deep copies of the avro data
>>>>
>>>> Sent from my iPhone
>>>>
>>>> On Mar 11, 2012, at 3:40 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>>>>
>>>> > Hi,
>>>> >
>>>> > In java, I'd like to make a deep copy of an avro record.
>>>> >
>>>> > Looking at the code that the avro compiler generates it looks like
>>>> the way to do this for records of type GraphNodeData
>>>> > would be
>>>> > GraphNodeData copy = GraphNodeData.newBuilder(existing_value).build();
>>>> >
>>>> > where GraphNodeData is the name of my avro record.
>>>> >
>>>> > Unfortunately, this generates a compile time error because
>>>> > the method GraphNodeData.build  is decorated with "@Override" but it
>>>> apparently does not override any method.
>>>> > Deleting "@Override" makes the code work. Unfortunately, I would have
>>>> to do this every time I regenerated my avro classes from the schema.
>>>> >
>>>> > Am I doing something wrong or should I file a bug?
>>>> >
>>>> > Thanks
>>>> > J
>>>>
>>>
>>>
>>
>

Looks like when I upgraded to Java 7 it ran fine.
On Sun, Mar 11, 2012 at 9:07 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:

> GenericData.deepCopy gives me an instance of GenericData.Record. How do I
> convert this to an instance of my specific data (e.g GraphNodeData).
>
> It looks like the issue with the @override statement for the build might
> have something to do with the fact that the specification for @Override
> changed in jdk1.6 (see this post)
>
> http://stackoverflow.com/questions/2335655/why-is-javac-failing-on-override-annotation.
>  However, I'm using java version 1.6.0_27 so I'm not sure why its not
> working for me.
>
> To clarify, the statement
> GraphNodeData copy = GraphNodeData.newBuilder(this.data).build()
> Compiles fine, but generates a runtime warning
> The method build() of type GraphNodeData.Builder must override a
> superclass method
>
> J
>
>
> On Sun, Mar 11, 2012 at 7:37 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>
>> Thanks.
>>
>> J
>>
>>
>> On Sun, Mar 11, 2012 at 6:20 PM, James Baldassari <[EMAIL PROTECTED]>wrote:
>>
>>> GenericData is a singleton.  You can use the static method
>>> GenericData#get() to obtain the singleton instance:
>>>
>>>
>>> http://avro.apache.org/docs/current/api/java/org/apache/avro/generic/GenericData.html#get()
>>>
>>> -James
>>>
>>>
>>> On Sun, Mar 11, 2012 at 5:22 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>>>
>>>> Yes. How do I instantiate an instance of GenericData though? It looks
>>>> like its constructors are protected.
>>>>
>>>> Thanks
>>>> J
>>>>
>>>>
>>>> On Sun, Mar 11, 2012 at 1:52 PM, <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> There is deepcopy method on genericdata I think
>>>>>
>>>>> I used it to create deep copies of the avro data
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On Mar 11, 2012, at 3:40 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>> > Hi,
>>>>> >
>>>>> > In java, I'd like to make a deep copy of an avro record.
>>>>> >
>>>>> > Looking at the code that the avro compiler generates it looks like
>>>>> the way to do this for records of type GraphNodeData
>>>>> > would be
>>>>> > GraphNodeData copy >>>>> GraphNodeData.newBuilder(existing_value).build();
>>>>> >
>>>>> > where GraphNodeData is the name of my avro record.
>>>>> >
>>>>> > Unfortunately, this generates a compile time error because
>>>>> > the method GraphNodeData.build  is decorated with "@Override" but it
>>>>> apparently does not override any method.
>>>>> > Deleting "@Override" makes the code work. Unfortunately, I would
>>>>> have to do this every time I regenerated my avro classes from the schema.
>>>>> >
>>>>> > Am I doing something wrong or should I file a bug?
>>>>> >
>>>>> > Thanks
>>>>> > J
>>>>>
>>>>
>>>>
>>>
>>
>

If you want to make a deep copy of a specific record, the easiest way is
probably to use the Builder API,
e.g. GraphNodeData.newBuilder(recordToCopy).build().  This will create a
deep copy of 'recordToBuild' and then return it as a new GraphNodeData
object.  Keep in mind that the performance of the Builder API is not great
at the moment.  There are a couple of issues out there for improving this:

https://issues.apache.org/jira/browse/AVRO-985
https://issues.apache.org/jira/browse/AVRO-989

-James
On Mon, Mar 12, 2012 at 12:26 AM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:

> Looks like when I upgraded to Java 7 it ran fine.
>
>
> On Sun, Mar 11, 2012 at 9:07 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>
>> GenericData.deepCopy gives me an instance of GenericData.Record. How do I
>> convert this to an instance of my specific data (e.g GraphNodeData).
>>
>> It looks like the issue with the @override statement for the build might
>> have something to do with the fact that the specification for @Override
>> changed in jdk1.6 (see this post)
>>
>> http://stackoverflow.com/questions/2335655/why-is-javac-failing-on-override-annotation.
>>  However, I'm using java version 1.6.0_27 so I'm not sure why its not
>> working for me.
>>
>> To clarify, the statement
>> GraphNodeData copy = GraphNodeData.newBuilder(this.data).build()
>> Compiles fine, but generates a runtime warning
>> The method build() of type GraphNodeData.Builder must override a
>> superclass method
>>
>> J
>>
>>
>> On Sun, Mar 11, 2012 at 7:37 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>>
>>> Thanks.
>>>
>>> J
>>>
>>>
>>> On Sun, Mar 11, 2012 at 6:20 PM, James Baldassari <[EMAIL PROTECTED]
>>> > wrote:
>>>
>>>> GenericData is a singleton.  You can use the static method
>>>> GenericData#get() to obtain the singleton instance:
>>>>
>>>>
>>>> http://avro.apache.org/docs/current/api/java/org/apache/avro/generic/GenericData.html#get()
>>>>
>>>> -James
>>>>
>>>>
>>>> On Sun, Mar 11, 2012 at 5:22 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> Yes. How do I instantiate an instance of GenericData though? It looks
>>>>> like its constructors are protected.
>>>>>
>>>>> Thanks
>>>>> J
>>>>>
>>>>>
>>>>> On Sun, Mar 11, 2012 at 1:52 PM, <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> There is deepcopy method on genericdata I think
>>>>>>
>>>>>> I used it to create deep copies of the avro data
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>> On Mar 11, 2012, at 3:40 PM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>> > Hi,
>>>>>> >
>>>>>> > In java, I'd like to make a deep copy of an avro record.
>>>>>> >
>>>>>> > Looking at the code that the avro compiler generates it looks like
>>>>>> the way to do this for records of type GraphNodeData
>>>>>> > would be
>>>>>> > GraphNodeData copy >>>>>> GraphNodeData.newBuilder(existing_value).build();
>>>>>> >
>>>>>> > where GraphNodeData is the name of my avro record.
>>>>>> >
>>>>>> > Unfortunately, this generates a compile time error because
>>>>>> > the method GraphNodeData.build  is decorated with "@Override" but
>>>>>> it apparently does not override any method.
>>>>>> > Deleting "@Override" makes the code work. Unfortunately, I would
>>>>>> have to do this every time I regenerated my avro classes from the schema.
>>>>>> >
>>>>>> > Am I doing something wrong or should I file a bug?
>>>>>> >
>>>>>> > Thanks
>>>>>> > J
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

On 03/11/2012 10:22 PM, James Baldassari wrote:
> If you want to make a deep copy of a specific record, the easiest way is
> probably to use the Builder API,
> e.g. GraphNodeData.newBuilder(recordToCopy).build().

SpecificData.get().deepCopy(record) should work too.

Doug

Thanks James and Doug. I was able to simply cast the output of
SpecificData...deepCopy to my type and it seems to bypass the problematic
methods decorated with @override.

What about the potential incompatibility with earlier versions of java due
to the change in semantics of @override? If this is really an issue this
seems like it would affect a lot of users particularly people using Avro
MapReduce on a cluster where upgrading java is not a trivial proposition.
In my particular case, the reduce processing requires loading all values
associated with the key into memory, which necessitates a deep copy because
the iterable object passed to the reducer seems to be reusing the same
instance.

Using SpecificData.get().deepCopy(record) seems like a viable workaround.
Nonetheless, it does seem a bit problematic if the compiler is generating
code that is incompatible with earlier versions of java.

J

On Mon, Mar 12, 2012 at 9:05 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> On 03/11/2012 10:22 PM, James Baldassari wrote:
> > If you want to make a deep copy of a specific record, the easiest way is
> > probably to use the Builder API,
> > e.g. GraphNodeData.newBuilder(recordToCopy).build().
>
> SpecificData.get().deepCopy(record) should work too.
>
> Doug
>

We should be generating Java 1.6 compatible code.

What version were you testing?

1.6.3 is near release, the RC is available here:
http://mail-archives.apache.org/mod_mbox/avro-dev/201203.mbox/%3C4F514F22.8
[EMAIL PROTECTED]%3E

Does it have the same problem?

On 3/12/12 9:27 AM, "Jeremy Lewi" <[EMAIL PROTECTED]> wrote:
>Thanks James and Doug. I was able to simply cast the output of
>SpecificData...deepCopy to my type and it seems to bypass the problematic
>methods decorated with @override.
>What about the potential incompatibility with earlier versions of java
>due to the change in semantics of @override? If this is really an issue
>this seems like it would affect a lot of users particularly people using
>Avro MapReduce on a cluster where upgrading java is not a trivial
>proposition. In my particular case, the reduce processing requires
>loading all values associated with the key into memory, which
>necessitates a deep copy because the iterable object passed to the
>reducer seems to be reusing the same instance.
>
>Using SpecificData.get().deepCopy(record) seems like a viable workaround.
>Nonetheless, it does seem a bit problematic if the compiler is generating
>code that is incompatible with earlier versions of java.
>
>J
>
>On Mon, Mar 12, 2012 at 9:05 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>
>On 03/11/2012 10:22 PM, James Baldassari wrote:
>> If you want to make a deep copy of a specific record, the easiest way is
>> probably to use the Builder API,
>> e.g. GraphNodeData.newBuilder(recordToCopy).build().
>
>
>SpecificData.get().deepCopy(record) should work too.
>
>Doug
>
>
>

On 03/12/2012 10:20 AM, Scott Carey wrote:
> We should be generating Java 1.6 compatible code.

I may have inadvertantly used Java 1.7 to compile recent releases.

Does this create problems?  If so, we might consider re-publising
binaries for these releases.  The sources should not change, but
re-generating jar files seems reasonable.

Doug

I was using 1.6.0_27 and the @override decorator was a problem, maybe newer
releases in the 1.6 set will work.

J

On Mon, Mar 12, 2012 at 10:42 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> On 03/12/2012 10:20 AM, Scott Carey wrote:
> > We should be generating Java 1.6 compatible code.
>
> I may have inadvertantly used Java 1.7 to compile recent releases.
>
> Does this create problems?  If so, we might consider re-publising
> binaries for these releases.  The sources should not change, but
> re-generating jar files seems reasonable.
>
> Doug
>

On 03/12/2012 11:09 AM, Jeremy Lewi wrote:
> I was using 1.6.0_27 and the @override decorator was a problem, maybe
> newer releases in the 1.6 set will work.

Re-reading the thread I now see the problem is compiling code generated
by Avro.  The builder's generated build() method has an @Override
declaration.  It implements a method in the RecordBuilder interface.
Using @Override for interface methods was not permitted in Java 5 but is
permitted in Java 6 and Java 7.  So I'm now confused how this fails in
any version of Java 6.  Regardless, if its really a problem for even
some versions of Java 6 then we should probably just remove it from the
generated code.  It doesn't add much value there.

Doug

I am able to use Avro 1.6.2 with the Specific builder API with no issue with
Java 1.6.0_29.

I also double checked the build, it sets the source and target versions to
1.6 when compiling.  The avro-service-archetype does as well.

On 3/12/12 11:09 AM, "Jeremy Lewi" <[EMAIL PROTECTED]> wrote:

> I was using 1.6.0_27 and the @override decorator was a problem, maybe newer
> releases in the 1.6 set will work.
>
> J
>
> On Mon, Mar 12, 2012 at 10:42 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>> On 03/12/2012 10:20 AM, Scott Carey wrote:
>>> > We should be generating Java 1.6 compatible code.
>>
>> I may have inadvertantly used Java 1.7 to compile recent releases.
>>
>> Does this create problems?  If so, we might consider re-publising
>> binaries for these releases.  The sources should not change, but
>> re-generating jar files seems reasonable.
>>
>> Doug
>