Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> different output schema when using org.apache.avro.mapred.AvroMultipleOutputs?


+
Pierre Mariani 2013-02-28, 23:11
+
Doug Cutting 2013-03-01, 19:07
+
Pierre Mariani 2013-03-04, 22:08
Copy link to this message
-
Re: different output schema when using org.apache.avro.mapred.AvroMultipleOutputs?
Can you please add this comment to that issue in Jira?

Thanks!

Doug

On Mon, Mar 4, 2013 at 2:08 PM, Pierre Mariani <[EMAIL PROTECTED]> wrote:
> It looks like what I am looking for, but after applying the patch to avro
> mapred 1.7.4 I get the following exception:
>
> java.lang.NullPointerException
>         at java.io.StringReader.<init>(StringReader.java:33)
>         at org.apache.avro.Schema$Parser.parse(Schema.java:917)
>         at org.apache.avro.Schema.parse(Schema.java:966)
>         at
> org.apache.avro.mapred.AvroMultipleOutputs$InternalFileOutputFormat.getRecordWriter(AvroMultipleOutputs.java:611)
>         at
> org.apache.avro.mapred.AvroMultipleOutputs.getRecordWriter(AvroMultipleOutputs.java:411)
>         at
> org.apache.avro.mapred.AvroMultipleOutputs.getCollector(AvroMultipleOutputs.java:570)
>         at
> org.apache.avro.mapred.AvroMultipleOutputs.getCollector(AvroMultipleOutputs.java:506)
>         at
> com.inome.diffhadoop.DiffSets$DiffReducer.reduce(DiffSets.java:93)
>         at
> com.inome.diffhadoop.DiffSets$DiffReducer.reduce(DiffSets.java:40)
>         at
> org.apache.avro.mapred.HadoopReducerBase.reduce(HadoopReducerBase.java:60)
>         at
> org.apache.avro.mapred.HadoopReducerBase.reduce(HadoopReducerBase.java:30)
>         at
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:463)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411)
>         at org.apache.hadoop.mapred.Child.main(Child.java:170)
>
>
>
> On 03/01/2013 11:07 AM, Doug Cutting wrote:
>>
>> Is this the issue addressed by AVRO-1266?
>>
>> https://issues.apache.org/jira/browse/AVRO-1266
>>
>> Doug
>>
>> On Thu, Feb 28, 2013 at 3:11 PM, Pierre Mariani
>> <[EMAIL PROTECTED]> wrote:
>>>
>>> Beside fixing an issue with baseOutputPath,
>>> https://issues.apache.org/jira/browse/AVRO-1215 allows to specify
>>> different
>>> schema for different outputs when using AvroMultipleOutputs.
>>>
>>> However, this is for the new Hadoop API (org.apache.avro.mapreduce).
>>>
>>> Is it possible that the same bug exists in the package for the old Hadoop
>>> API (org.apache.avro.mapred) ?
>>>
>>> Thank you
>>> Pierre
>
>
+
Pierre Mariani 2013-03-05, 16:53