Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> different output schema when using org.apache.avro.mapred.AvroMultipleOutputs?


Copy link to this message
-
Re: different output schema when using org.apache.avro.mapred.AvroMultipleOutputs?
Absolutely.
I'll add the backtrace at least, and possibly more details if I find
anything interesting.

Thank you.
Pierre

On 03/05/2013 08:37 AM, Doug Cutting wrote:
> Can you please add this comment to that issue in Jira?
>
> Thanks!
>
> Doug
>
> On Mon, Mar 4, 2013 at 2:08 PM, Pierre Mariani <[EMAIL PROTECTED]> wrote:
>> It looks like what I am looking for, but after applying the patch to avro
>> mapred 1.7.4 I get the following exception:
>>
>> java.lang.NullPointerException
>>          at java.io.StringReader.<init>(StringReader.java:33)
>>          at org.apache.avro.Schema$Parser.parse(Schema.java:917)
>>          at org.apache.avro.Schema.parse(Schema.java:966)
>>          at
>> org.apache.avro.mapred.AvroMultipleOutputs$InternalFileOutputFormat.getRecordWriter(AvroMultipleOutputs.java:611)
>>          at
>> org.apache.avro.mapred.AvroMultipleOutputs.getRecordWriter(AvroMultipleOutputs.java:411)
>>          at
>> org.apache.avro.mapred.AvroMultipleOutputs.getCollector(AvroMultipleOutputs.java:570)
>>          at
>> org.apache.avro.mapred.AvroMultipleOutputs.getCollector(AvroMultipleOutputs.java:506)
>>          at
>> com.inome.diffhadoop.DiffSets$DiffReducer.reduce(DiffSets.java:93)
>>          at
>> com.inome.diffhadoop.DiffSets$DiffReducer.reduce(DiffSets.java:40)
>>          at
>> org.apache.avro.mapred.HadoopReducerBase.reduce(HadoopReducerBase.java:60)
>>          at
>> org.apache.avro.mapred.HadoopReducerBase.reduce(HadoopReducerBase.java:30)
>>          at
>> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:463)
>>          at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411)
>>          at org.apache.hadoop.mapred.Child.main(Child.java:170)
>>
>>
>>
>> On 03/01/2013 11:07 AM, Doug Cutting wrote:
>>> Is this the issue addressed by AVRO-1266?
>>>
>>> https://issues.apache.org/jira/browse/AVRO-1266
>>>
>>> Doug
>>>
>>> On Thu, Feb 28, 2013 at 3:11 PM, Pierre Mariani
>>> <[EMAIL PROTECTED]> wrote:
>>>> Beside fixing an issue with baseOutputPath,
>>>> https://issues.apache.org/jira/browse/AVRO-1215 allows to specify
>>>> different
>>>> schema for different outputs when using AvroMultipleOutputs.
>>>>
>>>> However, this is for the new Hadoop API (org.apache.avro.mapreduce).
>>>>
>>>> Is it possible that the same bug exists in the package for the old Hadoop
>>>> API (org.apache.avro.mapred) ?
>>>>
>>>> Thank you
>>>> Pierre
>>