Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> different output schema when using org.apache.avro.mapred.AvroMultipleOutputs?


Copy link to this message
-
Re: different output schema when using org.apache.avro.mapred.AvroMultipleOutputs?
Absolutely.
I'll add the backtrace at least, and possibly more details if I find
anything interesting.

Thank you.
Pierre

On 03/05/2013 08:37 AM, Doug Cutting wrote:
> Can you please add this comment to that issue in Jira?
>
> Thanks!
>
> Doug
>
> On Mon, Mar 4, 2013 at 2:08 PM, Pierre Mariani <[EMAIL PROTECTED]> wrote:
>> It looks like what I am looking for, but after applying the patch to avro
>> mapred 1.7.4 I get the following exception:
>>
>> java.lang.NullPointerException
>>          at java.io.StringReader.<init>(StringReader.java:33)
>>          at org.apache.avro.Schema$Parser.parse(Schema.java:917)
>>          at org.apache.avro.Schema.parse(Schema.java:966)
>>          at
>> org.apache.avro.mapred.AvroMultipleOutputs$InternalFileOutputFormat.getRecordWriter(AvroMultipleOutputs.java:611)
>>          at
>> org.apache.avro.mapred.AvroMultipleOutputs.getRecordWriter(AvroMultipleOutputs.java:411)
>>          at
>> org.apache.avro.mapred.AvroMultipleOutputs.getCollector(AvroMultipleOutputs.java:570)
>>          at
>> org.apache.avro.mapred.AvroMultipleOutputs.getCollector(AvroMultipleOutputs.java:506)
>>          at
>> com.inome.diffhadoop.DiffSets$DiffReducer.reduce(DiffSets.java:93)
>>          at
>> com.inome.diffhadoop.DiffSets$DiffReducer.reduce(DiffSets.java:40)
>>          at
>> org.apache.avro.mapred.HadoopReducerBase.reduce(HadoopReducerBase.java:60)
>>          at
>> org.apache.avro.mapred.HadoopReducerBase.reduce(HadoopReducerBase.java:30)
>>          at
>> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:463)
>>          at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411)
>>          at org.apache.hadoop.mapred.Child.main(Child.java:170)
>>
>>
>>
>> On 03/01/2013 11:07 AM, Doug Cutting wrote:
>>> Is this the issue addressed by AVRO-1266?
>>>
>>> https://issues.apache.org/jira/browse/AVRO-1266
>>>
>>> Doug
>>>
>>> On Thu, Feb 28, 2013 at 3:11 PM, Pierre Mariani
>>> <[EMAIL PROTECTED]> wrote:
>>>> Beside fixing an issue with baseOutputPath,
>>>> https://issues.apache.org/jira/browse/AVRO-1215 allows to specify
>>>> different
>>>> schema for different outputs when using AvroMultipleOutputs.
>>>>
>>>> However, this is for the new Hadoop API (org.apache.avro.mapreduce).
>>>>
>>>> Is it possible that the same bug exists in the package for the old Hadoop
>>>> API (org.apache.avro.mapred) ?
>>>>
>>>> Thank you
>>>> Pierre
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB