Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # dev - Re: Review Request: PIG-3318 Patch to address default values when schemas are merged in AvroStorage. It does this for Records containing primitive values


Copy link to this message
-
Re: Review Request: PIG-3318 Patch to address default values when schemas are merged in AvroStorage. It does this for Records containing primitive values
Rohini Palaniswamy 2013-05-27, 02:28

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11135/#review21046
-----------------------------------------------------------

http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorage.java
<https://reviews.apache.org/r/11135/#comment43545>

    When user has specified a input schema, that needs to be used as the reader schema. If not, then the merged schema for that path needs to be used as the reader schema.
    
    Need to throw error if both multiple_schemas and user defined schema is present.

http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorageUtils.java
<https://reviews.apache.org/r/11135/#comment43543>

    formatting

http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorageUtils.java
<https://reviews.apache.org/r/11135/#comment43544>

    Why not just set default on the mergedSchema for FIXED and primitive types (or throw an error if there is a conflict) and be done with it?

http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/PigAvroRecordReader.java
<https://reviews.apache.org/r/11135/#comment43546>

    Why do we need to initialize with default values. If the merged reader schema, has the default value won't it be automatically set?
- Rohini Palaniswamy
On May 23, 2013, 12:12 a.m., Viraj Bhat wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11135/
> -----------------------------------------------------------
>
> (Updated May 23, 2013, 12:12 a.m.)
>
>
> Review request for pig.
>
>
> Description
> -------
>
> Default values are not honoured when merging default schema
>
>
> This addresses bug PIG-3318.
>     https://issues.apache.org/jira/browse/PIG-3318
>
>
> Diffs
> -----
>
>   http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorage.java 1484564
>   http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorageUtils.java 1484564
>   http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/PigAvroInputFormat.java 1484564
>   http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/PigAvroRecordReader.java 1484564
>   http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/TestAvroStorage.java 1484564
>
> Diff: https://reviews.apache.org/r/11135/diff/
>
>
> Testing
> -------
>
> Yes
>
>
> Thanks,
>
> Viraj Bhat
>
>