Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Review Request 17266: [PIG-3661] Piggybank AvroStorage fails if used in more than one load or store statement


Copy link to this message
-
Re: Review Request 17266: [PIG-3661] Piggybank AvroStorage fails if used in more than one load or store statement

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17266/#review32745
-----------------------------------------------------------

Ship it!
Ship It!

- Cheolsoo Park
On Jan. 24, 2014, 8:27 p.m., Rohini Palaniswamy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/17266/
> -----------------------------------------------------------
>
> (Updated Jan. 24, 2014, 8:27 p.m.)
>
>
> Review request for pig.
>
>
> Bugs: PIG-3661
>     https://issues.apache.org/jira/browse/PIG-3661
>
>
> Repository: pig
>
>
> Description
> -------
>
> This patch fixes other issues with AvroStorage as well apart from fixing multiple load and store
>    - Hidden files were not excluded (PIG-3717)
>    - mapred.input.dir was getting populated with all files instead of the top level directory making the conf very big
>    - Default value was not set for a Union
>
>
> Diffs
> -----
>
>   http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorage.java 1560805
>   http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorageUtils.java 1560805
>   http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/PigAvroInputFormat.java 1560805
>   http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/PigAvroRecordReader.java 1560805
>   http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/TestAvroStorage.java 1560805
>   http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/TestAvroStorageUtils.java 1560805
>
> Diff: https://reviews.apache.org/r/17266/diff/
>
>
> Testing
> -------
>
> Unit test added. testGlob was passing in git but failing when run in svn code base due to hidden .svn files(PIG-3717). That passes as well.
>
>
> Thanks,
>
> Rohini Palaniswamy
>
>