Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Review Request 17266: [PIG-3661] Piggybank AvroStorage fails if used in more than one load or store statement


Copy link to this message
-
Re: Review Request 17266: [PIG-3661] Piggybank AvroStorage fails if used in more than one load or store statement

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17266/
-----------------------------------------------------------

(Updated Jan. 24, 2014, 8:27 p.m.)
Review request for pig.
Changes
-------

Addressed review comments
Bugs: PIG-3661
    https://issues.apache.org/jira/browse/PIG-3661
Repository: pig
Description
-------

This patch fixes other issues with AvroStorage as well apart from fixing multiple load and store
   - Hidden files were not excluded (PIG-3717)
   - mapred.input.dir was getting populated with all files instead of the top level directory making the conf very big
   - Default value was not set for a Union
Diffs (updated)
-----

  http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorage.java 1560805
  http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorageUtils.java 1560805
  http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/PigAvroInputFormat.java 1560805
  http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/PigAvroRecordReader.java 1560805
  http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/TestAvroStorage.java 1560805
  http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/TestAvroStorageUtils.java 1560805

Diff: https://reviews.apache.org/r/17266/diff/
Testing
-------

Unit test added. testGlob was passing in git but failing when run in svn code base due to hidden .svn files(PIG-3717). That passes as well.
Thanks,

Rohini Palaniswamy