Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Re: Review Request: Changes for PIG-3321


Copy link to this message
-
Re: Review Request: Changes for PIG-3321

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11155/
-----------------------------------------------------------

(Updated May 15, 2013, 7:01 p.m.)
Review request for pig and Rohini Palaniswamy.
Changes
-------

Moved to trunk and made requested changes.
Description
-------

Changes for https://issues.apache.org/jira/browse/PIG-3321

Overview:
AvroStorage.java - If 'schema' argument is passed to constructor, use it as the reader schema on load.  Moved getSchema() to AvroStorageUtils and made public+static, so it can be called from PigAvroRecordReader.
AvroStorageUtils.java - Moved getSchema() here.
PigAvroInputFormat.java - nothing functional here, just renamed 'schema' to 'readerSchema' for clarity.
PigAvroRecordReader.java - The constructor now determines the writer schema for its split, and passes both reader and writer schema to the PigAvroDatumReader constructor, which will allow the Avro code to resolve the two.
PigAvroDatumReader.java - Changed readRecord() to add entries to the output Tuple in writer order rather than reader order.
TestAvroStorage.java - Added a new testcase for user specified schema in load.
Diffs (updated)
-----

  http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorage.java 1482982
  http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorageUtils.java 1482982
  http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/PigAvroDatumReader.java 1482982
  http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/PigAvroInputFormat.java 1482982
  http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/PigAvroRecordReader.java 1482982
  http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/TestAvroStorage.java 1482982

Diff: https://reviews.apache.org/r/11155/diff/
Testing
-------
Thanks,

Harvey Chong