Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Review Request 17191: [PIG-3626] Make combiners, custom partitioners and secondary key sort work for multiple outputs


Copy link to this message
-
Review Request 17191: [PIG-3626] Make combiners, custom partitioners and secondary key sort work for multiple outputs

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17191/
-----------------------------------------------------------

Review request for pig, Alex Bain, Cheolsoo Park, Daniel Dai, and Mark Wagner.
Repository: pig
Description
-------

Added support for secondary key sort. Also fixed combiners and custom partitioners to work for multiple outputs. Fixed bugs in couple of places as I encountered them.
Diffs
-----

  http://svn.apache.org/repos/asf/pig/branches/tez/shims/test/hadoop23/org/apache/pig/test/MiniCluster.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/shims/test/hadoop23/org/apache/pig/test/TezMiniCluster.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/PigConfiguration.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/ColumnInfo.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReduceLauncher.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/SecondaryKeyOptimizerMR.java PRE-CREATION
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/optimizer/SecondaryKeyOptimizer.java PRE-CREATION
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/PhysicalOperator.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POLocalRearrange.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POReservoirSample.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/util/PlanHelper.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/CombinerOptimizer.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/POShuffleTezLoad.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/SecondaryKeyOptimizerTez.java PRE-CREATION
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/SkewedPartitionerTez.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezCompiler.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezCompilerUtil.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezEdgeDescriptor.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezOperator.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezPrinter.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/util/CombinerOptimizerUtil.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/util/SecondaryKeyOptimizerUtil.java PRE-CREATION
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/impl/io/NullableTuple.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/impl/io/PigNullableWritable.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/MiniGenericCluster.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/TestAccumulator.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/TestCombiner.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/TestCustomPartitioner.java PRE-CREATION
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/TestEvalPipeline2.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/TestPigServer.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/TestSecondarySort.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/TestSecondarySortMR.java PRE-CREATION
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/TestSkewedJoin.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/TestSplitStore.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/Util.java 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC14.gld PRE-CREATION
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC15.gld PRE-CREATION
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC16.gld PRE-CREATION
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC4.gld 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC5.gld 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC7.gld 1559126
  http://svn.apache.org/repos/asf/pig/branches/tez/t