Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # dev - Review Request: PIG-3346 New property that controls the number of combined splits


Copy link to this message
-
Review Request: PIG-3346 New property that controls the number of combined splits
Cheolsoo Park 2013-06-10, 16:17

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11718/
-----------------------------------------------------------

Review request for pig.
Description
-------

Adds a new property pig.maxCombinedSplitNum. Pig won't combine more splits than the configured limit. By default, it is set to Long.MAX_VALUE.
This addresses bug PIG-3346.
    https://issues.apache.org/jira/browse/PIG-3346
Diffs
-----

  src/docs/src/documentation/content/xdocs/perf.xml 78bb3ed
  src/org/apache/pig/PigConfiguration.java b94e0c8
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java 292c0f1
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MRCompiler.java d785aac
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigInputFormat.java 2266305
  src/org/apache/pig/backend/hadoop/executionengine/util/MapRedUtil.java 0502917
  test/org/apache/pig/test/TestSplitCombine.java 541aae2
  test/org/apache/pig/test/TestSplitIndex.java 4bfdeb4

Diff: https://reviews.apache.org/r/11718/diff/
Testing
-------

All unit tests pass.

A new unit test case is added for the new property.
Thanks,

Cheolsoo Park