Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> Review Request 14801: PIG-3531 Add TezMiniCluster for unit tests

Cheolsoo Park 2013-10-21, 23:00
Cheolsoo Park 2013-10-22, 01:02
Copy link to this message
Re: Review Request 14801: PIG-3531 Add TezMiniCluster for unit tests

This is an automatically generated e-mail. To reply, visit:

(Updated Oct. 22, 2013, 9:21 p.m.)
Review request for pig, Daniel Dai, Mark Wagner, and Rohini Palaniswamy.

Thank you Ronihi for the comments. I incorporated them in a new patch
* Moved buildCluster() to MiniGenericCluster class.
* Introduced a system property that controls the mini cluster type.
* Deprecated MiniCluster.buildCluster(). It will be removed after test classes are updated.

In addition, I included PIG-3535, so you can now run the following commands:
* ant test-tez: Runs all the tez unit tests.
* ant test -Dexectype=[tez|mr] -Dtestcase=<testname>: Runs individual unit tests in tez or mr mode.
Bugs: PIG-3531
Repository: pig-git

Add TezMiniCluster that allows us to write unit tests. The TezMiniCluster class extends MiniGenericCluster and set up Tez-specific properties and jars.
Diffs (updated)

  build.xml 1b65894
  shims/test/hadoop20/org/apache/pig/test/MiniCluster.java d6384e9
  shims/test/hadoop20/org/apache/pig/test/TezMiniCluster.java e69de29
  shims/test/hadoop23/org/apache/pig/test/MiniCluster.java 1a4a430
  shims/test/hadoop23/org/apache/pig/test/TezMiniCluster.java e69de29
  test/org/apache/pig/test/MiniGenericCluster.java ac3f5bc
  test/org/apache/pig/test/Util.java 24103da
  test/org/apache/pig/tez/TestTezLauncher.java d4c3c6a
  test/tez-tests e69de29

Diff: https://reviews.apache.org/r/14801/diff/

Updated TestTezLauncher using TezMiniCluster. Now it asserts the output is the same as what's expected.

Cheolsoo Park

Cheolsoo Park 2013-10-22, 22:04
Rohini Palaniswamy 2013-10-22, 17:54