Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - about pig-0.8.1 unit test: TestMergeJoinOuter


Copy link to this message
-
Re: about pig-0.8.1 unit test: TestMergeJoinOuter
Ashutosh Chauhan 2011-08-25, 00:58
Hey Lulynn,

Plan certainly looks different. I am surprised it is because of different
JDK. Unlikely, but a bug in your JDK ? Which other test cases, do you see
fail with this JDK?

Ashutosh

2011/8/23 lulynn_2008 <[EMAIL PROTECTED]>

>  Hello Thejas Nair,
> During running ant -Dtestcase=TestMergeJoinOuter test with other JDK but
> not SUN JDK. I found my output is different from the one under SUN JDK:
> SUN JDK: passed
> Other JDK: failed with following information:
> Testcase: testCompilation took 0.443 sec
>        FAILED
> junit.framework.AssertionFailedError:
>        at
> org.apache.pig.test.TestMergeJoinOuter.testCompilation(TestMergeJoinOuter.java:116):assertTrue(oper.reducePlan.isEmpty());
>
> I found the variable oper(in TestMergeJoinOuter.testCompilation) is
> different, here is the initialization of oper:
>            LogicalPlan lp = lpt.buildPlan("C = join A by id left, B by id
> using 'merge';");
>            PigContext pc = new
> PigContext(ExecType.MAPREDUCE,cluster.getProperties());
>            pc.connect();
>            lp = lpt.buildPlan("store C into 'out';");
>            MROperPlan mrPlan = Util.buildMRPlan(Util.buildPhysicalPlan(lp,
> pc),pc);
>            Iterator<MapReduceOper> itr = mrPlan.iterator();
>            MapReduceOper oper = itr.next();
> Here are the output:
> SUN JDK:
> oper:(Name:
> MapReduce(-1,TestMapSideCogroup$DummyCollectableLoader,IsEmpty,PigStorage) -
> scope-39:
> Reduce Plan Empty
> |   C:
> Store(hdfs://localhost.localdomain:33564/user/root/out:org.apache.pig.builtin.PigStorage)
> - scope-38
> |   |
> |   |---C: New For Each(true,true)[tuple] - scope-37
> |       |   |
> |       |   Project[bag][1] - scope-31
> |       |   |
> |       |   POBinCond[bag] - scope-36
> |       |   |
> |       |   |---Project[bag][2] - scope-32
> |       |   |
> |       |   |---POUserFunc(org.apache.pig.builtin.IsEmpty)[boolean] -
> scope-34
> |       |   |   |
> |       |   |   |---Project[bag][2] - scope-33
> |       |   |
> |       |   |---Constant({(,,)}) - scope-35
> |       |
> |       |---C: MergeCogroup[tuple] - scope-30
> |           |
> |           |---A:
> Load(hdfs://localhost.localdomain:33564/user/root/data1:org.apache.pig.test.TestMapSideCogroup$DummyCollectableLoader)
> - scope-22 Operator Key: scope-39)
>
> Other JDK:
> oper:(Name: MapReduce(1) - scope-41:
> |
> Store(hdfs://localhost.localdomain:49160/tmp/temp-1456742965/tmp2077335416:org.apache.pig.impl.io.InterStorage)
> - scope-48
> |   |
> |   |---New For Each(true)[bag] - scope-47
> |       |   |
> |       |   Project[tuple][1] - scope-46
> |       |
> |       |---Package[tuple]{tuple} - scope-45
> |   Local Rearrange[tuple]{tuple}(false) - scope-44
> |   |   |
> |   |   Project[tuple][*] - scope-43
> |   |
> |
> |---Load(hdfs://localhost.localdomain:49160/user/root/data1:org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MergeJoinIndexer('org.apache.pig.test.TestMapSideCogroup$DummyCollectableLoader','kmonaaafhdhcaabdgkgbhggbcohfhegjgmcoebhchcgbhjemgjhdhehiibncbnjjmhgbjnadaaabejaaaehdgjhkgfhihaaaaaaaabhhaeaaaaaaakhdhcaaeogphcghcogbhagbgdgigfcohagjghcogcgbgdglgfgogecogigbgegpgphacogfhigfgdhfhegjgpgogfgoghgjgogfcohagihjhdgjgdgbgmemgbhjgfhccohagmgbgohdcofagihjhdgjgdgbgmfagmgbgoaaaaaaaaaaaaaaabacaaabfkaaangfgogeepggebgmgmejgohahfhehihcaacfgphcghcogbhagbgdgigfcohagjghcogjgnhagmcohagmgbgocoephagfhcgbhegphcfagmgbgoeaondlnechegpmlgacaaaiemaaakgneghcgpgnefgeghgfhdheaacdemgphcghcpgbhagbgdgigfcphagjghcpgjgnhagmcphfhegjgmcpenhfgmhegjengbhadlemaaafgnelgfhjhdheaaapemgkgbhggbcphfhegjgmcpengbhadlemaaahgnemgfgbhggfhdheaabaemgkgbhggbcphfhegjgmcpemgjhdhedlemaaaegnephahdhbaahoaaafemaaaggnfcgpgphehdhbaahoaaagemaaaognfdgpggheeghcgpgnefgeghgfhdhbaahoaaaeemaaamgnfdgpgghefegpefgeghgfhdhbaahoaaa
>
>  eemaaaignfegpefgeghgfhdhbaahoaaaehihahdhcaacbgphcghcogbhagbgdgigfcohagjghcogjgnhagmcohfhegjgmcoenhfgmhegjengbhaaaaaaaaaaaaaaaacacaaabemaaaegnengbhahbaahoaaafhihahdhcaabbgkgbhggbcohfhegjgmcoeigbhdgiengbhaafahnkmbmdbgganbadaaacegaaakgmgpgbgeeggbgdhegphcejaaajhegihcgfhdgigpgmgehihadpeaaaaaaaaaaaamhhaiaaaaaabaaaaaaaaahihdhbaahoaaakdpeaaaaaaaaaaaamhhaiaaaaaabaaaaaaaabhdhcaacegphcghcogbhagbgdgigfcohagjghcogjgnhagmcohagmgbgocoephagfhcgbhegphcelgfhjaaaaaaaaaaaaaaabacaaacekaaacgjgeemaaafhdgdgphagfheaabcemgkgbhggbcpgmgbgoghcpfdhehcgjgoghdlhihaaaaaaaaaaaaaaablheaaafhdgdgphagfhdhcaafjgphcghcogbhagbgdgigfcohagjghcogcgbgdglgfgogecogigbgegpgphacogfhigfgdhfhegjgpgogfgoghgjgogfcohagihjhdgjgdgbgmemgbhjgfhccogfhihahcgfhdhdgjgpgoephagfhcgbhegphchdcofaepfahcgpgkgfgdheaaaaaaaaaaaaaaabacaaaffkaaakgphggfhcgmgpgbgegfgefkaabfhahcgpgdgfhdhdgjgoghecgbghepggfehfhagmgfhdfkaabehcgfhdhfgmhefdgjgoghgmgffehfhagmgfecgbghfkaaaehdhegbhcemaaahgdgpgmhfgngohdheaabfemgkgbhggbcphfhegjgmcpebhchcgbhjemgjhdhedlhihcaagc