Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> about pig-0.8.1 unit test: TestMergeJoinOuter


Copy link to this message
-
Re: about pig-0.8.1 unit test: TestMergeJoinOuter
Hey Lulynn,

Plan certainly looks different. I am surprised it is because of different
JDK. Unlikely, but a bug in your JDK ? Which other test cases, do you see
fail with this JDK?

Ashutosh

2011/8/23 lulynn_2008 <[EMAIL PROTECTED]>

>  Hello Thejas Nair,
> During running ant -Dtestcase=TestMergeJoinOuter test with other JDK but
> not SUN JDK. I found my output is different from the one under SUN JDK:
> SUN JDK: passed
> Other JDK: failed with following information:
> Testcase: testCompilation took 0.443 sec
>        FAILED
> junit.framework.AssertionFailedError:
>        at
> org.apache.pig.test.TestMergeJoinOuter.testCompilation(TestMergeJoinOuter.java:116):assertTrue(oper.reducePlan.isEmpty());
>
> I found the variable oper(in TestMergeJoinOuter.testCompilation) is
> different, here is the initialization of oper:
>            LogicalPlan lp = lpt.buildPlan("C = join A by id left, B by id
> using 'merge';");
>            PigContext pc = new
> PigContext(ExecType.MAPREDUCE,cluster.getProperties());
>            pc.connect();
>            lp = lpt.buildPlan("store C into 'out';");
>            MROperPlan mrPlan = Util.buildMRPlan(Util.buildPhysicalPlan(lp,
> pc),pc);
>            Iterator<MapReduceOper> itr = mrPlan.iterator();
>            MapReduceOper oper = itr.next();
> Here are the output:
> SUN JDK:
> oper:(Name:
> MapReduce(-1,TestMapSideCogroup$DummyCollectableLoader,IsEmpty,PigStorage) -
> scope-39:
> Reduce Plan Empty
> |   C:
> Store(hdfs://localhost.localdomain:33564/user/root/out:org.apache.pig.builtin.PigStorage)
> - scope-38
> |   |
> |   |---C: New For Each(true,true)[tuple] - scope-37
> |       |   |
> |       |   Project[bag][1] - scope-31
> |       |   |
> |       |   POBinCond[bag] - scope-36
> |       |   |
> |       |   |---Project[bag][2] - scope-32
> |       |   |
> |       |   |---POUserFunc(org.apache.pig.builtin.IsEmpty)[boolean] -
> scope-34
> |       |   |   |
> |       |   |   |---Project[bag][2] - scope-33
> |       |   |
> |       |   |---Constant({(,,)}) - scope-35
> |       |
> |       |---C: MergeCogroup[tuple] - scope-30
> |           |
> |           |---A:
> Load(hdfs://localhost.localdomain:33564/user/root/data1:org.apache.pig.test.TestMapSideCogroup$DummyCollectableLoader)
> - scope-22 Operator Key: scope-39)
>
> Other JDK:
> oper:(Name: MapReduce(1) - scope-41:
> |
> Store(hdfs://localhost.localdomain:49160/tmp/temp-1456742965/tmp2077335416:org.apache.pig.impl.io.InterStorage)
> - scope-48
> |   |
> |   |---New For Each(true)[bag] - scope-47
> |       |   |
> |       |   Project[tuple][1] - scope-46
> |       |
> |       |---Package[tuple]{tuple} - scope-45
> |   Local Rearrange[tuple]{tuple}(false) - scope-44
> |   |   |
> |   |   Project[tuple][*] - scope-43
> |   |
> |
> |---Load(hdfs://localhost.localdomain:49160/user/root/data1:org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MergeJoinIndexer('org.apache.pig.test.TestMapSideCogroup$DummyCollectableLoader','kmonaaafhdhcaabdgkgbhggbcohfhegjgmcoebhchcgbhjemgjhdhehiibncbnjjmhgbjnadaaabejaaaehdgjhkgfhihaaaaaaaabhhaeaaaaaaakhdhcaaeogphcghcogbhagbgdgigfcohagjghcogcgbgdglgfgogecogigbgegpgphacogfhigfgdhfhegjgpgogfgoghgjgogfcohagihjhdgjgdgbgmemgbhjgfhccohagmgbgohdcofagihjhdgjgdgbgmfagmgbgoaaaaaaaaaaaaaaabacaaabfkaaangfgogeepggebgmgmejgohahfhehihcaacfgphcghcogbhagbgdgigfcohagjghcogjgnhagmcohagmgbgocoephagfhcgbhegphcfagmgbgoeaondlnechegpmlgacaaaiemaaakgneghcgpgnefgeghgfhdheaacdemgphcghcpgbhagbgdgigfcphagjghcpgjgnhagmcphfhegjgmcpenhfgmhegjengbhadlemaaafgnelgfhjhdheaaapemgkgbhggbcphfhegjgmcpengbhadlemaaahgnemgfgbhggfhdheaabaemgkgbhggbcphfhegjgmcpemgjhdhedlemaaaegnephahdhbaahoaaafemaaaggnfcgpgphehdhbaahoaaagemaaaognfdgpggheeghcgpgnefgeghgfhdhbaahoaaaeemaaamgnfdgpgghefegpefgeghgfhdhbaahoaaa
>
>  eemaaaignfegpefgeghgfhdhbaahoaaaehihahdhcaacbgphcghcogbhagbgdgigfcohagjghcogjgnhagmcohfhegjgmcoenhfgmhegjengbhaaaaaaaaaaaaaaaacacaaabemaaaegnengbhahbaahoaaafhihahdhcaabbgkgbhggbcohfhegjgmcoeigbhdgiengbhaafahnkmbmdbgganbadaaacegaaakgmgpgbgeeggbgdhegphcejaaajhegihcgfhdgigpgmgehihadpeaaaaaaaaaaaamhhaiaaaaaabaaaaaaaaahihdhbaahoaaakdpeaaaaaaaaaaaamhhaiaaaaaabaaaaaaaabhdhcaacegphcghcogbhagbgdgigfcohagjghcogjgnhagmcohagmgbgocoephagfhcgbhegphcelgfhjaaaaaaaaaaaaaaabacaaacekaaacgjgeemaaafhdgdgphagfheaabcemgkgbhggbcpgmgbgoghcpfdhehcgjgoghdlhihaaaaaaaaaaaaaaablheaaafhdgdgphagfhdhcaafjgphcghcogbhagbgdgigfcohagjghcogcgbgdglgfgogecogigbgegpgphacogfhigfgdhfhegjgpgogfgoghgjgogfcohagihjhdgjgdgbgmemgbhjgfhccogfhihahcgfhdhdgjgpgoephagfhcgbhegphchdcofaepfahcgpgkgfgdheaaaaaaaaaaaaaaabacaaaffkaaakgphggfhcgmgpgbgegfgefkaabfhahcgpgdgfhdhdgjgoghecgbghepggfehfhagmgfhdfkaabehcgfhdhfgmhefdgjgoghgmgffehfhagmgfecgbghfkaaaehdhegbhcemaaahgdgpgmhfgngohdheaabfemgkgbhggbcphfhegjgmcpebhchcgbhjemgjhdhedlhihcaagc
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB