Hello,
I runTestDataModel test case with IBM JDK, and got the following error:
Testcase: testTupleToString took 0.002 sec
        FAILED
toString expected:<...ad a little lamb)},[[hello#world,goodbye#all]],42,5000000000,3.14...> but was:<...ad a little lamb)},[[goodbye#all,hello#world]],42,5000000000,3.14...>
junit.framework.ComparisonFailure: toString expected:<...ad a little lamb)},[[hello#world,goodbye#all]],42,5000000000,3.14...> but was:<...ad a little lamb)},[[goodbye#all,hello#world]],42,5000000000,3.14...>
        at org.apache.pig.test.TestDataModel.testTupleToString(TestDataModel.java:269)
This is because IBM HashMap.toString() output different with SUN HashMap.toString(). My question is:
Should the output must be the one expected?
Are the output expected based on SUN JDK?
Does the failed test case mean pig-0.8.1 can not work with IBM JDk?
Thank you.

Since HashMap does not guarantee ordering, this test is not guaranteed
to pass, ie the test is faulty and it needs to be fixed. The result from
IBM JDK is also correct.
An easy fix for the test case would be to test for both sequences of
hash-map.
Can you please open a jira ?

-Thejas

On 8/16/11 7:56 PM, lulynn_2008 wrote:
>   Hello,
> I runTestDataModel test case with IBM JDK, and got the following error:
> Testcase: testTupleToString took 0.002 sec
>          FAILED
> toString expected:<...ad a little lamb)},[[hello#world,goodbye#all]],42,5000000000,3.14...>  but was:<...ad a little lamb)},[[goodbye#all,hello#world]],42,5000000000,3.14...>
> junit.framework.ComparisonFailure: toString expected:<...ad a little lamb)},[[hello#world,goodbye#all]],42,5000000000,3.14...>  but was:<...ad a little lamb)},[[goodbye#all,hello#world]],42,5000000000,3.14...>
>          at org.apache.pig.test.TestDataModel.testTupleToString(TestDataModel.java:269)
> This is because IBM HashMap.toString() output different with SUN HashMap.toString(). My question is:
> Should the output must be the one expected?
> Are the output expected based on SUN JDK?
> Does the failed test case mean pig-0.8.1 can not work with IBM JDk?
> Thank you.
>
>
>

Thejas Nair,Thank you

Actually there are other unit tests failed because of different order output. During parse the pig command, a HashMap variable "aliases" will be used in src-gen/org/apache/pig/impl/logicalLayer/parser/QueryParser.java. I think this unit test failure is also caused by HashMap problem. What is your opinion?
Here are two failure unit test examples:
TestLogToPhyCompiler:Query: split (load 'a') into x if $0 < '7', y if $0 > '7';
Testcase: testSplit took 0.198 sec
        FAILED
IBM JDK:
x: Filter[tuple] - Test-Plan-Builder-240
|   |
|   Less Than[boolean] - Test-Plan-Builder-243
|   |
|   |---Project[bytearray][0] - Test-Plan-Builder-241
|   |
|   |---Constant(7) - Test-Plan-Builder-242
|
|---Split - Test-Plan-Builder-239
    |
    |---229: Load()

y: Filter[tuple] - Test-Plan-Builder-244
|   |
|   Greater Than[boolean] - Test-Plan-Builder-247
|   |
|   |---Project[bytearray][0] - Test-Plan-Builder-245
|   |
|   |---Constant(7) - Test-Plan-Builder-246
|
|---Split - Test-Plan-Builder-239
    |
    |---229: Load()
SUN JDK:
y: Filter[tuple] - Test-Plan-Builder-240
|   |
|   Greater Than[boolean] - Test-Plan-Builder-243
|   |
|   |---Project[bytearray][0] - Test-Plan-Builder-241
|   |
|   |---Constant(7) - Test-Plan-Builder-242
|
|---Split - Test-Plan-Builder-239
    |
    |---229: Load()

x: Filter[tuple] - Test-Plan-Builder-244
|   |
|   Less Than[boolean] - Test-Plan-Builder-247
|   |
|   |---Project[bytearray][0] - Test-Plan-Builder-245
|   |
|   |---Constant(7) - Test-Plan-Builder-246
|
|---Split - Test-Plan-Builder-239
    |
    |---229: Load()
TestMRCompiler(IBM JDK):
Testcase: testSortUDF1 took 0.02 sec
        FAILED
null expected:<...---MapReduce(20,SUM,[COUNT,TestMRCompiler$WeirdComparator]) - -18:
        |  ...> but was:<...---MapReduce(20,SUM,[TestMRCompiler$WeirdComparator,COUNT]) - -18:
        |  ...>
junit.framework.ComparisonFailure: null expected:<...---MapReduce(20,SUM,[COUNT,TestMRCompiler$WeirdComparator]) - -18:
        |  ...> but was:<...---MapReduce(20,SUM,[TestMRCompiler$WeirdComparator,COUNT]) - -18:
        |  ...>
        at org.apache.pig.test.TestMRCompiler.run(TestMRCompiler.java:1056)
        at org.apache.pig.test.TestMRCompiler.testSortUDF1(TestMRCompiler.java:790)

At 2011-08-18 01:17:02,"Thejas Nair" <[EMAIL PROTECTED]> wrote:
>Since HashMap does not guarantee ordering, this test is not guaranteed
>to pass, ie the test is faulty and it needs to be fixed. The result from
>IBM JDK is also correct.
>An easy fix for the test case would be to test for both sequences of
>hash-map.
>Can you please open a jira ?
>
>-Thejas
>
>
>
>On 8/16/11 7:56 PM, lulynn_2008 wrote:
>>   Hello,
>> I runTestDataModel test case with IBM JDK, and got the following error:
>> Testcase: testTupleToString took 0.002 sec
>>          FAILED
>> toString expected:<...ad a little lamb)},[[hello#world,goodbye#all]],42,5000000000,3.14...>  but was:<...ad a little lamb)},[[goodbye#all,hello#world]],42,5000000000,3.14...>
>> junit.framework.ComparisonFailure: toString expected:<...ad a little lamb)},[[hello#world,goodbye#all]],42,5000000000,3.14...>  but was:<...ad a little lamb)},[[goodbye#all,hello#world]],42,5000000000,3.14...>
>>          at org.apache.pig.test.TestDataModel.testTupleToString(TestDataModel.java:269)
>> This is because IBM HashMap.toString() output different with SUN HashMap.toString(). My question is:
>> Should the output must be the one expected?
>> Are the output expected based on SUN JDK?
>> Does the failed test case mean pig-0.8.1 can not work with IBM JDk?
>> Thank you.
>>
>>
>>
>

yes, even in this case, both outputs look correct.
-Thejas

On 8/18/11 6:05 AM, lulynn_2008 wrote:
> Thejas  Nair,
>
> Thank you
> Actually there are other unit tests failed because of different order
> output. During parse the pig command, a HashMap variable "aliases" will
> be used in
> src-gen/org/apache/pig/impl/logicalLayer/parser/QueryParser.java. I
> think this unit test failure is also caused by HashMap problem. What is
> your opinion?
> Here are two failure unit test examples:
> TestLogToPhyCompiler: Query: split (load 'a') into x if $0 < '7', y if
> $0 > '7';
> Testcase: testSplit took 0.198 sec
> FAILED
> IBM JDK:
> x: Filter[tuple] - Test-Plan-Builder-240
> | |
> | Less Than[boolean] - Test-Plan-Builder-243
> | |
> | |---Project[bytearray][0] - Test-Plan-Builder-241
> | |
> | |---Constant(7) - Test-Plan-Builder-242
> |
> |---Split - Test-Plan-Builder-239
> |
> |---229: Load()
>
> y: Filter[tuple] - Test-Plan-Builder-244
> | |
> | Greater Than[boolean] - Test-Plan-Builder-247
> | |
> | |---Project[bytearray][0] - Test-Plan-Builder-245
> | |
> | |---Constant(7) - Test-Plan-Builder-246
> |
> |---Split - Test-Plan-Builder-239
> |
> |---229: Load()
> SUN JDK:
> y: Filter[tuple] - Test-Plan-Builder-240
> | |
> | Greater Than[boolean] - Test-Plan-Builder-243
> | |
> | |---Project[bytearray][0] - Test-Plan-Builder-241
> | |
> | |---Constant(7) - Test-Plan-Builder-242
> |
> |---Split - Test-Plan-Builder-239
> |
> |---229: Load()
>
> x: Filter[tuple] - Test-Plan-Builder-244
> | |
> | Less Than[boolean] - Test-Plan-Builder-247
> | |
> | |---Project[bytearray][0] - Test-Plan-Builder-245
> | |
> | |---Constant(7) - Test-Plan-Builder-246
> |
> |---Split - Test-Plan-Builder-239
> |
> |---229: Load()
> TestMRCompiler(IBM JDK):
> Testcase: testSortUDF1 took 0.02 sec
> FAILED
> null
> expected:<...---MapReduce(20,SUM,[COUNT,TestMRCompiler$WeirdComparator])
> - -18:
> | ...> but
> was:<...---MapReduce(20,SUM,[TestMRCompiler$WeirdComparator,COUNT]) - -18:
> | ...>
> junit.framework.ComparisonFailure: null
> expected:<...---MapReduce(20,SUM,[COUNT,TestMRCompiler$WeirdComparator])
> - -18:
> | ...> but
> was:<...---MapReduce(20,SUM,[TestMRCompiler$WeirdComparator,COUNT]) - -18:
> | ...>
> at org.apache.pig.test.TestMRCompiler.run(TestMRCompiler.java:1056)
> at org.apache.pig.test.TestMRCompiler.testSortUDF1(TestMRCompiler.java:790)
>
>
>
>
> At  2011-08-18  01:17:02,"Thejas  Nair"  <[EMAIL PROTECTED]>  wrote:
>>Since  HashMap  does  not  guarantee  ordering,  this  test  is  not  guaranteed
>>to  pass,  ie  the  test  is  faulty  and  it  needs  to  be  fixed.  The  result  from
>>IBM  JDK  is  also  correct.
>>An  easy  fix  for  the  test  case  would  be  to  test  for  both  sequences  of
>>hash-map.
>>Can  you  please  open  a  jira  ?
>>
>>-Thejas
>>
>>
>>
>>On  8/16/11  7:56  PM,  lulynn_2008  wrote:
>>>      Hello,
>>>  I  runTestDataModel  test  case  with  IBM  JDK,  and  got  the  following  error:
>>>  Testcase:  testTupleToString  took  0.002  sec
>>>                    FAILED
>>>  toString  expected:<...ad  a  little  lamb)},[[hello#world,goodbye#all]],42,5000000000,3.14...>    but  was:<...ad  a  little  lamb)},[[goodbye#all,hello#world]],42,5000000000,3.14...>
>>>  junit.framework.ComparisonFailure:  toString  expected:<...ad  a  little  lamb)},[[hello#world,goodbye#all]],42,5000000000,3.14...>    but  was:<...ad  a  little  lamb)},[[goodbye#all,hello#world]],42,5000000000,3.14...>
>>>                    at  org.apache.pig.test.TestDataModel.testTupleToString(TestDataModel.java:269)
>>>  This  is  because  IBM  HashMap.toString()  output  different  with  SUN  HashMap.toString().  My  question  is:
>>>  Should  the  output  must  be  the  one  expected?
>>>  Are  the  output  expected  based  on  SUN  JDK?
>>>  Does  the  failed  test  case  mean  pig-0.8.1  can  not  work  with  IBM  JDk?
>>>  Thank  you.
>>>
>>>
>>>
>>
>
>
>

Hello,
For test case TestPruneColumn:
    public void testMapKeyInSplit1() throws Exception {
        pigServer.registerQuery("A = load '"+ Util.generateURI(tmpFile12.toString(), pigServer.getPigContext()) + "' as (m:map[]);");
        pigServer.registerQuery("B = foreach A generate m#'key1' as key1;");
        pigServer.registerQuery("C = foreach A generate m#'key2' as key2;");
        pigServer.registerQuery("D = join B by key1, C by key2;");

        Iterator<Tuple> iter = pigServer.openIterator("D");

        assertTrue(iter.hasNext());
        Tuple t = iter.next();
        assertTrue(t.size()==2);
        assertTrue(t.get(0).toString().equals("2"));
        assertTrue(t.get(1).toString().equals("2"));

        assertFalse(iter.hasNext());

        assertTrue(checkLogFileMessage(new String[]{"Map key required for A: $0->[key2, key1]"}));
    }
I have some questions:
1. What content will be stored in logFile in methond checkLogFileMessage?
2. With other JDK but not SUN JDK, I found the variable item contain string like "Map key required for A: $0->[key1, key2]". I think this is caused by pigServer.registerQuery method which invoke method parse. And parse method is based on HashMap structure.Could you please tell me what HashMap methods will be invoked and cause the different output LogicalPlan in parse method?
    public LogicalPlan parse(String scope,
                             String query,
                             Map<LogicalOperator, LogicalPlan> aliases,
                             Map<OperatorKey, LogicalOperator> opTable,
                             Map<String, LogicalOperator> aliasOp,
                             int start,
                             Map<String, String> fileNameMap)
        throws IOException, ParseException {
        ByteArrayInputStream in = new ByteArrayInputStream(query.getBytes());      
        QueryParser parser = new QueryParser(in, pigContext, scope, aliases, opTable,
                                             aliasOp, start, fileNameMap);
        return parser.Parse();      
    }
3.The same situation also happened in:
Testcase: testMapKey2 took 8.133 sec
Should be:"Columns pruned for A: $0",
                "Map key required for A: $1->[key2, key1]"
But is:"Columns pruned for A: $0"
          "Map key required for A: $1->[key1, key2]"

Testcase: testMapKey3 took 8.815 sec
Should be:{([key2#2,key1#1],1),([key2#4,key1#2],2)}
But is:{([key1#1,key2#2],1),([key1#2,key2#4],2)}
Testcase: testSharedSchemaObject took 9.15 sec
Should be:([2#1,1#1])
But is:([1#1,2#1])
Are these results also correct?
At 2011-08-19 01:50:20,"Thejas Nair" <[EMAIL PROTECTED]> wrote:
>yes, even in this case, both outputs look correct.
>-Thejas
>
>
>
>On 8/18/11 6:05 AM, lulynn_2008 wrote:
>> Thejas  Nair,
>>
>> Thank you
>> Actually there are other unit tests failed because of different order
>> output. During parse the pig command, a HashMap variable "aliases" will
>> be used in
>> src-gen/org/apache/pig/impl/logicalLayer/parser/QueryParser.java. I
>> think this unit test failure is also caused by HashMap problem. What is
>> your opinion?
>> Here are two failure unit test examples:
>> TestLogToPhyCompiler: Query: split (load 'a') into x if $0 < '7', y if
>> $0 > '7';
>> Testcase: testSplit took 0.198 sec
>> FAILED
>> IBM JDK:
>> x: Filter[tuple] - Test-Plan-Builder-240
>> | |
>> | Less Than[boolean] - Test-Plan-Builder-243
>> | |
>> | |---Project[bytearray][0] - Test-Plan-Builder-241
>> | |
>> | |---Constant(7) - Test-Plan-Builder-242
>> |
>> |---Split - Test-Plan-Builder-239
>> |
>> |---229: Load()
>>
>> y: Filter[tuple] - Test-Plan-Builder-244
>> | |
>> | Greater Than[boolean] - Test-Plan-Builder-247
>> | |
>> | |---Project[bytearray][0] - Test-Plan-Builder-245
>> | |
>> | |---Constant(7) - Test-Plan-Builder-246
>> |
>> |---Split - Test-Plan-Builder-239
>> |
>> |---229: Load()
>> SUN JDK:
>> y: Filter[tuple] - Test-Plan-Builder-240
>> | |
>> | Greater Than[boolean] - Test-Plan-Builder-243
>> | |