Environment:
Hadoop-0.20.2: 4 nodes(1 namenode+3 datanode)
ant: 1.8.2
java: sun 1.6_27

Questions:
1. what is your pig e2e test result? If all passed, please give your environment setting(ant, java,hadoop version).
2. what is the differences between FAILED, SKIPPED and ABORTED tests? What most caused tests to be FAILED, SKIPPED or ABORTED?
3. what kind of tests should we pay attention to?

Thank you.

In general we don't do a release unless all pass.  Pig 0.9.1 with Hadoop 0.20.2 should be fine.  If you can give a list of failing and aborting tests and the few lines from the log saying why they failed/aborted that would be helpful.  In the log search for "FAILED at" to find the failed tests and "Failed to run" to find the aborted tests.

Failed tests are ones where results were produced, but they differed from the expected results.  Aborted tests are ones where running the test or generating the benchmark caused an unexpected error (e.g. Pig died during processing).  Skipped tests are tests that were not run because they were not appropriate in the current environment, e.g. some tests are only run in mapreduce mode and will be turned off when testing is being done in local mode.

Alan.

On Feb 20, 2012, at 9:52 PM, lulynn_2008 wrote:

> Environment:
> Hadoop-0.20.2: 4 nodes(1 namenode+3 datanode)
> ant: 1.8.2
> java: sun 1.6_27
>
> Questions:
> 1. what is your pig e2e test result? If all passed, please give your environment setting(ant, java,hadoop version).
> 2. what is the differences between FAILED, SKIPPED and ABORTED tests? What most caused tests to be FAILED, SKIPPED or ABORTED?
> 3. what kind of tests should we pay attention to?
>
> Thank you.

Thank you. As what you said, all the SKIPPED are because of "only run in mapreduce mode and will be turned off when testing is being done in local mode."
BTW, how to run one test each time?

"ant -Dharness.old.pig=old_pig -Dharness.cluster.conf=hadoop_conf_dir -Dharness.cluster.bin=hadoop_script -Dtests.to.run="-t Grunt_6" test-e2e" seems not work.

Here is the log got the failed one:
TEST: Grunt_6
******************************************************
Beginning test Grunt_6 at 1329779784
Returning Pig command /root/pig-e2e/pig-0.9.1/test/e2e/pig/../../../bin/pig
With PIG_CLASSPATH set to /root/pig-e2e/pig-0.9.1/test/e2e/pig/../../../build/ivy/lib/Pig/jython-2.5.0.jar:/root/pig-e2e/hadoop-0.20.2/conf
./test_harness.pl:TestDriverPig::runPigCmdLine Going to run command: (/root/pig-e2e/pig-0.9.1/test/e2e/pig/../../../bin/pig -logfile ./out/pigtest/root/root.1329779782/Grunt_6.log ./out/pigtest/root/root.1329779782/Grunt_6.pig 1> ./out/pigtest/root/root.1329779782/Grunt_6.out/stdout 2> ./out/pigtest/root/root.1329779782/Grunt_6.out/stderr)
./test_harness.pl:TestDriverPig::runPigCmdLine STD OUT IS IN FILE (./out/pigtest/root/root.1329779782/Grunt_6.out/stdout)
./test_harness.pl:TestDriverPig::runPigCmdLine STD ERROR IS IN FILE (./out/pigtest/root/root.1329779782/Grunt_6.out/stderr)
./test_harness.pl:TestDriverPig::runPigCmdLine PIG SCRIPT CONTAINS (./out/pigtest/root/root.1329779782/Grunt_6.pig):

sh touch /bin/bad(this test is to run this cmd?)

STD ERROR CONTAINS:
2012-02-20 15:16:25,155 [main] INFO  org.apache.pig.Main - Logging error messages to: /root/pig-e2e/pig-0.9.1/test/e2e/pig/testdist/out/pigtest/root/root.1329779782/Grunt_6.log(there is no this file)
2012-02-20 15:16:25,293 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://svltest329.svl.ibm.com:9000
2012-02-20 15:16:25,373 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: svltest329.svl.ibm.com:9001

./test_harness.pl::TestDriverPig::compareScript INFO Checking return code against expected <0>
./test_harness.pl::TestDriverPig::compareScript INFO Checking test stderr for regular expression <Permission denied>
./test_harness.pl::TestDriverPig::compareScript INFO Check failed: regex match of <Permission denied> expected in stderr: ./out/pigtest/root/root.1329779782/Grunt_6.out/stderr
INFO: TestDriver::run() at 444:Test Grunt_6 FAILED at 1329779785
Results so far, PASSED: 408 FAILED: 1 SKIPPED: 15 ABORTED: 4 FAILED DEPENDENCY: 0
At 2012-02-21 14:53:57,"Alan Gates" <[EMAIL PROTECTED]> wrote:
>In general we don't do a release unless all pass.  Pig 0.9.1 with Hadoop 0.20.2 should be fine.  If you can give a list of failing and aborting tests and the few lines from the log saying why they failed/aborted that would be helpful.  In the log search for "FAILED at" to find the failed tests and "Failed to run" to find the aborted tests.
>
>Failed tests are ones where results were produced, but they differed from the expected results.  Aborted tests are ones where running the test or generating the benchmark caused an unexpected error (e.g. Pig died during processing).  Skipped tests are tests that were not run because they were not appropriate in the current environment, e.g. some tests are only run in mapreduce mode and will be turned off when testing is being done in local mode.
>
>Alan.
>
>On Feb 20, 2012, at 9:52 PM, lulynn_2008 wrote:
>
>> Environment:
>> Hadoop-0.20.2: 4 nodes(1 namenode+3 datanode)
>> ant: 1.8.2
>> java: sun 1.6_27
>>
>> Questions:
>> 1. what is your pig e2e test result? If all passed, please give your environment setting(ant, java,hadoop version).
>> 2. what is the differences between FAILED, SKIPPED and ABORTED tests? What most caused tests to be FAILED, SKIPPED or ABORTED?
>> 3. what kind of tests should we pay attention to?
>>
>> Thank you.
>

-Dtests.to.run="-t testname" works on my end.  Are you seeing it not run anything or still run everything?

Alan.

On Feb 20, 2012, at 11:22 PM, lulynn_2008 wrote:

> Thank you. As what you said, all the SKIPPED are because of "only run in mapreduce mode and will be turned off when testing is being done in local mode."
> BTW, how to run one test each time?
>
> "ant -Dharness.old.pig=old_pig -Dharness.cluster.conf=hadoop_conf_dir -Dharness.cluster.bin=hadoop_script -Dtests.to.run="-t Grunt_6" test-e2e" seems not work.
>
> Here is the log got the failed one:
> TEST: Grunt_6
> ******************************************************
> Beginning test Grunt_6 at 1329779784
> Returning Pig command /root/pig-e2e/pig-0.9.1/test/e2e/pig/../../../bin/pig
> With PIG_CLASSPATH set to /root/pig-e2e/pig-0.9.1/test/e2e/pig/../../../build/ivy/lib/Pig/jython-2.5.0.jar:/root/pig-e2e/hadoop-0.20.2/conf
> ./test_harness.pl:TestDriverPig::runPigCmdLine Going to run command: (/root/pig-e2e/pig-0.9.1/test/e2e/pig/../../../bin/pig -logfile ./out/pigtest/root/root.1329779782/Grunt_6.log ./out/pigtest/root/root.1329779782/Grunt_6.pig 1> ./out/pigtest/root/root.1329779782/Grunt_6.out/stdout 2> ./out/pigtest/root/root.1329779782/Grunt_6.out/stderr)
> ./test_harness.pl:TestDriverPig::runPigCmdLine STD OUT IS IN FILE (./out/pigtest/root/root.1329779782/Grunt_6.out/stdout)
> ./test_harness.pl:TestDriverPig::runPigCmdLine STD ERROR IS IN FILE (./out/pigtest/root/root.1329779782/Grunt_6.out/stderr)
> ./test_harness.pl:TestDriverPig::runPigCmdLine PIG SCRIPT CONTAINS (./out/pigtest/root/root.1329779782/Grunt_6.pig):
>
> sh touch /bin/bad(this test is to run this cmd?)
>
> STD ERROR CONTAINS:
> 2012-02-20 15:16:25,155 [main] INFO  org.apache.pig.Main - Logging error messages to: /root/pig-e2e/pig-0.9.1/test/e2e/pig/testdist/out/pigtest/root/root.1329779782/Grunt_6.log(there is no this file)
> 2012-02-20 15:16:25,293 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://svltest329.svl.ibm.com:9000
> 2012-02-20 15:16:25,373 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: svltest329.svl.ibm.com:9001
>
> ./test_harness.pl::TestDriverPig::compareScript INFO Checking return code against expected <0>
> ./test_harness.pl::TestDriverPig::compareScript INFO Checking test stderr for regular expression <Permission denied>
> ./test_harness.pl::TestDriverPig::compareScript INFO Check failed: regex match of <Permission denied> expected in stderr: ./out/pigtest/root/root.1329779782/Grunt_6.out/stderr
> INFO: TestDriver::run() at 444:Test Grunt_6 FAILED at 1329779785
> Results so far, PASSED: 408 FAILED: 1 SKIPPED: 15 ABORTED: 4 FAILED DEPENDENCY: 0
>
>
>
>
>
>
> At 2012-02-21 14:53:57,"Alan Gates" <[EMAIL PROTECTED]> wrote:
>> In general we don't do a release unless all pass.  Pig 0.9.1 with Hadoop 0.20.2 should be fine.  If you can give a list of failing and aborting tests and the few lines from the log saying why they failed/aborted that would be helpful.  In the log search for "FAILED at" to find the failed tests and "Failed to run" to find the aborted tests.
>>
>> Failed tests are ones where results were produced, but they differed from the expected results.  Aborted tests are ones where running the test or generating the benchmark caused an unexpected error (e.g. Pig died during processing).  Skipped tests are tests that were not run because they were not appropriate in the current environment, e.g. some tests are only run in mapreduce mode and will be turned off when testing is being done in local mode.
>>
>> Alan.
>>
>> On Feb 20, 2012, at 9:52 PM, lulynn_2008 wrote:
>>
>>> Environment:
>>> Hadoop-0.20.2: 4 nodes(1 namenode+3 datanode)
>>> ant: 1.8.2
>>> java: sun 1.6_27
>>>
>>> Questions:
>>> 1. what is your pig e2e test result? If all passed, please give your environment setting(ant, java,hadoop version).
>>> 2. what is the differences between FAILED, SKIPPED and ABORTED tests? What most caused tests to be FAILED, SKIPPED or ABORTED?

-Dtests.to.run works now. And all tests passed.
The failed one is caused by we use root to run pig e2e.
2 aborted because:
- pig-0.9.1 use /usr/local/bin/perl, but per default install directory is /usr/bin/perl
- "too many files open" error: caused by "ulimit -n" value is not enough.
At 2012-02-24 05:59:01,"Alan Gates" <[EMAIL PROTECTED]> wrote:
>-Dtests.to.run="-t testname" works on my end.  Are you seeing it not run anything or still run everything?
>
>Alan.
>
>On Feb 20, 2012, at 11:22 PM, lulynn_2008 wrote:
>
>> Thank you. As what you said, all the SKIPPED are because of "only run in mapreduce mode and will be turned off when testing is being done in local mode."
>> BTW, how to run one test each time?
>>
>> "ant -Dharness.old.pig=old_pig -Dharness.cluster.conf=hadoop_conf_dir -Dharness.cluster.bin=hadoop_script -Dtests.to.run="-t Grunt_6" test-e2e" seems not work.
>>
>> Here is the log got the failed one:
>> TEST: Grunt_6
>> ******************************************************
>> Beginning test Grunt_6 at 1329779784
>> Returning Pig command /root/pig-e2e/pig-0.9.1/test/e2e/pig/../../../bin/pig
>> With PIG_CLASSPATH set to /root/pig-e2e/pig-0.9.1/test/e2e/pig/../../../build/ivy/lib/Pig/jython-2.5.0.jar:/root/pig-e2e/hadoop-0.20.2/conf
>> ./test_harness.pl:TestDriverPig::runPigCmdLine Going to run command: (/root/pig-e2e/pig-0.9.1/test/e2e/pig/../../../bin/pig -logfile ./out/pigtest/root/root.1329779782/Grunt_6.log ./out/pigtest/root/root.1329779782/Grunt_6.pig 1> ./out/pigtest/root/root.1329779782/Grunt_6.out/stdout 2> ./out/pigtest/root/root.1329779782/Grunt_6.out/stderr)
>> ./test_harness.pl:TestDriverPig::runPigCmdLine STD OUT IS IN FILE (./out/pigtest/root/root.1329779782/Grunt_6.out/stdout)
>> ./test_harness.pl:TestDriverPig::runPigCmdLine STD ERROR IS IN FILE (./out/pigtest/root/root.1329779782/Grunt_6.out/stderr)
>> ./test_harness.pl:TestDriverPig::runPigCmdLine PIG SCRIPT CONTAINS (./out/pigtest/root/root.1329779782/Grunt_6.pig):
>>
>> sh touch /bin/bad(this test is to run this cmd?)
>>
>> STD ERROR CONTAINS:
>> 2012-02-20 15:16:25,155 [main] INFO  org.apache.pig.Main - Logging error messages to: /root/pig-e2e/pig-0.9.1/test/e2e/pig/testdist/out/pigtest/root/root.1329779782/Grunt_6.log(there is no this file)
>> 2012-02-20 15:16:25,293 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://svltest329.svl.ibm.com:9000
>> 2012-02-20 15:16:25,373 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: svltest329.svl.ibm.com:9001
>>
>> ./test_harness.pl::TestDriverPig::compareScript INFO Checking return code against expected <0>
>> ./test_harness.pl::TestDriverPig::compareScript INFO Checking test stderr for regular expression <Permission denied>
>> ./test_harness.pl::TestDriverPig::compareScript INFO Check failed: regex match of <Permission denied> expected in stderr: ./out/pigtest/root/root.1329779782/Grunt_6.out/stderr
>> INFO: TestDriver::run() at 444:Test Grunt_6 FAILED at 1329779785
>> Results so far, PASSED: 408 FAILED: 1 SKIPPED: 15 ABORTED: 4 FAILED DEPENDENCY: 0
>>
>>
>>
>>
>>
>>
>> At 2012-02-21 14:53:57,"Alan Gates" <[EMAIL PROTECTED]> wrote:
>>> In general we don't do a release unless all pass.  Pig 0.9.1 with Hadoop 0.20.2 should be fine.  If you can give a list of failing and aborting tests and the few lines from the log saying why they failed/aborted that would be helpful.  In the log search for "FAILED at" to find the failed tests and "Failed to run" to find the aborted tests.
>>>
>>> Failed tests are ones where results were produced, but they differed from the expected results.  Aborted tests are ones where running the test or generating the benchmark caused an unexpected error (e.g. Pig died during processing).  Skipped tests are tests that were not run because they were not appropriate in the current environment, e.g. some tests are only run in mapreduce mode and will be turned off when testing is being done in local mode.