I just pulled trunk, did ant clean + ant test, and I see this test failing.

It seems that there are some hardcoded paths in there:

Failed to generate logical plan for query [define CMD `perl GroupBy.pl '
   ' 0 1` ship('/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl');A load '/user/pig/tests/data/singlefile/studenttab10k';B = group A by $0;C foreach B {   D = order A by $1;    generate flatten(D);};E = stream C
through CMD;store E into
'/user/pig/out/jianyong.1297238871/ComputeSpec_8.out';] due to
exception: *Failed
to generate logical plan. Nested exception: java.io.IOException: Invalid
ship specification: '/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl'
does not exist!*
junit.framework.AssertionFailedError: Failed to generate logical plan for
query [define CMD `perl GroupBy.pl '  ' 0 1`
ship('/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl');A = load
'/user/pig/tests/data/singlefile/studenttab10k';B = group A by $0;C foreach B {   D = order A by $1;    generate flatten(D);};E = stream C
through CMD;store E into
'/user/pig/out/jianyong.1297238871/ComputeSpec_8.out';] due to exception:
Failed to generate logical plan. Nested exception: java.io.IOException:
Invalid ship specification:
'/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl' does not exist!
        at
org.apache.pig.parser.TestLogicalPlanGenerator.generateLogicalPlan(TestLogicalPlanGenerator.java:70)
        at
org.apache.pig.parser.TestLogicalPlanGenerator.test12(TestLogicalPlanGenerator.java:156)

TestDBStorage in contrib/ is failing too.
Different issue:

java.io.IOException: Cannot create table
        at
org.apache.pig.piggybank.test.storage.TestDBStorage.createTable(TestDBStorage.java:116)
        at
org.apache.pig.piggybank.test.storage.TestDBStorage.setUp(TestDBStorage.java:123)
Caused by: java.sql.SQLException: Table already exists: TTT in statement
[create table ttt]
        at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
        at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
        at org.hsqldb.jdbc.jdbcStatement.executeUpdate(Unknown Source)
        at
org.apache.pig.piggybank.test.storage.TestDBStorage.createTable(TestDBStorage.java:111)
D

On Sun, Mar 6, 2011 at 5:16 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> I just pulled trunk, did ant clean + ant test, and I see this test
> failing.
>
> It seems that there are some hardcoded paths in there:
>
> Failed to generate logical plan for query [define CMD `perl GroupBy.pl '
>      ' 0 1` ship('/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl');A
> = load '/user/pig/tests/data/singlefile/studenttab10k';B = group A by $0;C > foreach B {   D = order A by $1;    generate flatten(D);};E = stream C
> through CMD;store E into
> '/user/pig/out/jianyong.1297238871/ComputeSpec_8.out';] due to exception:
> *Failed to generate logical plan. Nested exception: java.io.IOException:
> Invalid ship specification:
> '/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl' does not exist!*
> junit.framework.AssertionFailedError: Failed to generate logical plan for
> query [define CMD `perl GroupBy.pl '  ' 0 1`
> ship('/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl');A = load
> '/user/pig/tests/data/singlefile/studenttab10k';B = group A by $0;C > foreach B {   D = order A by $1;    generate flatten(D);};E = stream C
> through CMD;store E into
> '/user/pig/out/jianyong.1297238871/ComputeSpec_8.out';] due to exception:
> Failed to generate logical plan. Nested exception: java.io.IOException:
> Invalid ship specification:
> '/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl' does not exist!
>         at
> org.apache.pig.parser.TestLogicalPlanGenerator.generateLogicalPlan(TestLogicalPlanGenerator.java:70)
>         at
> org.apache.pig.parser.TestLogicalPlanGenerator.test12(TestLogicalPlanGenerator.java:156)
>
>

This looks like an environment issue. Dmitry, did you start with a clean tree? The error states that a table already exists so probably not related to the parser changes.

Who owns this function? We should aim for stateless tests.

Olga

-----Original Message-----
From: Dmitriy Ryaboy [mailto:[EMAIL PROTECTED]]
Sent: Sunday, March 06, 2011 6:28 PM
To: [EMAIL PROTECTED]
Subject: Re: Failing Test org.apache.pig.parser.TestLogicalPlanGenerator

TestDBStorage in contrib/ is failing too.
Different issue:

java.io.IOException: Cannot create table
        at
org.apache.pig.piggybank.test.storage.TestDBStorage.createTable(TestDBStorage.java:116)
        at
org.apache.pig.piggybank.test.storage.TestDBStorage.setUp(TestDBStorage.java:123)
Caused by: java.sql.SQLException: Table already exists: TTT in statement
[create table ttt]
        at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
        at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
        at org.hsqldb.jdbc.jdbcStatement.executeUpdate(Unknown Source)
        at
org.apache.pig.piggybank.test.storage.TestDBStorage.createTable(TestDBStorage.java:111)
D

On Sun, Mar 6, 2011 at 5:16 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> I just pulled trunk, did ant clean + ant test, and I see this test
> failing.
>
> It seems that there are some hardcoded paths in there:
>
> Failed to generate logical plan for query [define CMD `perl GroupBy.pl '
>      ' 0 1` ship('/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl');A
> = load '/user/pig/tests/data/singlefile/studenttab10k';B = group A by $0;C > foreach B {   D = order A by $1;    generate flatten(D);};E = stream C
> through CMD;store E into
> '/user/pig/out/jianyong.1297238871/ComputeSpec_8.out';] due to exception:
> *Failed to generate logical plan. Nested exception: java.io.IOException:
> Invalid ship specification:
> '/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl' does not exist!*
> junit.framework.AssertionFailedError: Failed to generate logical plan for
> query [define CMD `perl GroupBy.pl '  ' 0 1`
> ship('/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl');A = load
> '/user/pig/tests/data/singlefile/studenttab10k';B = group A by $0;C > foreach B {   D = order A by $1;    generate flatten(D);};E = stream C
> through CMD;store E into
> '/user/pig/out/jianyong.1297238871/ComputeSpec_8.out';] due to exception:
> Failed to generate logical plan. Nested exception: java.io.IOException:
> Invalid ship specification:
> '/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl' does not exist!
>         at
> org.apache.pig.parser.TestLogicalPlanGenerator.generateLogicalPlan(TestLogicalPlanGenerator.java:70)
>         at
> org.apache.pig.parser.TestLogicalPlanGenerator.test12(TestLogicalPlanGenerator.java:156)
>
>

I am certain I have a clean tree.

TestDBStorage:
The error with the table already existing is caused by another test in the
same test suite creating the table, and not dropping it. Pretty sure this
test has been failing for a while.

TestLogicalPlanGenerator:
Hardcoded path may have started failing recently if we added some existence
checks where there were none before? Or do you guys test in an environment
where all the user accounts exist, and so this wouldn't show up until I ran
it?

One of the tests in TestMultiQueryLocal is failing as well:
Testcase: testMultiQueryWithIllustrate took 0.064 sec
        FAILED
null
junit.framework.AssertionFailedError: null
        at
org.apache.pig.test.TestMultiQueryLocal.testMultiQueryWithIllustrate(TestMultiQueryLocal.java:431)
On Mon, Mar 7, 2011 at 8:21 AM, Olga Natkovich <[EMAIL PROTECTED]> wrote:

> This looks like an environment issue. Dmitry, did you start with a clean
> tree? The error states that a table already exists so probably not related
> to the parser changes.
>
> Who owns this function? We should aim for stateless tests.
>
> Olga
>
> -----Original Message-----
> From: Dmitriy Ryaboy [mailto:[EMAIL PROTECTED]]
> Sent: Sunday, March 06, 2011 6:28 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Failing Test org.apache.pig.parser.TestLogicalPlanGenerator
>
> TestDBStorage in contrib/ is failing too.
> Different issue:
>
> java.io.IOException: Cannot create table
>        at
>
> org.apache.pig.piggybank.test.storage.TestDBStorage.createTable(TestDBStorage.java:116)
>        at
>
> org.apache.pig.piggybank.test.storage.TestDBStorage.setUp(TestDBStorage.java:123)
> Caused by: java.sql.SQLException: Table already exists: TTT in statement
> [create table ttt]
>        at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
>        at org.hsqldb.jdbc.jdbcStatement.fetchResult(Unknown Source)
>        at org.hsqldb.jdbc.jdbcStatement.executeUpdate(Unknown Source)
>        at
>
> org.apache.pig.piggybank.test.storage.TestDBStorage.createTable(TestDBStorage.java:111)
>
>
> D
>
> On Sun, Mar 6, 2011 at 5:16 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:
>
> > I just pulled trunk, did ant clean + ant test, and I see this test
> > failing.
> >
> > It seems that there are some hardcoded paths in there:
> >
> > Failed to generate logical plan for query [define CMD `perl GroupBy.pl '
> >      ' 0 1`
> ship('/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl');A
> > = load '/user/pig/tests/data/singlefile/studenttab10k';B = group A by
> $0;C > > foreach B {   D = order A by $1;    generate flatten(D);};E = stream C
> > through CMD;store E into
> > '/user/pig/out/jianyong.1297238871/ComputeSpec_8.out';] due to exception:
> > *Failed to generate logical plan. Nested exception: java.io.IOException:
> > Invalid ship specification:
> > '/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl' does not exist!*
> > junit.framework.AssertionFailedError: Failed to generate logical plan for
> > query [define CMD `perl GroupBy.pl '  ' 0 1`
> > ship('/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl');A = load
> > '/user/pig/tests/data/singlefile/studenttab10k';B = group A by $0;C > > foreach B {   D = order A by $1;    generate flatten(D);};E = stream C
> > through CMD;store E into
> > '/user/pig/out/jianyong.1297238871/ComputeSpec_8.out';] due to exception:
> > Failed to generate logical plan. Nested exception: java.io.IOException:
> > Invalid ship specification:
> > '/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl' does not exist!
> >         at
> >
> org.apache.pig.parser.TestLogicalPlanGenerator.generateLogicalPlan(TestLogicalPlanGenerator.java:70)
> >         at
> >
> org.apache.pig.parser.TestLogicalPlanGenerator.test12(TestLogicalPlanGenerator.java:156)
> >
> >
>

It hardcoded my working directory. I promise I didn't do that :). Create
PIG-1888 for it.

Daniel

On 03/06/2011 05:16 PM, Dmitriy Ryaboy wrote:
> I just pulled trunk, did ant clean + ant test, and I see this test failing.
>
> It seems that there are some hardcoded paths in there:
>
> Failed to generate logical plan for query [define CMD `perl GroupBy.pl '
>     ' 0 1` ship('/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl');A > load '/user/pig/tests/data/singlefile/studenttab10k';B = group A by $0;C > foreach B {   D = order A by $1;    generate flatten(D);};E = stream C
> through CMD;store E into
> '/user/pig/out/jianyong.1297238871/ComputeSpec_8.out';] due to
> exception: *Failed
> to generate logical plan. Nested exception: java.io.IOException: Invalid
> ship specification: '/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl'
> does not exist!*
> junit.framework.AssertionFailedError: Failed to generate logical plan for
> query [define CMD `perl GroupBy.pl '  ' 0 1`
> ship('/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl');A = load
> '/user/pig/tests/data/singlefile/studenttab10k';B = group A by $0;C > foreach B {   D = order A by $1;    generate flatten(D);};E = stream C
> through CMD;store E into
> '/user/pig/out/jianyong.1297238871/ComputeSpec_8.out';] due to exception:
> Failed to generate logical plan. Nested exception: java.io.IOException:
> Invalid ship specification:
> '/homes/jianyong/pig_harness/libexec/PigTest/GroupBy.pl' does not exist!
>          at
> org.apache.pig.parser.TestLogicalPlanGenerator.generateLogicalPlan(TestLogicalPlanGenerator.java:70)
>          at
> org.apache.pig.parser.TestLogicalPlanGenerator.test12(TestLogicalPlanGenerator.java:156)