Is anyone else seeing a bunch of errors when trying to run the pig or
piggybank unit tests? I had to change ivy to use xercesImpl 2.9.1 instead
of xerces 2.4.4 to make this exception go away:

12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
setXIncludeAware(true) for parser
org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364:java.lang.UnsupportedOperationException:
This parser does not support specification "null" version "null"
java.lang.UnsupportedOperationException: This parser does not support
specification "null" version "null"
        at
javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
        at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1117)
        at
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1103)
        at
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)
        at
org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1079)
        at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.recomputeProperties(HExecutionEngine.java:366)
        at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:177)
        at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:119)
        at org.apache.pig.impl.PigContext.connect(PigContext.java:206)
        at org.apache.pig.PigServer.<init>(PigServer.java:246)
        at org.apache.pig.PigServer.<init>(PigServer.java:231)
        at org.apache.pig.PigServer.<init>(PigServer.java:227)
        at org.apache.pig.PigServer.<init>(PigServer.java:223)
...

thanks,
Bill

Is that something new? Are you testing trunk? What's the revision number?
Could that relate to 205 upgrade?
(PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
Daniel

On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <[EMAIL PROTECTED]> wrote:

> Is anyone else seeing a bunch of errors when trying to run the pig or
> piggybank unit tests? I had to change ivy to use xercesImpl 2.9.1 instead
> of xerces 2.4.4 to make this exception go away:
>
> 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
> setXIncludeAware(true) for parser
> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
> :java.lang.UnsupportedOperationException:
> This parser does not support specification "null" version "null"
> java.lang.UnsupportedOperationException: This parser does not support
> specification "null" version "null"
>        at
>
> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
>        at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1117)
>        at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1103)
>        at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)
>        at
> org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1079)
>        at
>
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.recomputeProperties(HExecutionEngine.java:366)
>        at
>
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:177)
>        at
>
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:119)
>        at org.apache.pig.impl.PigContext.connect(PigContext.java:206)
>        at org.apache.pig.PigServer.<init>(PigServer.java:246)
>        at org.apache.pig.PigServer.<init>(PigServer.java:231)
>        at org.apache.pig.PigServer.<init>(PigServer.java:227)
>        at org.apache.pig.PigServer.<init>(PigServer.java:223)
> ...
>
> thanks,
> Bill
>

Aha! I've been having a bunch of unit tests fail mysteriously, and it
started Friday..and they've been giving local filesystem permissions
errors. Seems like it is related to that. I was going crazy.

2012/1/9 Daniel Dai <[EMAIL PROTECTED]>

> Is that something new? Are you testing trunk? What's the revision number?
> Could that relate to 205 upgrade?
> (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
>
>
> Daniel
>
> On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
>
> > Is anyone else seeing a bunch of errors when trying to run the pig or
> > piggybank unit tests? I had to change ivy to use xercesImpl 2.9.1 instead
> > of xerces 2.4.4 to make this exception go away:
> >
> > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
> > setXIncludeAware(true) for parser
> > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
> > :java.lang.UnsupportedOperationException:
> > This parser does not support specification "null" version "null"
> > java.lang.UnsupportedOperationException: This parser does not support
> > specification "null" version "null"
> >        at
> >
> >
> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
> >        at
> >
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1117)
> >        at
> >
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1103)
> >        at
> > org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)
> >        at
> > org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1079)
> >        at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.recomputeProperties(HExecutionEngine.java:366)
> >        at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:177)
> >        at
> >
> >
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:119)
> >        at org.apache.pig.impl.PigContext.connect(PigContext.java:206)
> >        at org.apache.pig.PigServer.<init>(PigServer.java:246)
> >        at org.apache.pig.PigServer.<init>(PigServer.java:231)
> >        at org.apache.pig.PigServer.<init>(PigServer.java:227)
> >        at org.apache.pig.PigServer.<init>(PigServer.java:223)
> > ...
> >
> > thanks,
> > Bill
> >
>

I just dusted my repo checkout off after not using it for a while and I now
see these errors. I was wondering the same thing about PIG-2431. Let me see
if I can reproduce from a fresh SVN checkout. I'm working off a cleaned and
remotely pulled git repos.
On Mon, Jan 9, 2012 at 4:37 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> Is that something new? Are you testing trunk? What's the revision number?
> Could that relate to 205 upgrade? (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
>
>
> Daniel
>
>
> On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
>
>> Is anyone else seeing a bunch of errors when trying to run the pig or
>> piggybank unit tests? I had to change ivy to use xercesImpl 2.9.1 instead
>> of xerces 2.4.4 to make this exception go away:
>>
>> 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
>> setXIncludeAware(true) for parser
>> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
>> :java.lang.UnsupportedOperationException:
>> This parser does not support specification "null" version "null"
>> java.lang.UnsupportedOperationException: This parser does not support
>> specification "null" version "null"
>>        at
>>
>> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
>>        at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1117)
>>        at
>>
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1103)
>>        at
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)
>>        at
>> org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1079)
>>        at
>>
>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.recomputeProperties(HExecutionEngine.java:366)
>>        at
>>
>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:177)
>>        at
>>
>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:119)
>>        at org.apache.pig.impl.PigContext.connect(PigContext.java:206)
>>        at org.apache.pig.PigServer.<init>(PigServer.java:246)
>>        at org.apache.pig.PigServer.<init>(PigServer.java:231)
>>        at org.apache.pig.PigServer.<init>(PigServer.java:227)
>>        at org.apache.pig.PigServer.<init>(PigServer.java:223)
>> ...
>>
>> thanks,
>> Bill
>>
>
>

Hi, Jonathan, can you paste your error message?

Daniel

On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <[EMAIL PROTECTED]> wrote:

> Aha! I've been having a bunch of unit tests fail mysteriously, and it
> started Friday..and they've been giving local filesystem permissions
> errors. Seems like it is related to that. I was going crazy.
>
> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>
> > Is that something new? Are you testing trunk? What's the revision number?
> > Could that relate to 205 upgrade?
> > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
> >
> >
> > Daniel
> >
> > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <[EMAIL PROTECTED]>
> wrote:
> >
> > > Is anyone else seeing a bunch of errors when trying to run the pig or
> > > piggybank unit tests? I had to change ivy to use xercesImpl 2.9.1
> instead
> > > of xerces 2.4.4 to make this exception go away:
> > >
> > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
> > > setXIncludeAware(true) for parser
> > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
> > > :java.lang.UnsupportedOperationException:
> > > This parser does not support specification "null" version "null"
> > > java.lang.UnsupportedOperationException: This parser does not support
> > > specification "null" version "null"
> > >        at
> > >
> > >
> >
> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
> > >        at
> > >
> >
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1117)
> > >        at
> > >
> >
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1103)
> > >        at
> > > org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)
> > >        at
> > > org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1079)
> > >        at
> > >
> > >
> >
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.recomputeProperties(HExecutionEngine.java:366)
> > >        at
> > >
> > >
> >
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:177)
> > >        at
> > >
> > >
> >
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:119)
> > >        at org.apache.pig.impl.PigContext.connect(PigContext.java:206)
> > >        at org.apache.pig.PigServer.<init>(PigServer.java:246)
> > >        at org.apache.pig.PigServer.<init>(PigServer.java:231)
> > >        at org.apache.pig.PigServer.<init>(PigServer.java:227)
> > >        at org.apache.pig.PigServer.<init>(PigServer.java:223)
> > > ...
> > >
> > > thanks,
> > > Bill
> > >
> >
>

Attached is the log for ant -Dtestcase=TestOrderBy test on a clean clone of
trunk.

2012/1/9 Daniel Dai <[EMAIL PROTECTED]>

> Hi, Jonathan, can you paste your error message?
>
> Daniel
>
> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> wrote:
>
> > Aha! I've been having a bunch of unit tests fail mysteriously, and it
> > started Friday..and they've been giving local filesystem permissions
> > errors. Seems like it is related to that. I was going crazy.
> >
> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >
> > > Is that something new? Are you testing trunk? What's the revision
> number?
> > > Could that relate to 205 upgrade?
> > > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
> > >
> > >
> > > Daniel
> > >
> > > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > Is anyone else seeing a bunch of errors when trying to run the pig or
> > > > piggybank unit tests? I had to change ivy to use xercesImpl 2.9.1
> > instead
> > > > of xerces 2.4.4 to make this exception go away:
> > > >
> > > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
> > > > setXIncludeAware(true) for parser
> > > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
> > > > :java.lang.UnsupportedOperationException:
> > > > This parser does not support specification "null" version "null"
> > > > java.lang.UnsupportedOperationException: This parser does not support
> > > > specification "null" version "null"
> > > >        at
> > > >
> > > >
> > >
> >
> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
> > > >        at
> > > >
> > >
> >
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1117)
> > > >        at
> > > >
> > >
> >
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1103)
> > > >        at
> > > >
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)
> > > >        at
> > > >
> org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1079)
> > > >        at
> > > >
> > > >
> > >
> >
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.recomputeProperties(HExecutionEngine.java:366)
> > > >        at
> > > >
> > > >
> > >
> >
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:177)
> > > >        at
> > > >
> > > >
> > >
> >
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:119)
> > > >        at org.apache.pig.impl.PigContext.connect(PigContext.java:206)
> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:246)
> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:231)
> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:227)
> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:223)
> > > > ...
> > > >
> > > > thanks,
> > > > Bill
> > > >
> > >
> >
>

Thanks Jonathan,
I saw this in the log:
Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney, expected:
file:///

Where does this host come from? Is there a hadoop config file in your
CLASSPATH?

Daniel

On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <[EMAIL PROTECTED]> wrote:

> Attached is the log for ant -Dtestcase=TestOrderBy test on a clean clone
> of trunk.
>
> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>
>> Hi, Jonathan, can you paste your error message?
>>
>> Daniel
>>
>> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <[EMAIL PROTECTED]>
>> wrote:
>>
>> > Aha! I've been having a bunch of unit tests fail mysteriously, and it
>> > started Friday..and they've been giving local filesystem permissions
>> > errors. Seems like it is related to that. I was going crazy.
>> >
>> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>> >
>> > > Is that something new? Are you testing trunk? What's the revision
>> number?
>> > > Could that relate to 205 upgrade?
>> > > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
>> > >
>> > >
>> > > Daniel
>> > >
>> > > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <[EMAIL PROTECTED]>
>> > wrote:
>> > >
>> > > > Is anyone else seeing a bunch of errors when trying to run the pig
>> or
>> > > > piggybank unit tests? I had to change ivy to use xercesImpl 2.9.1
>> > instead
>> > > > of xerces 2.4.4 to make this exception go away:
>> > > >
>> > > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
>> > > > setXIncludeAware(true) for parser
>> > > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
>> > > > :java.lang.UnsupportedOperationException:
>> > > > This parser does not support specification "null" version "null"
>> > > > java.lang.UnsupportedOperationException: This parser does not
>> support
>> > > > specification "null" version "null"
>> > > >        at
>> > > >
>> > > >
>> > >
>> >
>> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
>> > > >        at
>> > > >
>> > >
>> >
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1117)
>> > > >        at
>> > > >
>> > >
>> >
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1103)
>> > > >        at
>> > > >
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)
>> > > >        at
>> > > >
>> org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1079)
>> > > >        at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.recomputeProperties(HExecutionEngine.java:366)
>> > > >        at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:177)
>> > > >        at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:119)
>> > > >        at
>> org.apache.pig.impl.PigContext.connect(PigContext.java:206)
>> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:246)
>> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:231)
>> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:227)
>> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:223)
>> > > > ...
>> > > >
>> > > > thanks,
>> > > > Bill
>> > > >
>> > >
>> >
>>
>
>

In my /etc/hosts, oauth.localhost.twitter.com is redirected to
127.0.0.1...but maybe the presence of this mapping makes it think that the
cluster is remote, and not local? I should note that I've always had this
set in my /etc/hosts, but only Friday started having issues.

I removed just that line, and it chose a random other one. I removed all of
the mappings to 127.0.0.1 except localhost, and here is the error I got:

org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Wrong FS:
hdfs://localhost:60145/user/jcoveney, expected: file:///

Just to see, I got rid of all of my mappings in /etc/hosts (though mapping
localhost to 127.0.0.1 is totally valid), and got this error:

org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Wrong FS:
hdfs://127.0.0.1:60420/user/jcoveney, expected: file:///

So It's not related to that. For whatever reason, it's treating it like an
hdfs file system instead of just going for file:///

2012/1/9 Daniel Dai <[EMAIL PROTECTED]>

> Thanks Jonathan,
> I saw this in the log:
> Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
> expected:
> file:///
>
> Where does this host come from? Is there a hadoop config file in your
> CLASSPATH?
>
> Daniel
>
> On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> wrote:
>
> > Attached is the log for ant -Dtestcase=TestOrderBy test on a clean clone
> > of trunk.
> >
> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >
> >> Hi, Jonathan, can you paste your error message?
> >>
> >> Daniel
> >>
> >> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> >> wrote:
> >>
> >> > Aha! I've been having a bunch of unit tests fail mysteriously, and it
> >> > started Friday..and they've been giving local filesystem permissions
> >> > errors. Seems like it is related to that. I was going crazy.
> >> >
> >> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >> >
> >> > > Is that something new? Are you testing trunk? What's the revision
> >> number?
> >> > > Could that relate to 205 upgrade?
> >> > > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
> >> > >
> >> > >
> >> > > Daniel
> >> > >
> >> > > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <[EMAIL PROTECTED]>
> >> > wrote:
> >> > >
> >> > > > Is anyone else seeing a bunch of errors when trying to run the pig
> >> or
> >> > > > piggybank unit tests? I had to change ivy to use xercesImpl 2.9.1
> >> > instead
> >> > > > of xerces 2.4.4 to make this exception go away:
> >> > > >
> >> > > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
> >> > > > setXIncludeAware(true) for parser
> >> > > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
> >> > > > :java.lang.UnsupportedOperationException:
> >> > > > This parser does not support specification "null" version "null"
> >> > > > java.lang.UnsupportedOperationException: This parser does not
> >> support
> >> > > > specification "null" version "null"
> >> > > >        at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
> >> > > >        at
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1117)
> >> > > >        at
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1103)
> >> > > >        at
> >> > > >
> >> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)
> >> > > >        at
> >> > > >
> >> org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1079)
> >> > > >        at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.recomputeProperties(HExecutionEngine.java:366)
> >> > > >        at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:177)
> >> > > >        at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:119)

I just rolled back 205 changes. Can you try if tests pass?

Thanks,
Daniel

On Mon, Jan 9, 2012 at 6:57 PM, Jonathan Coveney <[EMAIL PROTECTED]> wrote:

> In my /etc/hosts, oauth.localhost.twitter.com is redirected to
> 127.0.0.1...but maybe the presence of this mapping makes it think that the
> cluster is remote, and not local? I should note that I've always had this
> set in my /etc/hosts, but only Friday started having issues.
>
> I removed just that line, and it chose a random other one. I removed all of
> the mappings to 127.0.0.1 except localhost, and here is the error I got:
>
> org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Wrong FS:
> hdfs://localhost:60145/user/jcoveney, expected: file:///
>
> Just to see, I got rid of all of my mappings in /etc/hosts (though mapping
> localhost to 127.0.0.1 is totally valid), and got this error:
>
> org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Wrong FS:
> hdfs://127.0.0.1:60420/user/jcoveney, expected: file:///
>
> So It's not related to that. For whatever reason, it's treating it like an
> hdfs file system instead of just going for file:///
>
> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>
> > Thanks Jonathan,
> > I saw this in the log:
> > Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
> > expected:
> > file:///
> >
> > Where does this host come from? Is there a hadoop config file in your
> > CLASSPATH?
> >
> > Daniel
> >
> > On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Attached is the log for ant -Dtestcase=TestOrderBy test on a clean
> clone
> > > of trunk.
> > >
> > > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> > >
> > >> Hi, Jonathan, can you paste your error message?
> > >>
> > >> Daniel
> > >>
> > >> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> > >> wrote:
> > >>
> > >> > Aha! I've been having a bunch of unit tests fail mysteriously, and
> it
> > >> > started Friday..and they've been giving local filesystem permissions
> > >> > errors. Seems like it is related to that. I was going crazy.
> > >> >
> > >> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> > >> >
> > >> > > Is that something new? Are you testing trunk? What's the revision
> > >> number?
> > >> > > Could that relate to 205 upgrade?
> > >> > > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
> > >> > >
> > >> > >
> > >> > > Daniel
> > >> > >
> > >> > > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <[EMAIL PROTECTED]
> >
> > >> > wrote:
> > >> > >
> > >> > > > Is anyone else seeing a bunch of errors when trying to run the
> pig
> > >> or
> > >> > > > piggybank unit tests? I had to change ivy to use xercesImpl
> 2.9.1
> > >> > instead
> > >> > > > of xerces 2.4.4 to make this exception go away:
> > >> > > >
> > >> > > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
> > >> > > > setXIncludeAware(true) for parser
> > >> > > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
> > >> > > > :java.lang.UnsupportedOperationException:
> > >> > > > This parser does not support specification "null" version "null"
> > >> > > > java.lang.UnsupportedOperationException: This parser does not
> > >> support
> > >> > > > specification "null" version "null"
> > >> > > >        at
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
> > >> > > >        at
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1117)
> > >> > > >        at
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1103)
> > >> > > >        at
> > >> > > >
> > >> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)
> > >> > > >        at
> > >> > > >
> > >> org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1079)
> > >> > > >        at
> > >> > > >
> > >> > > >

Running now, will let you know.

2012/1/9 Daniel Dai <[EMAIL PROTECTED]>

> I just rolled back 205 changes. Can you try if tests pass?
>
> Thanks,
> Daniel
>
> On Mon, Jan 9, 2012 at 6:57 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> wrote:
>
> > In my /etc/hosts, oauth.localhost.twitter.com is redirected to
> > 127.0.0.1...but maybe the presence of this mapping makes it think that
> the
> > cluster is remote, and not local? I should note that I've always had this
> > set in my /etc/hosts, but only Friday started having issues.
> >
> > I removed just that line, and it chose a random other one. I removed all
> of
> > the mappings to 127.0.0.1 except localhost, and here is the error I got:
> >
> > org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Wrong
> FS:
> > hdfs://localhost:60145/user/jcoveney, expected: file:///
> >
> > Just to see, I got rid of all of my mappings in /etc/hosts (though
> mapping
> > localhost to 127.0.0.1 is totally valid), and got this error:
> >
> > org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Wrong
> FS:
> > hdfs://127.0.0.1:60420/user/jcoveney, expected: file:///
> >
> > So It's not related to that. For whatever reason, it's treating it like
> an
> > hdfs file system instead of just going for file:///
> >
> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >
> > > Thanks Jonathan,
> > > I saw this in the log:
> > > Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
> > > expected:
> > > file:///
> > >
> > > Where does this host come from? Is there a hadoop config file in your
> > > CLASSPATH?
> > >
> > > Daniel
> > >
> > > On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> > > wrote:
> > >
> > > > Attached is the log for ant -Dtestcase=TestOrderBy test on a clean
> > clone
> > > > of trunk.
> > > >
> > > > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> > > >
> > > >> Hi, Jonathan, can you paste your error message?
> > > >>
> > > >> Daniel
> > > >>
> > > >> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <
> [EMAIL PROTECTED]>
> > > >> wrote:
> > > >>
> > > >> > Aha! I've been having a bunch of unit tests fail mysteriously, and
> > it
> > > >> > started Friday..and they've been giving local filesystem
> permissions
> > > >> > errors. Seems like it is related to that. I was going crazy.
> > > >> >
> > > >> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> > > >> >
> > > >> > > Is that something new? Are you testing trunk? What's the
> revision
> > > >> number?
> > > >> > > Could that relate to 205 upgrade?
> > > >> > > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
> > > >> > >
> > > >> > >
> > > >> > > Daniel
> > > >> > >
> > > >> > > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <
> [EMAIL PROTECTED]
> > >
> > > >> > wrote:
> > > >> > >
> > > >> > > > Is anyone else seeing a bunch of errors when trying to run the
> > pig
> > > >> or
> > > >> > > > piggybank unit tests? I had to change ivy to use xercesImpl
> > 2.9.1
> > > >> > instead
> > > >> > > > of xerces 2.4.4 to make this exception go away:
> > > >> > > >
> > > >> > > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
> > > >> > > > setXIncludeAware(true) for parser
> > > >> > > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
> > > >> > > > :java.lang.UnsupportedOperationException:
> > > >> > > > This parser does not support specification "null" version
> "null"
> > > >> > > > java.lang.UnsupportedOperationException: This parser does not
> > > >> support
> > > >> > > > specification "null" version "null"
> > > >> > > >        at
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
> > > >> > > >        at
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1117)
> > > >> > > >        at
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1103)

I'm not getting the file permissions error anymore. I tried TestOrderBy
(which still has some failing tests, but not related to this) and TestUDF.

2012/1/9 Jonathan Coveney <[EMAIL PROTECTED]>

> Running now, will let you know.
>
>
> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>
>> I just rolled back 205 changes. Can you try if tests pass?
>>
>> Thanks,
>> Daniel
>>
>> On Mon, Jan 9, 2012 at 6:57 PM, Jonathan Coveney <[EMAIL PROTECTED]>
>> wrote:
>>
>> > In my /etc/hosts, oauth.localhost.twitter.com is redirected to
>> > 127.0.0.1...but maybe the presence of this mapping makes it think that
>> the
>> > cluster is remote, and not local? I should note that I've always had
>> this
>> > set in my /etc/hosts, but only Friday started having issues.
>> >
>> > I removed just that line, and it chose a random other one. I removed
>> all of
>> > the mappings to 127.0.0.1 except localhost, and here is the error I got:
>> >
>> > org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Wrong
>> FS:
>> > hdfs://localhost:60145/user/jcoveney, expected: file:///
>> >
>> > Just to see, I got rid of all of my mappings in /etc/hosts (though
>> mapping
>> > localhost to 127.0.0.1 is totally valid), and got this error:
>> >
>> > org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Wrong
>> FS:
>> > hdfs://127.0.0.1:60420/user/jcoveney, expected: file:///
>> >
>> > So It's not related to that. For whatever reason, it's treating it like
>> an
>> > hdfs file system instead of just going for file:///
>> >
>> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>> >
>> > > Thanks Jonathan,
>> > > I saw this in the log:
>> > > Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
>> > > expected:
>> > > file:///
>> > >
>> > > Where does this host come from? Is there a hadoop config file in your
>> > > CLASSPATH?
>> > >
>> > > Daniel
>> > >
>> > > On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <[EMAIL PROTECTED]>
>> > > wrote:
>> > >
>> > > > Attached is the log for ant -Dtestcase=TestOrderBy test on a clean
>> > clone
>> > > > of trunk.
>> > > >
>> > > > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>> > > >
>> > > >> Hi, Jonathan, can you paste your error message?
>> > > >>
>> > > >> Daniel
>> > > >>
>> > > >> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <
>> [EMAIL PROTECTED]>
>> > > >> wrote:
>> > > >>
>> > > >> > Aha! I've been having a bunch of unit tests fail mysteriously,
>> and
>> > it
>> > > >> > started Friday..and they've been giving local filesystem
>> permissions
>> > > >> > errors. Seems like it is related to that. I was going crazy.
>> > > >> >
>> > > >> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>> > > >> >
>> > > >> > > Is that something new? Are you testing trunk? What's the
>> revision
>> > > >> number?
>> > > >> > > Could that relate to 205 upgrade?
>> > > >> > > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
>> > > >> > >
>> > > >> > >
>> > > >> > > Daniel
>> > > >> > >
>> > > >> > > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <
>> [EMAIL PROTECTED]
>> > >
>> > > >> > wrote:
>> > > >> > >
>> > > >> > > > Is anyone else seeing a bunch of errors when trying to run
>> the
>> > pig
>> > > >> or
>> > > >> > > > piggybank unit tests? I had to change ivy to use xercesImpl
>> > 2.9.1
>> > > >> > instead
>> > > >> > > > of xerces 2.4.4 to make this exception go away:
>> > > >> > > >
>> > > >> > > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
>> > > >> > > > setXIncludeAware(true) for parser
>> > > >> > > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
>> > > >> > > > :java.lang.UnsupportedOperationException:
>> > > >> > > > This parser does not support specification "null" version
>> "null"
>> > > >> > > > java.lang.UnsupportedOperationException: This parser does not
>> > > >> support
>> > > >> > > > specification "null" version "null"
>> > > >> > > >        at
>> > > >> > > >
>> > > >> > > >
>> > > >> > >
>> > > >> >
>> > > >>
>> > >
>> >
>> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)

FYI, I'm still getting the same errors after a fresh svn checkout and the
following:

$ ant package
$ and compile-test
$ cd contrib/piggybank/java
$ ant test -Dtestcase=TestAvroStorage

Logs are attached. The same errors appear for other tests in core Pig as
well.

On Mon, Jan 9, 2012 at 6:01 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> Thanks Jonathan,
> I saw this in the log:
> Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
> expected:
> file:///
>
> Where does this host come from? Is there a hadoop config file in your
> CLASSPATH?
>
> Daniel
>
> On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> wrote:
>
> > Attached is the log for ant -Dtestcase=TestOrderBy test on a clean clone
> > of trunk.
> >
> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >
> >> Hi, Jonathan, can you paste your error message?
> >>
> >> Daniel
> >>
> >> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> >> wrote:
> >>
> >> > Aha! I've been having a bunch of unit tests fail mysteriously, and it
> >> > started Friday..and they've been giving local filesystem permissions
> >> > errors. Seems like it is related to that. I was going crazy.
> >> >
> >> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >> >
> >> > > Is that something new? Are you testing trunk? What's the revision
> >> number?
> >> > > Could that relate to 205 upgrade?
> >> > > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
> >> > >
> >> > >
> >> > > Daniel
> >> > >
> >> > > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <[EMAIL PROTECTED]>
> >> > wrote:
> >> > >
> >> > > > Is anyone else seeing a bunch of errors when trying to run the pig
> >> or
> >> > > > piggybank unit tests? I had to change ivy to use xercesImpl 2.9.1
> >> > instead
> >> > > > of xerces 2.4.4 to make this exception go away:
> >> > > >
> >> > > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
> >> > > > setXIncludeAware(true) for parser
> >> > > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
> >> > > > :java.lang.UnsupportedOperationException:
> >> > > > This parser does not support specification "null" version "null"
> >> > > > java.lang.UnsupportedOperationException: This parser does not
> >> support
> >> > > > specification "null" version "null"
> >> > > >        at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
> >> > > >        at
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1117)
> >> > > >        at
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1103)
> >> > > >        at
> >> > > >
> >> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)
> >> > > >        at
> >> > > >
> >> org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1079)
> >> > > >        at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.recomputeProperties(HExecutionEngine.java:366)
> >> > > >        at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:177)
> >> > > >        at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:119)
> >> > > >        at
> >> org.apache.pig.impl.PigContext.connect(PigContext.java:206)
> >> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:246)
> >> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:231)
> >> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:227)
> >> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:223)
> >> > > > ...
> >> > > >
> >> > > > thanks,
> >> > > > Bill
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Are you checking out trunk?

On Mon, Jan 9, 2012 at 8:21 PM, Bill Graham <[EMAIL PROTECTED]> wrote:

> FYI, I'm still getting the same errors after a fresh svn checkout and the
> following:
>
> $ ant package
> $ and compile-test
> $ cd contrib/piggybank/java
> $ ant test -Dtestcase=TestAvroStorage
>
> Logs are attached. The same errors appear for other tests in core Pig as
> well.
>
>
> On Mon, Jan 9, 2012 at 6:01 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:
>
>> Thanks Jonathan,
>> I saw this in the log:
>> Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
>> expected:
>> file:///
>>
>> Where does this host come from? Is there a hadoop config file in your
>> CLASSPATH?
>>
>> Daniel
>>
>> On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <[EMAIL PROTECTED]>
>> wrote:
>>
>> > Attached is the log for ant -Dtestcase=TestOrderBy test on a clean clone
>> > of trunk.
>> >
>> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>> >
>> >> Hi, Jonathan, can you paste your error message?
>> >>
>> >> Daniel
>> >>
>> >> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <[EMAIL PROTECTED]>
>> >> wrote:
>> >>
>> >> > Aha! I've been having a bunch of unit tests fail mysteriously, and it
>> >> > started Friday..and they've been giving local filesystem permissions
>> >> > errors. Seems like it is related to that. I was going crazy.
>> >> >
>> >> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>> >> >
>> >> > > Is that something new? Are you testing trunk? What's the revision
>> >> number?
>> >> > > Could that relate to 205 upgrade?
>> >> > > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
>> >> > >
>> >> > >
>> >> > > Daniel
>> >> > >
>> >> > > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <[EMAIL PROTECTED]>
>> >> > wrote:
>> >> > >
>> >> > > > Is anyone else seeing a bunch of errors when trying to run the
>> pig
>> >> or
>> >> > > > piggybank unit tests? I had to change ivy to use xercesImpl 2.9.1
>> >> > instead
>> >> > > > of xerces 2.4.4 to make this exception go away:
>> >> > > >
>> >> > > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
>> >> > > > setXIncludeAware(true) for parser
>> >> > > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
>> >> > > > :java.lang.UnsupportedOperationException:
>> >> > > > This parser does not support specification "null" version "null"
>> >> > > > java.lang.UnsupportedOperationException: This parser does not
>> >> support
>> >> > > > specification "null" version "null"
>> >> > > >        at
>> >> > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
>> >> > > >        at
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1117)
>> >> > > >        at
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1103)
>> >> > > >        at
>> >> > > >
>> >> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1037)
>> >> > > >        at
>> >> > > >
>> >> org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1079)
>> >> > > >        at
>> >> > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.recomputeProperties(HExecutionEngine.java:366)
>> >> > > >        at
>> >> > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:177)
>> >> > > >        at
>> >> > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:119)
>> >> > > >        at
>> >> org.apache.pig.impl.PigContext.connect(PigContext.java:206)
>> >> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:246)
>> >> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:231)
>> >> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:227)
>> >> > > >        at org.apache.pig.PigServer.<init>(PigServer.java:223)

Hi, Jonathan,
What is the file permissions error you saw? Your previous log only shows
WrongFS exception.

Daniel

On Mon, Jan 9, 2012 at 8:01 PM, Jonathan Coveney <[EMAIL PROTECTED]> wrote:

> I'm not getting the file permissions error anymore. I tried TestOrderBy
> (which still has some failing tests, but not related to this) and TestUDF.
>
> 2012/1/9 Jonathan Coveney <[EMAIL PROTECTED]>
>
> > Running now, will let you know.
> >
> >
> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >
> >> I just rolled back 205 changes. Can you try if tests pass?
> >>
> >> Thanks,
> >> Daniel
> >>
> >> On Mon, Jan 9, 2012 at 6:57 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> >> wrote:
> >>
> >> > In my /etc/hosts, oauth.localhost.twitter.com is redirected to
> >> > 127.0.0.1...but maybe the presence of this mapping makes it think that
> >> the
> >> > cluster is remote, and not local? I should note that I've always had
> >> this
> >> > set in my /etc/hosts, but only Friday started having issues.
> >> >
> >> > I removed just that line, and it chose a random other one. I removed
> >> all of
> >> > the mappings to 127.0.0.1 except localhost, and here is the error I
> got:
> >> >
> >> > org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
> Wrong
> >> FS:
> >> > hdfs://localhost:60145/user/jcoveney, expected: file:///
> >> >
> >> > Just to see, I got rid of all of my mappings in /etc/hosts (though
> >> mapping
> >> > localhost to 127.0.0.1 is totally valid), and got this error:
> >> >
> >> > org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
> Wrong
> >> FS:
> >> > hdfs://127.0.0.1:60420/user/jcoveney, expected: file:///
> >> >
> >> > So It's not related to that. For whatever reason, it's treating it
> like
> >> an
> >> > hdfs file system instead of just going for file:///
> >> >
> >> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >> >
> >> > > Thanks Jonathan,
> >> > > I saw this in the log:
> >> > > Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
> >> > > expected:
> >> > > file:///
> >> > >
> >> > > Where does this host come from? Is there a hadoop config file in
> your
> >> > > CLASSPATH?
> >> > >
> >> > > Daniel
> >> > >
> >> > > On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <
> [EMAIL PROTECTED]>
> >> > > wrote:
> >> > >
> >> > > > Attached is the log for ant -Dtestcase=TestOrderBy test on a clean
> >> > clone
> >> > > > of trunk.
> >> > > >
> >> > > > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >> > > >
> >> > > >> Hi, Jonathan, can you paste your error message?
> >> > > >>
> >> > > >> Daniel
> >> > > >>
> >> > > >> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <
> >> [EMAIL PROTECTED]>
> >> > > >> wrote:
> >> > > >>
> >> > > >> > Aha! I've been having a bunch of unit tests fail mysteriously,
> >> and
> >> > it
> >> > > >> > started Friday..and they've been giving local filesystem
> >> permissions
> >> > > >> > errors. Seems like it is related to that. I was going crazy.
> >> > > >> >
> >> > > >> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >> > > >> >
> >> > > >> > > Is that something new? Are you testing trunk? What's the
> >> revision
> >> > > >> number?
> >> > > >> > > Could that relate to 205 upgrade?
> >> > > >> > > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
> >> > > >> > >
> >> > > >> > >
> >> > > >> > > Daniel
> >> > > >> > >
> >> > > >> > > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <
> >> [EMAIL PROTECTED]
> >> > >
> >> > > >> > wrote:
> >> > > >> > >
> >> > > >> > > > Is anyone else seeing a bunch of errors when trying to run
> >> the
> >> > pig
> >> > > >> or
> >> > > >> > > > piggybank unit tests? I had to change ivy to use xercesImpl
> >> > 2.9.1
> >> > > >> > instead
> >> > > >> > > > of xerces 2.4.4 to make this exception go away:
> >> > > >> > > >
> >> > > >> > > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
> >> > > >> > > > setXIncludeAware(true) for parser
> >> > > >> > > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364

Err, I mistyped. I did see the file permission error mentioned in the Jura ticket, but setting umask 0022 fixed that and led to the wrongfs exception

Sent from my iPhone

On Jan 9, 2012, at 9:42 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> Hi, Jonathan,
> What is the file permissions error you saw? Your previous log only shows
> WrongFS exception.
>
> Daniel
>
> On Mon, Jan 9, 2012 at 8:01 PM, Jonathan Coveney <[EMAIL PROTECTED]> wrote:
>
>> I'm not getting the file permissions error anymore. I tried TestOrderBy
>> (which still has some failing tests, but not related to this) and TestUDF.
>>
>> 2012/1/9 Jonathan Coveney <[EMAIL PROTECTED]>
>>
>>> Running now, will let you know.
>>>
>>>
>>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>>>
>>>> I just rolled back 205 changes. Can you try if tests pass?
>>>>
>>>> Thanks,
>>>> Daniel
>>>>
>>>> On Mon, Jan 9, 2012 at 6:57 PM, Jonathan Coveney <[EMAIL PROTECTED]>
>>>> wrote:
>>>>
>>>>> In my /etc/hosts, oauth.localhost.twitter.com is redirected to
>>>>> 127.0.0.1...but maybe the presence of this mapping makes it think that
>>>> the
>>>>> cluster is remote, and not local? I should note that I've always had
>>>> this
>>>>> set in my /etc/hosts, but only Friday started having issues.
>>>>>
>>>>> I removed just that line, and it chose a random other one. I removed
>>>> all of
>>>>> the mappings to 127.0.0.1 except localhost, and here is the error I
>> got:
>>>>>
>>>>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
>> Wrong
>>>> FS:
>>>>> hdfs://localhost:60145/user/jcoveney, expected: file:///
>>>>>
>>>>> Just to see, I got rid of all of my mappings in /etc/hosts (though
>>>> mapping
>>>>> localhost to 127.0.0.1 is totally valid), and got this error:
>>>>>
>>>>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
>> Wrong
>>>> FS:
>>>>> hdfs://127.0.0.1:60420/user/jcoveney, expected: file:///
>>>>>
>>>>> So It's not related to that. For whatever reason, it's treating it
>> like
>>>> an
>>>>> hdfs file system instead of just going for file:///
>>>>>
>>>>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>>>>>
>>>>>> Thanks Jonathan,
>>>>>> I saw this in the log:
>>>>>> Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
>>>>>> expected:
>>>>>> file:///
>>>>>>
>>>>>> Where does this host come from? Is there a hadoop config file in
>> your
>>>>>> CLASSPATH?
>>>>>>
>>>>>> Daniel
>>>>>>
>>>>>> On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <
>> [EMAIL PROTECTED]>
>>>>>> wrote:
>>>>>>
>>>>>>> Attached is the log for ant -Dtestcase=TestOrderBy test on a clean
>>>>> clone
>>>>>>> of trunk.
>>>>>>>
>>>>>>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>>>>>>>
>>>>>>>> Hi, Jonathan, can you paste your error message?
>>>>>>>>
>>>>>>>> Daniel
>>>>>>>>
>>>>>>>> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <
>>>> [EMAIL PROTECTED]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Aha! I've been having a bunch of unit tests fail mysteriously,
>>>> and
>>>>> it
>>>>>>>>> started Friday..and they've been giving local filesystem
>>>> permissions
>>>>>>>>> errors. Seems like it is related to that. I was going crazy.
>>>>>>>>>
>>>>>>>>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>>>>>>>>>
>>>>>>>>>> Is that something new? Are you testing trunk? What's the
>>>> revision
>>>>>>>> number?
>>>>>>>>>> Could that relate to 205 upgrade?
>>>>>>>>>> (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Daniel
>>>>>>>>>>
>>>>>>>>>> On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <
>>>> [EMAIL PROTECTED]
>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Is anyone else seeing a bunch of errors when trying to run
>>>> the
>>>>> pig
>>>>>>>> or
>>>>>>>>>>> piggybank unit tests? I had to change ivy to use xercesImpl
>>>>> 2.9.1
>>>>>>>>> instead
>>>>>>>>>>> of xerces 2.4.4 to make this exception go away:
>>>>>>>>>>>
>>>>>>>>>>> 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
>>>>>>>>>>> setXIncludeAware(true) for parser

Yes, this is on the trunk.

On Monday, January 9, 2012, Daniel Dai <[EMAIL PROTECTED]> wrote:
> Are you checking out trunk?
>
> On Mon, Jan 9, 2012 at 8:21 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
>
>> FYI, I'm still getting the same errors after a fresh svn checkout and the
>> following:
>>
>> $ ant package
>> $ and compile-test
>> $ cd contrib/piggybank/java
>> $ ant test -Dtestcase=TestAvroStorage
>>
>> Logs are attached. The same errors appear for other tests in core Pig as
>> well.
>>
>>
>> On Mon, Jan 9, 2012 at 6:01 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:
>>
>>> Thanks Jonathan,
>>> I saw this in the log:
>>> Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
>>> expected:
>>> file:///
>>>
>>> Where does this host come from? Is there a hadoop config file in your
>>> CLASSPATH?
>>>
>>> Daniel
>>>
>>> On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>> > Attached is the log for ant -Dtestcase=TestOrderBy test on a clean
clone
>>> > of trunk.
>>> >
>>> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>>> >
>>> >> Hi, Jonathan, can you paste your error message?
>>> >>
>>> >> Daniel
>>> >>
>>> >> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <[EMAIL PROTECTED]>
>>> >> wrote:
>>> >>
>>> >> > Aha! I've been having a bunch of unit tests fail mysteriously, and
it
>>> >> > started Friday..and they've been giving local filesystem
permissions
>>> >> > errors. Seems like it is related to that. I was going crazy.
>>> >> >
>>> >> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>>> >> >
>>> >> > > Is that something new? Are you testing trunk? What's the revision
>>> >> number?
>>> >> > > Could that relate to 205 upgrade?
>>> >> > > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
>>> >> > >
>>> >> > >
>>> >> > > Daniel
>>> >> > >
>>> >> > > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <[EMAIL PROTECTED]
>
>>> >> > wrote:
>>> >> > >
>>> >> > > > Is anyone else seeing a bunch of errors when trying to run the
>>> pig
>>> >> or
>>> >> > > > piggybank unit tests? I had to change ivy to use xercesImpl
2.9.1
>>> >> > instead
>>> >> > > > of xerces 2.4.4 to make this exception go away:
>>> >> > > >
>>> >> > > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
>>> >> > > > setXIncludeAware(true) for parser
>>> >> > > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
>>> >> > > > :java.lang.UnsupportedOperationException:
>>> >> > > > This parser does not support specification "null" version
"null"
>>> >> > > > java.lang.UnsupportedOperationException: This parser does not
>>> >> support
>>> >> > > > specification "null" version "null"
>>> >> > > >        at
>>> >> > > >
>>> >> > > >
>>> >> > >
>>> >> >
>>> >>
>>>
javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
>>> >> > > >        at
>>> >> > > >
>>> >> > >
>>> >> >
>>>

Hi, Bill,
Is that something new? Or you never run tests on that machine before?

Daniel

On Tue, Jan 10, 2012 at 8:55 AM, Bill Graham <[EMAIL PROTECTED]> wrote:

> Yes, this is on the trunk.
>
> On Monday, January 9, 2012, Daniel Dai <[EMAIL PROTECTED]> wrote:
> > Are you checking out trunk?
> >
> > On Mon, Jan 9, 2012 at 8:21 PM, Bill Graham <[EMAIL PROTECTED]>
> wrote:
> >
> >> FYI, I'm still getting the same errors after a fresh svn checkout and
> the
> >> following:
> >>
> >> $ ant package
> >> $ and compile-test
> >> $ cd contrib/piggybank/java
> >> $ ant test -Dtestcase=TestAvroStorage
> >>
> >> Logs are attached. The same errors appear for other tests in core Pig as
> >> well.
> >>
> >>
> >> On Mon, Jan 9, 2012 at 6:01 PM, Daniel Dai <[EMAIL PROTECTED]>
> wrote:
> >>
> >>> Thanks Jonathan,
> >>> I saw this in the log:
> >>> Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
> >>> expected:
> >>> file:///
> >>>
> >>> Where does this host come from? Is there a hadoop config file in your
> >>> CLASSPATH?
> >>>
> >>> Daniel
> >>>
> >>> On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> >>> wrote:
> >>>
> >>> > Attached is the log for ant -Dtestcase=TestOrderBy test on a clean
> clone
> >>> > of trunk.
> >>> >
> >>> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >>> >
> >>> >> Hi, Jonathan, can you paste your error message?
> >>> >>
> >>> >> Daniel
> >>> >>
> >>> >> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <
> [EMAIL PROTECTED]>
> >>> >> wrote:
> >>> >>
> >>> >> > Aha! I've been having a bunch of unit tests fail mysteriously, and
> it
> >>> >> > started Friday..and they've been giving local filesystem
> permissions
> >>> >> > errors. Seems like it is related to that. I was going crazy.
> >>> >> >
> >>> >> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >>> >> >
> >>> >> > > Is that something new? Are you testing trunk? What's the
> revision
> >>> >> number?
> >>> >> > > Could that relate to 205 upgrade?
> >>> >> > > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
> >>> >> > >
> >>> >> > >
> >>> >> > > Daniel
> >>> >> > >
> >>> >> > > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <
> [EMAIL PROTECTED]
> >
> >>> >> > wrote:
> >>> >> > >
> >>> >> > > > Is anyone else seeing a bunch of errors when trying to run the
> >>> pig
> >>> >> or
> >>> >> > > > piggybank unit tests? I had to change ivy to use xercesImpl
> 2.9.1
> >>> >> > instead
> >>> >> > > > of xerces 2.4.4 to make this exception go away:
> >>> >> > > >
> >>> >> > > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
> >>> >> > > > setXIncludeAware(true) for parser
> >>> >> > > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
> >>> >> > > > :java.lang.UnsupportedOperationException:
> >>> >> > > > This parser does not support specification "null" version
> "null"
> >>> >> > > > java.lang.UnsupportedOperationException: This parser does not
> >>> >> support
> >>> >> > > > specification "null" version "null"
> >>> >> > > >        at
> >>> >> > > >
> >>> >> > > >
> >>> >> > >
> >>> >> >
> >>> >>
> >>>
>
> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
> >>> >> > > >        at
> >>> >> > > >
> >>> >> > >
> >>> >> >
> >>>
>

Hi Daniel,

I don't think I've ever run the tests on this machine. Let me try it on my
home machine tonight and report back. A coworker wasn't able to reproduce
so this could be something in my environment.

thanks,
Bill

On Tue, Jan 10, 2012 at 12:12 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> Hi, Bill,
> Is that something new? Or you never run tests on that machine before?
>
> Daniel
>
>
> On Tue, Jan 10, 2012 at 8:55 AM, Bill Graham <[EMAIL PROTECTED]> wrote:
>
>> Yes, this is on the trunk.
>>
>> On Monday, January 9, 2012, Daniel Dai <[EMAIL PROTECTED]> wrote:
>> > Are you checking out trunk?
>> >
>> > On Mon, Jan 9, 2012 at 8:21 PM, Bill Graham <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> FYI, I'm still getting the same errors after a fresh svn checkout and
>> the
>> >> following:
>> >>
>> >> $ ant package
>> >> $ and compile-test
>> >> $ cd contrib/piggybank/java
>> >> $ ant test -Dtestcase=TestAvroStorage
>> >>
>> >> Logs are attached. The same errors appear for other tests in core Pig
>> as
>> >> well.
>> >>
>> >>
>> >> On Mon, Jan 9, 2012 at 6:01 PM, Daniel Dai <[EMAIL PROTECTED]>
>> wrote:
>> >>
>> >>> Thanks Jonathan,
>> >>> I saw this in the log:
>> >>> Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
>> >>> expected:
>> >>> file:///
>> >>>
>> >>> Where does this host come from? Is there a hadoop config file in your
>> >>> CLASSPATH?
>> >>>
>> >>> Daniel
>> >>>
>> >>> On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <[EMAIL PROTECTED]>
>> >>> wrote:
>> >>>
>> >>> > Attached is the log for ant -Dtestcase=TestOrderBy test on a clean
>> clone
>> >>> > of trunk.
>> >>> >
>> >>> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>> >>> >
>> >>> >> Hi, Jonathan, can you paste your error message?
>> >>> >>
>> >>> >> Daniel
>> >>> >>
>> >>> >> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <
>> [EMAIL PROTECTED]>
>> >>> >> wrote:
>> >>> >>
>> >>> >> > Aha! I've been having a bunch of unit tests fail mysteriously,
>> and
>> it
>> >>> >> > started Friday..and they've been giving local filesystem
>> permissions
>> >>> >> > errors. Seems like it is related to that. I was going crazy.
>> >>> >> >
>> >>> >> > 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
>> >>> >> >
>> >>> >> > > Is that something new? Are you testing trunk? What's the
>> revision
>> >>> >> number?
>> >>> >> > > Could that relate to 205 upgrade?
>> >>> >> > > (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
>> >>> >> > >
>> >>> >> > >
>> >>> >> > > Daniel
>> >>> >> > >
>> >>> >> > > On Mon, Jan 9, 2012 at 3:54 PM, Bill Graham <
>> [EMAIL PROTECTED]
>> >
>> >>> >> > wrote:
>> >>> >> > >
>> >>> >> > > > Is anyone else seeing a bunch of errors when trying to run
>> the
>> >>> pig
>> >>> >> or
>> >>> >> > > > piggybank unit tests? I had to change ivy to use xercesImpl
>> 2.9.1
>> >>> >> > instead
>> >>> >> > > > of xerces 2.4.4 to make this exception go away:
>> >>> >> > > >
>> >>> >> > > > 12/01/09 15:47:52 ERROR conf.Configuration: Failed to set
>> >>> >> > > > setXIncludeAware(true) for parser
>> >>> >> > > > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@358b3364
>> >>> >> > > > :java.lang.UnsupportedOperationException:
>> >>> >> > > > This parser does not support specification "null" version
>> "null"
>> >>> >> > > > java.lang.UnsupportedOperationException: This parser does not
>> >>> >> support
>> >>> >> > > > specification "null" version "null"
>> >>> >> > > >        at
>> >>> >> > > >
>> >>> >> > > >
>> >>> >> > >
>> >>> >> >
>> >>> >>
>> >>>
>>
>> javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)
>> >>> >> > > >        at
>> >>> >> > > >
>> >>> >> > >
>> >>> >> >
>> >>>
>>
>
>

Hi, Jonathan,
TestOrderBy is an excluded test. It does not seems to be written in a right
way. It runs against MiniCluster, but it uses local file as input. This is
not supported even in current code. All regular tests runs fine for me.

Daniel

On Mon, Jan 9, 2012 at 9:54 PM, Jon Coveney <[EMAIL PROTECTED]> wrote:

> Err, I mistyped. I did see the file permission error mentioned in the Jura
> ticket, but setting umask 0022 fixed that and led to the wrongfs exception
>
> Sent from my iPhone
>
> On Jan 9, 2012, at 9:42 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:
>
> > Hi, Jonathan,
> > What is the file permissions error you saw? Your previous log only shows
> > WrongFS exception.
> >
> > Daniel
> >
> > On Mon, Jan 9, 2012 at 8:01 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> wrote:
> >
> >> I'm not getting the file permissions error anymore. I tried TestOrderBy
> >> (which still has some failing tests, but not related to this) and
> TestUDF.
> >>
> >> 2012/1/9 Jonathan Coveney <[EMAIL PROTECTED]>
> >>
> >>> Running now, will let you know.
> >>>
> >>>
> >>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >>>
> >>>> I just rolled back 205 changes. Can you try if tests pass?
> >>>>
> >>>> Thanks,
> >>>> Daniel
> >>>>
> >>>> On Mon, Jan 9, 2012 at 6:57 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> >>>> wrote:
> >>>>
> >>>>> In my /etc/hosts, oauth.localhost.twitter.com is redirected to
> >>>>> 127.0.0.1...but maybe the presence of this mapping makes it think
> that
> >>>> the
> >>>>> cluster is remote, and not local? I should note that I've always had
> >>>> this
> >>>>> set in my /etc/hosts, but only Friday started having issues.
> >>>>>
> >>>>> I removed just that line, and it chose a random other one. I removed
> >>>> all of
> >>>>> the mappings to 127.0.0.1 except localhost, and here is the error I
> >> got:
> >>>>>
> >>>>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
> >> Wrong
> >>>> FS:
> >>>>> hdfs://localhost:60145/user/jcoveney, expected: file:///
> >>>>>
> >>>>> Just to see, I got rid of all of my mappings in /etc/hosts (though
> >>>> mapping
> >>>>> localhost to 127.0.0.1 is totally valid), and got this error:
> >>>>>
> >>>>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
> >> Wrong
> >>>> FS:
> >>>>> hdfs://127.0.0.1:60420/user/jcoveney, expected: file:///
> >>>>>
> >>>>> So It's not related to that. For whatever reason, it's treating it
> >> like
> >>>> an
> >>>>> hdfs file system instead of just going for file:///
> >>>>>
> >>>>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >>>>>
> >>>>>> Thanks Jonathan,
> >>>>>> I saw this in the log:
> >>>>>> Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
> >>>>>> expected:
> >>>>>> file:///
> >>>>>>
> >>>>>> Where does this host come from? Is there a hadoop config file in
> >> your
> >>>>>> CLASSPATH?
> >>>>>>
> >>>>>> Daniel
> >>>>>>
> >>>>>> On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <
> >> [EMAIL PROTECTED]>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Attached is the log for ant -Dtestcase=TestOrderBy test on a clean
> >>>>> clone
> >>>>>>> of trunk.
> >>>>>>>
> >>>>>>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >>>>>>>
> >>>>>>>> Hi, Jonathan, can you paste your error message?
> >>>>>>>>
> >>>>>>>> Daniel
> >>>>>>>>
> >>>>>>>> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <
> >>>> [EMAIL PROTECTED]>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Aha! I've been having a bunch of unit tests fail mysteriously,
> >>>> and
> >>>>> it
> >>>>>>>>> started Friday..and they've been giving local filesystem
> >>>> permissions
> >>>>>>>>> errors. Seems like it is related to that. I was going crazy.
> >>>>>>>>>
> >>>>>>>>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> >>>>>>>>>
> >>>>>>>>>> Is that something new? Are you testing trunk? What's the
> >>>> revision
> >>>>>>>> number?
> >>>>>>>>>> Could that relate to 205 upgrade?
> >>>>>>>>>> (PIG-2431<https://issues.apache.org/jira/browse/PIG-2431>).
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>

Ack, I had wondered if it was something like that :) Is there a place where
that is documented? (I suppose it's just not included in the ant test or
ant test-commit suite?)

Thanks for the update on that. You just unwound a lot of consternation!

2012/1/11 Daniel Dai <[EMAIL PROTECTED]>

> Hi, Jonathan,
> TestOrderBy is an excluded test. It does not seems to be written in a right
> way. It runs against MiniCluster, but it uses local file as input. This is
> not supported even in current code. All regular tests runs fine for me.
>
> Daniel
>
> On Mon, Jan 9, 2012 at 9:54 PM, Jon Coveney <[EMAIL PROTECTED]> wrote:
>
> > Err, I mistyped. I did see the file permission error mentioned in the
> Jura
> > ticket, but setting umask 0022 fixed that and led to the wrongfs
> exception
> >
> > Sent from my iPhone
> >
> > On Jan 9, 2012, at 9:42 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:
> >
> > > Hi, Jonathan,
> > > What is the file permissions error you saw? Your previous log only
> shows
> > > WrongFS exception.
> > >
> > > Daniel
> > >
> > > On Mon, Jan 9, 2012 at 8:01 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> > wrote:
> > >
> > >> I'm not getting the file permissions error anymore. I tried
> TestOrderBy
> > >> (which still has some failing tests, but not related to this) and
> > TestUDF.
> > >>
> > >> 2012/1/9 Jonathan Coveney <[EMAIL PROTECTED]>
> > >>
> > >>> Running now, will let you know.
> > >>>
> > >>>
> > >>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> > >>>
> > >>>> I just rolled back 205 changes. Can you try if tests pass?
> > >>>>
> > >>>> Thanks,
> > >>>> Daniel
> > >>>>
> > >>>> On Mon, Jan 9, 2012 at 6:57 PM, Jonathan Coveney <
> [EMAIL PROTECTED]>
> > >>>> wrote:
> > >>>>
> > >>>>> In my /etc/hosts, oauth.localhost.twitter.com is redirected to
> > >>>>> 127.0.0.1...but maybe the presence of this mapping makes it think
> > that
> > >>>> the
> > >>>>> cluster is remote, and not local? I should note that I've always
> had
> > >>>> this
> > >>>>> set in my /etc/hosts, but only Friday started having issues.
> > >>>>>
> > >>>>> I removed just that line, and it chose a random other one. I
> removed
> > >>>> all of
> > >>>>> the mappings to 127.0.0.1 except localhost, and here is the error I
> > >> got:
> > >>>>>
> > >>>>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
> > >> Wrong
> > >>>> FS:
> > >>>>> hdfs://localhost:60145/user/jcoveney, expected: file:///
> > >>>>>
> > >>>>> Just to see, I got rid of all of my mappings in /etc/hosts (though
> > >>>> mapping
> > >>>>> localhost to 127.0.0.1 is totally valid), and got this error:
> > >>>>>
> > >>>>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
> > >> Wrong
> > >>>> FS:
> > >>>>> hdfs://127.0.0.1:60420/user/jcoveney, expected: file:///
> > >>>>>
> > >>>>> So It's not related to that. For whatever reason, it's treating it
> > >> like
> > >>>> an
> > >>>>> hdfs file system instead of just going for file:///
> > >>>>>
> > >>>>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> > >>>>>
> > >>>>>> Thanks Jonathan,
> > >>>>>> I saw this in the log:
> > >>>>>> Wrong FS: hdfs://oauth.localhost.twitter.com:55237/user/jcoveney,
> > >>>>>> expected:
> > >>>>>> file:///
> > >>>>>>
> > >>>>>> Where does this host come from? Is there a hadoop config file in
> > >> your
> > >>>>>> CLASSPATH?
> > >>>>>>
> > >>>>>> Daniel
> > >>>>>>
> > >>>>>> On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <
> > >> [EMAIL PROTECTED]>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Attached is the log for ant -Dtestcase=TestOrderBy test on a
> clean
> > >>>>> clone
> > >>>>>>> of trunk.
> > >>>>>>>
> > >>>>>>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> > >>>>>>>
> > >>>>>>>> Hi, Jonathan, can you paste your error message?
> > >>>>>>>>
> > >>>>>>>> Daniel
> > >>>>>>>>
> > >>>>>>>> On Mon, Jan 9, 2012 at 4:47 PM, Jonathan Coveney <
> > >>>> [EMAIL PROTECTED]>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Aha! I've been having a bunch of unit tests fail mysteriously,

It is in excluded-tests file. I don't recall why we exclude these tests. It
might related to the deprecation of order UDF.

Now seems nothing block PIG-2431 :)

Thanks,
Daniel

On Wed, Jan 11, 2012 at 8:55 PM, Jonathan Coveney <[EMAIL PROTECTED]>wrote:

> Ack, I had wondered if it was something like that :) Is there a place where
> that is documented? (I suppose it's just not included in the ant test or
> ant test-commit suite?)
>
> Thanks for the update on that. You just unwound a lot of consternation!
>
> 2012/1/11 Daniel Dai <[EMAIL PROTECTED]>
>
> > Hi, Jonathan,
> > TestOrderBy is an excluded test. It does not seems to be written in a
> right
> > way. It runs against MiniCluster, but it uses local file as input. This
> is
> > not supported even in current code. All regular tests runs fine for me.
> >
> > Daniel
> >
> > On Mon, Jan 9, 2012 at 9:54 PM, Jon Coveney <[EMAIL PROTECTED]> wrote:
> >
> > > Err, I mistyped. I did see the file permission error mentioned in the
> > Jura
> > > ticket, but setting umask 0022 fixed that and led to the wrongfs
> > exception
> > >
> > > Sent from my iPhone
> > >
> > > On Jan 9, 2012, at 9:42 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:
> > >
> > > > Hi, Jonathan,
> > > > What is the file permissions error you saw? Your previous log only
> > shows
> > > > WrongFS exception.
> > > >
> > > > Daniel
> > > >
> > > > On Mon, Jan 9, 2012 at 8:01 PM, Jonathan Coveney <[EMAIL PROTECTED]
> >
> > > wrote:
> > > >
> > > >> I'm not getting the file permissions error anymore. I tried
> > TestOrderBy
> > > >> (which still has some failing tests, but not related to this) and
> > > TestUDF.
> > > >>
> > > >> 2012/1/9 Jonathan Coveney <[EMAIL PROTECTED]>
> > > >>
> > > >>> Running now, will let you know.
> > > >>>
> > > >>>
> > > >>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> > > >>>
> > > >>>> I just rolled back 205 changes. Can you try if tests pass?
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Daniel
> > > >>>>
> > > >>>> On Mon, Jan 9, 2012 at 6:57 PM, Jonathan Coveney <
> > [EMAIL PROTECTED]>
> > > >>>> wrote:
> > > >>>>
> > > >>>>> In my /etc/hosts, oauth.localhost.twitter.com is redirected to
> > > >>>>> 127.0.0.1...but maybe the presence of this mapping makes it think
> > > that
> > > >>>> the
> > > >>>>> cluster is remote, and not local? I should note that I've always
> > had
> > > >>>> this
> > > >>>>> set in my /etc/hosts, but only Friday started having issues.
> > > >>>>>
> > > >>>>> I removed just that line, and it chose a random other one. I
> > removed
> > > >>>> all of
> > > >>>>> the mappings to 127.0.0.1 except localhost, and here is the
> error I
> > > >> got:
> > > >>>>>
> > > >>>>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
> > > >> Wrong
> > > >>>> FS:
> > > >>>>> hdfs://localhost:60145/user/jcoveney, expected: file:///
> > > >>>>>
> > > >>>>> Just to see, I got rid of all of my mappings in /etc/hosts
> (though
> > > >>>> mapping
> > > >>>>> localhost to 127.0.0.1 is totally valid), and got this error:
> > > >>>>>
> > > >>>>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118:
> > > >> Wrong
> > > >>>> FS:
> > > >>>>> hdfs://127.0.0.1:60420/user/jcoveney, expected: file:///
> > > >>>>>
> > > >>>>> So It's not related to that. For whatever reason, it's treating
> it
> > > >> like
> > > >>>> an
> > > >>>>> hdfs file system instead of just going for file:///
> > > >>>>>
> > > >>>>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> > > >>>>>
> > > >>>>>> Thanks Jonathan,
> > > >>>>>> I saw this in the log:
> > > >>>>>> Wrong FS: hdfs://
> oauth.localhost.twitter.com:55237/user/jcoveney,
> > > >>>>>> expected:
> > > >>>>>> file:///
> > > >>>>>>
> > > >>>>>> Where does this host come from? Is there a hadoop config file in
> > > >> your
> > > >>>>>> CLASSPATH?
> > > >>>>>>
> > > >>>>>> Daniel
> > > >>>>>>
> > > >>>>>> On Mon, Jan 9, 2012 at 5:46 PM, Jonathan Coveney <
> > > >> [EMAIL PROTECTED]>
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> Attached is the log for ant -Dtestcase=TestOrderBy test on a

Expect tests tomorrow :D

2012/1/11 Daniel Dai <[EMAIL PROTECTED]>

> It is in excluded-tests file. I don't recall why we exclude these tests. It
> might related to the deprecation of order UDF.
>
> Now seems nothing block PIG-2431 :)
>
> Thanks,
> Daniel
>
> On Wed, Jan 11, 2012 at 8:55 PM, Jonathan Coveney <[EMAIL PROTECTED]
> >wrote:
>
> > Ack, I had wondered if it was something like that :) Is there a place
> where
> > that is documented? (I suppose it's just not included in the ant test or
> > ant test-commit suite?)
> >
> > Thanks for the update on that. You just unwound a lot of consternation!
> >
> > 2012/1/11 Daniel Dai <[EMAIL PROTECTED]>
> >
> > > Hi, Jonathan,
> > > TestOrderBy is an excluded test. It does not seems to be written in a
> > right
> > > way. It runs against MiniCluster, but it uses local file as input. This
> > is
> > > not supported even in current code. All regular tests runs fine for me.
> > >
> > > Daniel
> > >
> > > On Mon, Jan 9, 2012 at 9:54 PM, Jon Coveney <[EMAIL PROTECTED]>
> wrote:
> > >
> > > > Err, I mistyped. I did see the file permission error mentioned in the
> > > Jura
> > > > ticket, but setting umask 0022 fixed that and led to the wrongfs
> > > exception
> > > >
> > > > Sent from my iPhone
> > > >
> > > > On Jan 9, 2012, at 9:42 PM, Daniel Dai <[EMAIL PROTECTED]>
> wrote:
> > > >
> > > > > Hi, Jonathan,
> > > > > What is the file permissions error you saw? Your previous log only
> > > shows
> > > > > WrongFS exception.
> > > > >
> > > > > Daniel
> > > > >
> > > > > On Mon, Jan 9, 2012 at 8:01 PM, Jonathan Coveney <
> [EMAIL PROTECTED]
> > >
> > > > wrote:
> > > > >
> > > > >> I'm not getting the file permissions error anymore. I tried
> > > TestOrderBy
> > > > >> (which still has some failing tests, but not related to this) and
> > > > TestUDF.
> > > > >>
> > > > >> 2012/1/9 Jonathan Coveney <[EMAIL PROTECTED]>
> > > > >>
> > > > >>> Running now, will let you know.
> > > > >>>
> > > > >>>
> > > > >>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> > > > >>>
> > > > >>>> I just rolled back 205 changes. Can you try if tests pass?
> > > > >>>>
> > > > >>>> Thanks,
> > > > >>>> Daniel
> > > > >>>>
> > > > >>>> On Mon, Jan 9, 2012 at 6:57 PM, Jonathan Coveney <
> > > [EMAIL PROTECTED]>
> > > > >>>> wrote:
> > > > >>>>
> > > > >>>>> In my /etc/hosts, oauth.localhost.twitter.com is redirected to
> > > > >>>>> 127.0.0.1...but maybe the presence of this mapping makes it
> think
> > > > that
> > > > >>>> the
> > > > >>>>> cluster is remote, and not local? I should note that I've
> always
> > > had
> > > > >>>> this
> > > > >>>>> set in my /etc/hosts, but only Friday started having issues.
> > > > >>>>>
> > > > >>>>> I removed just that line, and it chose a random other one. I
> > > removed
> > > > >>>> all of
> > > > >>>>> the mappings to 127.0.0.1 except localhost, and here is the
> > error I
> > > > >> got:
> > > > >>>>>
> > > > >>>>> org.apache.pig.backend.executionengine.ExecException: ERROR
> 2118:
> > > > >> Wrong
> > > > >>>> FS:
> > > > >>>>> hdfs://localhost:60145/user/jcoveney, expected: file:///
> > > > >>>>>
> > > > >>>>> Just to see, I got rid of all of my mappings in /etc/hosts
> > (though
> > > > >>>> mapping
> > > > >>>>> localhost to 127.0.0.1 is totally valid), and got this error:
> > > > >>>>>
> > > > >>>>> org.apache.pig.backend.executionengine.ExecException: ERROR
> 2118:
> > > > >> Wrong
> > > > >>>> FS:
> > > > >>>>> hdfs://127.0.0.1:60420/user/jcoveney, expected: file:///
> > > > >>>>>
> > > > >>>>> So It's not related to that. For whatever reason, it's treating
> > it
> > > > >> like
> > > > >>>> an
> > > > >>>>> hdfs file system instead of just going for file:///
> > > > >>>>>
> > > > >>>>> 2012/1/9 Daniel Dai <[EMAIL PROTECTED]>
> > > > >>>>>
> > > > >>>>>> Thanks Jonathan,
> > > > >>>>>> I saw this in the log:
> > > > >>>>>> Wrong FS: hdfs://
> > oauth.localhost.twitter.com:55237/user/jcoveney,
> > > > >>>>>> expected:
> > > > >>>>>> file:///
> > > > >>>>>>