There was a failure of TestLocalHBaseCluster in
https://builds.apache.org/job/PreCommit-HBASE-Build/3879 that I was able to
reproduce locally, but what I think this means is a couple of tests are
using a data dir of /tmp/hbase-${user}.

I bumped HFileV2.MAX_MINOR_VERSION to 3, then ran tests. Later I changed it
back to 2 after review advice from Ted. TestLocalHBaseCluster then started
failing because some HFiles with a version of 2.3 remained in the
filesystem under /tmp/hbase-${user} from earlier tests and could not be
read in by current tests. The precommit builds don't save the test logs so
I can't confirm if what is going on up on Jenkins is the same thing, but I
suspect so.

I'm not sure if JUnit or Surefire can support a global post test action,
but I'll look into it. I want to test for the existence
of /tmp/hbase-${user} after a unit test has run and if so fail it to flag
it as an offender.
--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

On Mon, Jan 7, 2013 at 10:00 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> There was a failure of TestLocalHBaseCluster in
> https://builds.apache.org/job/PreCommit-HBASE-Build/3879 that I was able
> to
> reproduce locally, but what I think this means is a couple of tests are
> using a data dir of /tmp/hbase-${user}.
>
> I bumped HFileV2.MAX_MINOR_VERSION to 3, then ran tests. Later I changed it
> back to 2 after review advice from Ted. TestLocalHBaseCluster then started
> failing because some HFiles with a version of 2.3 remained in the
> filesystem under /tmp/hbase-${user} from earlier tests and could not be
> read in by current tests. The precommit builds don't save the test logs so
> I can't confirm if what is going on up on Jenkins is the same thing, but I
> suspect so.
>
> I'm not sure if JUnit or Surefire can support a global post test action,
> but I'll look into it. I want to test for the existence
> of /tmp/hbase-${user} after a unit test has run and if so fail it to flag
> it as an offender.
>
See
./hbase-common/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java

Is it hdfs that is putting the files in /tmp?

St.Ack

I've also noticed this before, but though we fixed all instances in 96.
Instead of jenkins cleaning up, we should fix the tests themselves to not
use the /tmp directory. I guess we can use the ResourceChecker to detect
and fail those tests.

Enis
On Mon, Jan 7, 2013 at 11:32 AM, Stack <[EMAIL PROTECTED]> wrote:

> On Mon, Jan 7, 2013 at 10:00 AM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
>
> > There was a failure of TestLocalHBaseCluster in
> > https://builds.apache.org/job/PreCommit-HBASE-Build/3879 that I was able
> > to
> > reproduce locally, but what I think this means is a couple of tests are
> > using a data dir of /tmp/hbase-${user}.
> >
> > I bumped HFileV2.MAX_MINOR_VERSION to 3, then ran tests. Later I changed
> it
> > back to 2 after review advice from Ted. TestLocalHBaseCluster then
> started
> > failing because some HFiles with a version of 2.3 remained in the
> > filesystem under /tmp/hbase-${user} from earlier tests and could not be
> > read in by current tests. The precommit builds don't save the test logs
> so
> > I can't confirm if what is going on up on Jenkins is the same thing, but
> I
> > suspect so.
> >
> > I'm not sure if JUnit or Surefire can support a global post test action,
> > but I'll look into it. I want to test for the existence
> > of /tmp/hbase-${user} after a unit test has run and if so fail it to flag
> > it as an offender.
> >
>
>
> See
> ./hbase-common/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java
>
> Is it hdfs that is putting the files in /tmp?
>
> St.Ack
>

There are tests writing stuff like
/tmp/..-577547068.TestCP3.1357574431574.jar.crc or
/tmp/..-1797055647.TestCPInvalid.1357574424182.jar.crc as well.

Note the '.' to make them invisible :-)
On Mon, Jan 7, 2013 at 8:38 PM, Enis Söztutar <[EMAIL PROTECTED]> wrote:

> should fix the tests themselves to not
> use the /tmp directory. I guess we can use the ResourceChecker to detect
> and fail those tests.
>

For CP's, we have a candidate patch at:
http://issues.apache.org/jira/browse/HBASE-6824. We can use this patch to
migrate the remaining usages from the main code side, and also set this in
HBTU.

Enis

On Mon, Jan 7, 2013 at 11:42 AM, Nicolas Liochon <[EMAIL PROTECTED]> wrote:

> There are tests writing stuff like
> /tmp/..-577547068.TestCP3.1357574431574.jar.crc or
> /tmp/..-1797055647.TestCPInvalid.1357574424182.jar.crc as well.
>
> Note the '.' to make them invisible :-)
>
>
> On Mon, Jan 7, 2013 at 8:38 PM, Enis Söztutar <[EMAIL PROTECTED]> wrote:
>
> > should fix the tests themselves to not
> > use the /tmp directory. I guess we can use the ResourceChecker to detect
> > and fail those tests.
> >
>

Sounds perfect to me.

On Mon, Jan 7, 2013 at 11:38 AM, Enis Söztutar <[EMAIL PROTECTED]> wrote:

> I guess we can use the ResourceChecker to detect and fail those tests.
>
--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Andrew, could you please review HBASE-6824 then. Let's get the ball
rolling.

Enis
On Mon, Jan 7, 2013 at 12:26 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> Sounds perfect to me.
>
> On Mon, Jan 7, 2013 at 11:38 AM, Enis Söztutar <[EMAIL PROTECTED]> wrote:
>
> > I guess we can use the ResourceChecker to detect and fail those tests.
> >
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

No, I'm seeing hbase-${user}
On Mon, Jan 7, 2013 at 11:32 AM, Stack <[EMAIL PROTECTED]> wrote:

> On Mon, Jan 7, 2013 at 10:00 AM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
>
> > There was a failure of TestLocalHBaseCluster in
> > https://builds.apache.org/job/PreCommit-HBASE-Build/3879 that I was able
> > to
> > reproduce locally, but what I think this means is a couple of tests are
> > using a data dir of /tmp/hbase-${user}.
> >
> > I bumped HFileV2.MAX_MINOR_VERSION to 3, then ran tests. Later I changed
> it
> > back to 2 after review advice from Ted. TestLocalHBaseCluster then
> started
> > failing because some HFiles with a version of 2.3 remained in the
> > filesystem under /tmp/hbase-${user} from earlier tests and could not be
> > read in by current tests. The precommit builds don't save the test logs
> so
> > I can't confirm if what is going on up on Jenkins is the same thing, but
> I
> > suspect so.
> >
> > I'm not sure if JUnit or Surefire can support a global post test action,
> > but I'll look into it. I want to test for the existence
> > of /tmp/hbase-${user} after a unit test has run and if so fail it to flag
> > it as an offender.
> >
>
>
> See
> ./hbase-common/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java
>
> Is it hdfs that is putting the files in /tmp?
>
> St.Ack
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)