Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - hbase 0.20.6 - HBaseClusterTestCase, DU, cygwin, IntelliJ - arg!


Copy link to this message
-
RE: hbase 0.20.6 - HBaseClusterTestCase, DU, cygwin, IntelliJ - arg!
Jonathan Gray 2011-01-21, 00:56
Sounds like a bug.  Want to file a JIRA with as much detail as you can including log snippets?

> -----Original Message-----
> From: Mark Laffoon [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, January 20, 2011 2:43 PM
> To: [EMAIL PROTECTED]
> Subject: RE: hbase 0.20.6 - HBaseClusterTestCase, DU, cygwin, IntelliJ - arg!
>
> Follow up on this, hopefully Jonathan and/or Stack are still watching  ;)
>
> I switched to HBaseTestingUtility (we had a class which was doing more-or-
> less the same thing with HBaseClusterTestCase). I dealt with my IDE issues by
> jamming cygwin/bin into my Windows PATH <sigh>. However, we still can't
> run multiple test suites using command-line maven. When the test suite
> finishes running and calls HTU.shutdownMiniCluster, it logs a warning that it
> can't delete the temporary data folder. Looking in the folder, the zookeeper
> subfolder is being left around. This implies that there is a zookeeper process
> that isn't letting go. When the next test suite tries to run, it bumps into port
> issues with the previous process.
> Or something.
>
> Anybody out there doing anything like this?
>
> Is my only solution to run *all* my persist tests in one giant test suite?
> I really hope not.'
>
> Thanks,
> Mark
>
>
> -----Original Message-----
> From: Jonathan Gray [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, January 18, 2011 7:56 PM
> To: [EMAIL PROTECTED]
> Subject: RE: hbase 0.20.6 - HBaseClusterTestCase, DU, cygwin, IntelliJ - arg!
>
> Hey Mark.  Sorry to hear about your troubles.
>
> There is a new testing facility that has replaced HBaseClusterTestCase.
> Check out HBaseTestingUtility.  It's JUnit4 based.  One sample usage of it is
> the test TestFromClientSide.
>
> This new one includes support for multiple DataNodes, RegionServers, and
> Masters.  Hopefully that will work better for you.
>
> As far as working in your IDE, du is part of Cygwin.  So if it's saying not found,
> something in your environment (like your path) is not configured properly for
> Cygwin and is not finding du.
>
> Hope that helps.
>
> JG
>
> > -----Original Message-----
> > From: Mark Laffoon [mailto:[EMAIL PROTECTED]]
> > Sent: Tuesday, January 18, 2011 2:57 PM
> > To: [EMAIL PROTECTED]
> > Subject: hbase 0.20.6 - HBaseClusterTestCase, DU, cygwin, IntelliJ -
> arg!
> >
> > Sorry about the goofy subject but this is driving me nuts. Any answers
> to any
> > of these questions would be greatly appreciated (well, except a
> > generic "yes").
> >
> >
> >
> > My goal is to be able to run a sequence of unit tests (well,
> > integration
> tests
> > really) on our persistence stack. We're using HBaseClusterTestCase.
> > We have about 25 sets of 20-40 tests that we want to run. Each set
> > needs
> to
> > stand up a fresh, independent store. We're using maven to do our builds.
> >
> > 1.      In our build machine environment (Ubuntu, Anthill) and my build
> > environment (Windows, cygwin), it doesn't work because the test cases
> > don't tear down properly, and the test folders stick around, messing
> > up
> the
> > subsequent tests.
> >
> > 2.      In my IDE (IntelliJ on Windows using IDE's internal maven), the
> > tests don't work because MiniDFSCluster eventually calls "du" which
> doesn't
> > exist.
> >
> > 3.      Anybody have sample code for setting up a simulated data node?
> Is
> > there such thing as a simulated persistence for zookeeper? (Our test
> data
> > isn't very large so this would be fine)
> >
> >
> >
> > One final question that may at least help my blood pressure: has hbase
> unit
> > testing improved at all in version 0.90?
> >
> >
> >
> > Thanks,
> >
> > Mark