Hi Roman and Stack,
> let us know if your system tests can be deployed to Maven with every
nightly build of HBase as a standalone artifact that Bigtop's iTest can
simply depend on
I believe we have an "hbase-it" artifact containing integration tests that
pulls in all of the other HBase artifacts.
> let us know what's your sweet spot for the # of nodes/RS
One of our large tests -- TestReplication -- is going to be moved out into
an integration test, and this will want to configure two independent
clusters with client access to both. Assume two 3 slave clusters, that's a
total of 8. Otherwise for single cluster tests 5 slaves seems a good
starting point IMO.
> what are you expectations for tests that do manipulate the state of the
cluster (like ChaosMonkey) -- do you expect unrestricted ssh, etc?
I think we want, at least, unrestricted access on the service accounts for
hdfs, hbase, zookeeper, so we can kill -9 or kill -STOP processes at
different system layers.
> All in all, I think this makes perfect sense and if there are folks in
the HBase community willing to help us get this going it'll be a 'beginning
of a beautiful friendship' ;-)
On Fri, Dec 21, 2012 at 11:44 AM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> Hi Stack!
> Let me know at which point this discussion needs to be CCed
> over to dev@hbase, but I'll just reply here for now:
> On Fri, Dec 21, 2012 at 10:58 AM, Stack <[EMAIL PROTECTED]> wrote:
> > The HBase Project has been developing a set of big tests that are too
> > to run as part of the regular hbase project unit tests suite. They are
> > currently run adhoc by individual developers. We are looking for a home
> > for these tests, a location where they could be automatically run with
> > periodicity.
> > Would it be possible to run them on the bigtop test infrastructure?
> I think Bigtop makes a perfect home for these tests. After all, Bigtop
> as a project is exactly about facilitating integration efforts of Hadoop
> and its ecosystem projects.
> > Their general nature is set up a cluster, run a number of pretty
> > steps -- e.g. loading lots of artificial data -- while optionally, a
> > monkey is doing its damndest to break things. The tests can be sized to
> > match cluster size, from mini-all-in-the-one-jvm up to clusters of
> > of nodes. On the end, the test generates a report and logs. A fuller
> > description can be found here .
> > What you all think?
> Bigtop infrastructure can offer the following (provided that there's
> in HBase community to help with keeping an eye on things since Bigtop
> is as short on cycles as any other ASF project):
> * regular builds of Bigtop's trunk and thus a particular point
> of HBase tree that Bigtop's trunk is tracking (on all supported
> Linux platforms)
> * regular builds of HBase's trunk (on all supported Linux platforms)
> * regular test runs on dynamically deployed Hadoop clusters. We
> are currently deploying to ~5 nodes which gives us total HDFS
> capacity of around couple of T.
> * jenkins reporting of those test runs
> Here's what you guys can do in order to make the job of hooking up
> your tests easier:
> * let us know what's your sweet spot for the # of nodes/RS
> * let us know what's your sweet spot for the total storage capacity
> * let us know if your system tests can be deployed to Maven
> with every nightly build of HBase as a standalone artifact that
> Bigtop's iTest can simply depend on
> * provide a bit more details on the desired configs for the tests
> * what are you expectations for tests that do manipulate the
> state of the cluster (like ChaosMonkey) -- do you expect
> unrestricted ssh, etc?
> All in all, I think this makes perfect sense and if there are folks
> in the HBase community willing to help us get this going it'll
> be a 'beginning of a beautiful friendship' ;-)
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)