Steve Loughran 2013-06-03, 14:12
-Re: HADOOP-8545 OpenStack Swift patch ready for commit -reviewers needed
Steve, I will review this. Might need a couple of days though.
On Mon, Jun 3, 2013 at 7:12 AM, Steve Loughran <[EMAIL PROTECTED]>wrote:
> We've got the HADOOP-8545<
> stable enough that I'd like it checked in and in the 2.1 beta so that
> we can see what problems occur in the field.
> The patch adds a new module, hadoop-tools/hadoop-openstack, so it doesn't
> break anything in hadoop-common/hdfs/yarn, and it includes more
> method-by-method tests than any of the non-HDFS filesystem clients have,
> including basic scale tests and the corner-case problems that we're going
> to have to talk about in terms of FS API spec and tests. In github, there
> are some very basic scale tests running Pig jobs against a configurable
> amount of data, showing that we can run pig against GB of data.
> We've run these tests against rackspace US, rackspace US and hpcloud -the
> public Swift services -as well as private deployments of the latest version
> of OpenStack. This makes us confident that we've got authentication right,
> and as we get consistent outcomes from all clusters, we aren't relying some
> cluster-/provider- specific detail.
> So: please can people review it! The site documentation (did I mention
> we've got that?) lists how to connect to a swift service, and how to test
> against them. If anyone does want to run the test themselves, try the docs
> and then escalate to us if there are any problems.
> Steve Loughran, Dmitry Mezhenskiy (Mirantis) & David Dobbins (Rackspace)
Steve Loughran 2013-06-03, 17:51