Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - [DISCUSS] Ensuring Consistent Behavior for Alternative Hadoop FileSystems + Workshop


Copy link to this message
-
Re: [DISCUSS] Ensuring Consistent Behavior for Alternative Hadoop FileSystems + Workshop
Andrew Wang 2013-06-10, 21:14
Thanks for the summary Steve, very useful.

I'm wondering a bit about the point on testing AbstractFileSystem rather
than FileSystem. While these are both wrappers for DFSClient, they're
pretty different in terms of the APIs they expose. Furthermore, AFS is not
actually a client-facing API; clients interact with an AFS through
FileContext.

I ask because I did some work trying to unify the symlink tests for both
FileContext and FileSystem (HADOOP-9370 and HADOOP-9355). Subtle things
like the default mkdir semantics are different; you can see some of the
contortions in HADOOP-9370. I ultimately ended up just adhering to the
FileContext-style behavior, but as a result I'm not really testing some
parts of FileSystem.

Are we going to end up with two different sets of validation tests? Or just
choose one API over the other? FileSystem is supposed to eventually be
deprecated in favor of FileContext (HADOOP-6446, filed in 2009), but actual
uptake in practice has been slow.

Best,
Andrew
On Mon, Jun 10, 2013 at 1:49 PM, Stephen Watt <[EMAIL PROTECTED]> wrote:

> For those interested - I posted a recap of this mornings Google Hangout on
> the Wiki Page at https://wiki.apache.org/hadoop/HCFS/Progress
>
> On Jun 5, 2013, at 8:14 PM, Stephen Watt wrote:
>
> > Hi Folks
> >
> > Per Roman's recommendation I've created a Wiki Page for organizing the
> work and managing the logistics -
> https://wiki.apache.org/hadoop/HCFS/Progress
> >
> > I'd like to propose a Google Hangout at 9am PST on Monday June 10th to
> get together and discuss the initiative. Please respond back to me if
> you're interested or would like to propose a different time. I'll update
> our Wiki page with the logistics.
> >
> > Regards
> > Steve Watt
> >
> > ----- Original Message -----
> > From: "Roman Shaposhnik" <[EMAIL PROTECTED]>
> > To: "Stephen Watt" <[EMAIL PROTECTED]>
> > Cc: [EMAIL PROTECTED], [EMAIL PROTECTED], "shv
> hadoop" <[EMAIL PROTECTED]>, [EMAIL PROTECTED], [EMAIL PROTECTED],
> [EMAIL PROTECTED]
> > Sent: Friday, May 31, 2013 5:28:58 PM
> > Subject: Re: [DISCUSS] Ensuring Consistent Behavior for Alternative
> Hadoop FileSystems + Workshop
> >
> > On Fri, May 31, 2013 at 1:00 PM, Stephen Watt <[EMAIL PROTECTED]> wrote:
> >> What is the protocol for organizing the logistics and collaborating? I
> am loathe to flood common-dev with "does this time work for you?" emails
> from the interested parties. Do we create a high level JIRA ticket and
> collaborate and post comments and G+ meetup times on that ? Another option
> might be the Wiki, I'd be happy to be responsible with tracking progress on
> https://wiki.apache.org/hadoop/HCFS/Progress until we are able to break
> initiatives down into more granular JIRA tickets.
> >
> > I'd go with a wiki page and perhaps http://www.doodle.com/
> >
> >> After we've had a few G+ hangouts, for those that would like to meet
> face to face, I have also made an all day reservation for a meeting room
> that can hold up to 20 people at our Red Hat Office in Castro Street,
> Mountain View on Tuesday June 25th (the day before Hadoop Summit and a
> short drive away). We don't have to use the whole day, but it gives us some
> flexibility around the availability of interested parties. I was thinking
> something along the lines of 10am - 3pm. We are happy to cater lunch.
> >
> > That also would be very much appreciated!
> >
> > Thanks,
> > Roman.
>