-Re: How to support the (HDFS) FileSystem API of various Hadoop Distributions?
Christian Schneider 2013-09-07, 10:44
Thanks for the suggestion. And yes, the .isRoot() lead to this question :).
Also I wasn't sure about the ideal way of packaging this utility.
My "dream" is to have 1 binary that is able to deal with different versions
(IPC 3, 5, 7, ...).
The users will download the binary package and it is compatible with a wide
range of versions.
>From your suggestions I tried to run 0.20.2 against CDH4.3, - but it is not
compatible with IPC V7.
So I think I need to do the same then HBase Compat does. "Somehow" the tool
needs to check
the version of the cluster and then load the correct implementation for
But how to check the IPC version?
P.S.: Thanks, that motivates me to continue :)
2013/9/6 Harsh J <[EMAIL PROTECTED]>
> Oh and btw, nice utility! :)
> On Fri, Sep 6, 2013 at 7:50 AM, Harsh J <[EMAIL PROTECTED]> wrote:
> > Hello,
> > There are a few additions to the FileSystem that may bite you across
> > versions, but if you pick an old stable version such as Apache Hadoop
> > 0.20.2, and stick to only its offered APIs, it would work better
> > across different version dependencies as we try to maintain FileSystem
> > as a stable interface as much as we can (there was also more recent
> > work to ensure the stabilization). I looked over your current code
> > state and it seemed to have pretty stable calls that I think have
> > existed across several versions and exists today, but I did notice you
> > had to remove an isRoot as part of a previous commit, which may have
> > lead to this question?
> > If that doesn't work for you, you can also switch out to using
> > sub-modules carrying code specific to a build version type (such as
> > what HBase does at https://github.com/apache/hbase/tree/trunk/ (see
> > the hbase-hadoop-compat directories)).
> > On Fri, Sep 6, 2013 at 2:59 AM, Christian Schneider
> > <[EMAIL PROTECTED]> wrote:
> >> Hi,
> >> I start to write a small ncdu clone to browse HDFS on the CLI
> >> (http://nchadoop.org/). Currently i'm testing it against CDH4, - but I
> >> to make it available for a wider group of users (Hortonworks, ..).
> >> Is it enough to pick different vanilla Versions (for IPC 5, 7)?
> >> Best Regards,
> >> Christian.
> > --
> > Harsh J
> Harsh J