Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> testHDFSConf.xml


Copy link to this message
-
Re: testHDFSConf.xml
The test cases weren't split  - there's only testHDFS suite. However, the
drivers available for common and mapreduce, so all you need to do is to add
your tests cases a-la hdfs.

Cos

On Thu, Apr 11, 2013 at 09:14AM, Jay Vyas wrote:
> Thats great. Yes this is certainly something we could benefit from.
>
> But the "trunk" github appears to only have
> "./src/test/hdfs/org/apache/hadoop/cli/testHDFSConf.xml" in the source tree.
>
> Also, I don't see it in branch-0.22:
>
> # git checkout branch-0.22
> Branch branch-0.22 set up to track remote branch branch-0.22 from origin.
> Switched to a new branch 'branch-0.22'
> # find ./ -name *Conf.xml
> ./src/test/hdfs/org/apache/hadoop/cli/testHDFSConf.xml
>
>
> Maybe I'm missing something?
>
>
> On Thu, Apr 11, 2013 at 1:28 AM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:
>
> > I have split CLI test infrastructure into hierarchical pieces that allow to
> > have different configurations for different components. E.g. you can have
> > one
> > for YARN that would exist independently of HDFS, etc. The change has been
> > in
> > since like 0.22 and committed to 0.203.x as well IIRC, hence should be
> > usable
> > across the Hadoop versions. May be it something you would benefit from.
> >
> > Cos
> >
> > On Wed, Apr 10, 2013 at 10:43AM, Colin McCabe wrote:
> > > On Wed, Apr 10, 2013 at 10:16 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
> > >
> > > > Hello HDFS brethren !
> > > >
> > > > I've noticed that the testHDFSConf.xml has alot of references to
> > > > supergroup.
> > > >
> > > >
> > > >
> > https://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testHDFSConf.xml
> > > >
> > > > 1) I wonder why this is hardcoded in the testHDFSConf.xml
> > > >
> > > >
> > > "supergroup" is the default supergroup in HDFS.  Check
> > DFSConfigKeys.java:
> > >
> > >   public static final String  DFS_PERMISSIONS_SUPERUSERGROUP_KEY > > > "dfs.permissions.superusergroup";
> > >   public static final String  DFS_PERMISSIONS_SUPERUSERGROUP_DEFAULT > > > "supergroup";
> > >
> > > It seems fine to use "supergroup" in a test.  after all, we do control
> > the
> > > configuration we pass into the test.
> > >
> > >
> > > > 2) Also, Im wondering if there are any good ideas for
> > extending/modifying
> > > > this file for a extention of the FileSystem implementation.
> > > >
> > > >
> > > It would be interesting to think about pulling the non-hdfs-specific
> > > components of TestHDFSCLI into another test; perhaps one in common.
> > >  Theoretically, what we print on the console should be really similar, no
> > > matter whether HDFS or some other filesystem is being used.  In practice,
> > > there may be some differences, however...
> > >
> > > I find it a little bit challenging to modify TestHDFSCLI because the test
> > > is really long and executes as a single unit.  Breaking it down into
> > > multiple units would probably be another good improvement, at least in my
> > > opinion.
> > >
> > > best,
> > > Colin
> > >
> > >
> > > Right  now im doing some global find replace statements - but was
> > thinking
> > > > that maybe parameterizing the file would be a good JIRA - so that
> > people
> > > > could use this as a base test for FileSystem implementations
> > > >
> > > > Depending on feedback im certainly willing to submit and put in a first
> > > > pass at a more modular version of this file.
> > > >
> > > > Its in many ways a very generalizable component of the hdfs trunk.
> > > >
> > > > Thanks!
> > > > --
> > > > Jay Vyas
> > > > http://jayunit100.blogspot.com
> > > >
> >
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com