Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: core-site.xml file is being ignored by new Configuration()


+
Tom Melendez 2013-01-17, 00:45
Copy link to this message
-
Re: core-site.xml file is being ignored by new Configuration()
no difference.

This is really odd.

The only way I can get it to work is by the following:
/**
 * A simple test harness for glusterfs
 * @author jvyas
 */
public class TestMapReducerGlusterFS {

    @Test
    public void test() throws Exception{
        Configuration conf = new Configuration(false);
        System.out.println("before reload:"+conf);
        //necessary?
        conf.set("fs.default.name", ".......");
        conf.set("fs.blah.mount", "/mnt/......");
        conf.set("fs.blahserver", "192.168.1.36");
        conf.set("iomod","Off");
         ...
    }
}

Thinking that this is a little known issue in the current hadoop
parser/configurator? because very rarely do people write unit tests which
utilize abnormal ( i.e. non-local or non-hdfs) file systems.
On Wed, Jan 16, 2013 at 7:45 PM, Tom Melendez <[EMAIL PROTECTED]> wrote:

> Result any different if you call reloadConfiguration()?
>
> Thanks,
>
> Tom
>
> On Wed, Jan 16, 2013 at 4:33 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
> > Hi guys:
> >
> > I've finally extracted my problem of loading a special filesystem
> > into a unit test.
> >
> > Below, clearly, Im creating a raw configuration and adding a single
> resource
> > to it (core-site.xml).
> >
> > Afterwards, i print to confirm that the file exists.
> >
> > Finally, I try to create a FileSystem from that file.  However, no file
> > system is created :(.
> >
> > Any thoughts on why this core-site.xml file is being ignoreed?
> > To note: I've also tried putting core-site.xml in the working directory,
> and
> > in a local conf/ directory.  This absolute path was a last effort to
> really
> > confirm that the local path wasn't somehow being mangled by the junit
> > runtime.
> >
> > @Test
> >     public void test() throws Exception{
> >         Configuration conf = new Configuration(false);
> >         System.out.println("b4:"+conf);
> >         String
> > theXMLFile="/Users/jayunit100/Development/blah/conf/core-site.xml";
> >         conf.addResource(theXMLFile);
> >         System.out.println("File exists : " + theXMLFile.length());
> >         System.out.println("after:"+conf);
> >         System.out.println("file sytem " + conf.get("fs.default.name"));
> >         conf.writeXml(System.out);
> >
> >
> >         System.out.println("Working directory = " +
> > System.getProperty("user.dir"));
> >         org.apache.hadoop.fs.FileSystem fs > > org.apache.hadoop.fs.FileSystem.get(conf);
> >         System.out.println("before test filesystem = " + fs);
> >         //WordCount.main(new
> > String[]{"resources/wordcount.txt","wordcountout.txt"});
> >     }
> >
> > --
> > Jay Vyas
> > http://jayunit100.blogspot.com
>

--
Jay Vyas
http://jayunit100.blogspot.com
+
Julien Muller 2013-01-17, 15:36
+
Vinod Kumar Vavilapalli 2013-01-29, 01:58
+
Colin McCabe 2013-01-28, 22:47