Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: core-site.xml file is being ignored by new Configuration()


+
Tom Melendez 2013-01-17, 00:45
Copy link to this message
-
Re: core-site.xml file is being ignored by new Configuration()
no difference.

This is really odd.

The only way I can get it to work is by the following:
/**
 * A simple test harness for glusterfs
 * @author jvyas
 */
public class TestMapReducerGlusterFS {

    @Test
    public void test() throws Exception{
        Configuration conf = new Configuration(false);
        System.out.println("before reload:"+conf);
        //necessary?
        conf.set("fs.default.name", ".......");
        conf.set("fs.blah.mount", "/mnt/......");
        conf.set("fs.blahserver", "192.168.1.36");
        conf.set("iomod","Off");
         ...
    }
}

Thinking that this is a little known issue in the current hadoop
parser/configurator? because very rarely do people write unit tests which
utilize abnormal ( i.e. non-local or non-hdfs) file systems.
On Wed, Jan 16, 2013 at 7:45 PM, Tom Melendez <[EMAIL PROTECTED]> wrote:

> Result any different if you call reloadConfiguration()?
>
> Thanks,
>
> Tom
>
> On Wed, Jan 16, 2013 at 4:33 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
> > Hi guys:
> >
> > I've finally extracted my problem of loading a special filesystem
> > into a unit test.
> >
> > Below, clearly, Im creating a raw configuration and adding a single
> resource
> > to it (core-site.xml).
> >
> > Afterwards, i print to confirm that the file exists.
> >
> > Finally, I try to create a FileSystem from that file.  However, no file
> > system is created :(.
> >
> > Any thoughts on why this core-site.xml file is being ignoreed?
> > To note: I've also tried putting core-site.xml in the working directory,
> and
> > in a local conf/ directory.  This absolute path was a last effort to
> really
> > confirm that the local path wasn't somehow being mangled by the junit
> > runtime.
> >
> > @Test
> >     public void test() throws Exception{
> >         Configuration conf = new Configuration(false);
> >         System.out.println("b4:"+conf);
> >         String
> > theXMLFile="/Users/jayunit100/Development/blah/conf/core-site.xml";
> >         conf.addResource(theXMLFile);
> >         System.out.println("File exists : " + theXMLFile.length());
> >         System.out.println("after:"+conf);
> >         System.out.println("file sytem " + conf.get("fs.default.name"));
> >         conf.writeXml(System.out);
> >
> >
> >         System.out.println("Working directory = " +
> > System.getProperty("user.dir"));
> >         org.apache.hadoop.fs.FileSystem fs > > org.apache.hadoop.fs.FileSystem.get(conf);
> >         System.out.println("before test filesystem = " + fs);
> >         //WordCount.main(new
> > String[]{"resources/wordcount.txt","wordcountout.txt"});
> >     }
> >
> > --
> > Jay Vyas
> > http://jayunit100.blogspot.com
>

--
Jay Vyas
http://jayunit100.blogspot.com
+
Julien Muller 2013-01-17, 15:36
+
Vinod Kumar Vavilapalli 2013-01-29, 01:58
+
Colin McCabe 2013-01-28, 22:47
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB