Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: core-site.xml file is being ignored by new Configuration()


+
Tom Melendez 2013-01-17, 00:45
+
Jay Vyas 2013-01-17, 05:01
+
Julien Muller 2013-01-17, 15:36
Copy link to this message
-
Re: core-site.xml file is being ignored by new Configuration()
Can you try prepending the path with file:// or pass in a URL and use the method which takes in a URL?

I remember class loading issues with paths without the file: scheme info from a different scenario (while playing with logj4 config files).

HTH,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/

On Jan 16, 2013, at 4:33 PM, Jay Vyas wrote:

> Hi guys:
>
> I've finally extracted my problem of loading a special filesystem
> into a unit test.
>
> Below, clearly, Im creating a raw configuration and adding a single resource to it (core-site.xml).
>
> Afterwards, i print to confirm that the file exists.  
>
> Finally, I try to create a FileSystem from that file.  However, no file system is created :(.
>
> Any thoughts on why this core-site.xml file is being ignoreed?
> To note: I've also tried putting core-site.xml in the working directory, and in a local conf/ directory.  This absolute path was a last effort to really confirm that the local path wasn't somehow being mangled by the junit runtime.
>
> @Test
>     public void test() throws Exception{
>         Configuration conf = new Configuration(false);
>         System.out.println("b4:"+conf);
>         String theXMLFile="/Users/jayunit100/Development/blah/conf/core-site.xml";
>         conf.addResource(theXMLFile);
>         System.out.println("File exists : " + theXMLFile.length());
>         System.out.println("after:"+conf);
>         System.out.println("file sytem " + conf.get("fs.default.name"));
>         conf.writeXml(System.out);
>
>        
>         System.out.println("Working directory = " + System.getProperty("user.dir"));
>         org.apache.hadoop.fs.FileSystem fs = org.apache.hadoop.fs.FileSystem.get(conf);
>         System.out.println("before test filesystem = " + fs);
>         //WordCount.main(new String[]{"resources/wordcount.txt","wordcountout.txt"});
>     }
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com

+
Colin McCabe 2013-01-28, 22:47
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB