Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Re: core-site.xml file is being ignored by new Configuration()


+
Tom Melendez 2013-01-17, 00:45
+
Jay Vyas 2013-01-17, 05:01
+
Julien Muller 2013-01-17, 15:36
Copy link to this message
-
Re: core-site.xml file is being ignored by new Configuration()
Vinod Kumar Vavilapalli 2013-01-29, 01:58
Can you try prepending the path with file:// or pass in a URL and use the method which takes in a URL?

I remember class loading issues with paths without the file: scheme info from a different scenario (while playing with logj4 config files).

HTH,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/

On Jan 16, 2013, at 4:33 PM, Jay Vyas wrote:

> Hi guys:
>
> I've finally extracted my problem of loading a special filesystem
> into a unit test.
>
> Below, clearly, Im creating a raw configuration and adding a single resource to it (core-site.xml).
>
> Afterwards, i print to confirm that the file exists.  
>
> Finally, I try to create a FileSystem from that file.  However, no file system is created :(.
>
> Any thoughts on why this core-site.xml file is being ignoreed?
> To note: I've also tried putting core-site.xml in the working directory, and in a local conf/ directory.  This absolute path was a last effort to really confirm that the local path wasn't somehow being mangled by the junit runtime.
>
> @Test
>     public void test() throws Exception{
>         Configuration conf = new Configuration(false);
>         System.out.println("b4:"+conf);
>         String theXMLFile="/Users/jayunit100/Development/blah/conf/core-site.xml";
>         conf.addResource(theXMLFile);
>         System.out.println("File exists : " + theXMLFile.length());
>         System.out.println("after:"+conf);
>         System.out.println("file sytem " + conf.get("fs.default.name"));
>         conf.writeXml(System.out);
>
>        
>         System.out.println("Working directory = " + System.getProperty("user.dir"));
>         org.apache.hadoop.fs.FileSystem fs = org.apache.hadoop.fs.FileSystem.get(conf);
>         System.out.println("before test filesystem = " + fs);
>         //WordCount.main(new String[]{"resources/wordcount.txt","wordcountout.txt"});
>     }
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com

+
Colin McCabe 2013-01-28, 22:47