Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Question on adding Hadoop XML to Configuration Object


Copy link to this message
-
Re: Question on adding Hadoop XML to Configuration Object
Dave Beech 2012-08-09, 08:34
Also - make sure the fs.default.name parameter is actually in
core-site.xml, not hdfs-site.xml

On 9 August 2012 09:32, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Try this and let me know if it works,
>
>                 Configuration conf = new Configuration();
>                 conf.addResource(new
> Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
>                 conf.addResource(new
> Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
>                 FileSystem fs = FileSystem.get(conf);
> Regards,
>     Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > I have added other xml files too. But the issue is configuration object
> is not getting updated with the xml conents.
> >
> > When I add
> >
> >         System.out.println(configuration.get("fs.default.name"));
> >
> > I am not getting my hdfs url which I have in core-site.xml. When I run
> my code in eclipse, it works. But when I make it as a runnable jar file, it
> outputs file:///
> >
> > Regards,
> > Anand.C
> >
> > -----Original Message-----
> > From: Mohammad Tariq [mailto:[EMAIL PROTECTED]]
> > Sent: Thursday, August 09, 2012 1:28 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: Question on adding Hadoop XML to Configuration Object
> >
> > Hello there,
> >
> >         Add the "conf/core-site.xml" file as well.
> >
> > Regards,
> >     Mohammad Tariq
> >
> >
> > On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> > <[EMAIL PROTECTED]> wrote:
> >> Hi,
> >>
> >>
> >>
> >> I am trying to add a file to HDFS programmatically.
> >>
> >>
> >>
> >> In my code, I am adding hdfs-site.xml and other xml to Hadoop
> Configuration
> >> object as follows
> >>
> >>
> >>
> >> Configuration configuration = null;
> >>
> >> configuration.addResource(new
> >> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >>
> >> configuration.addResource(new
> >> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >>
> >> configuration.reloadConfiguration();
> >>
> >> fileSystem = FileSystem.get(configuration);
> >>
> >> System.out.println(fileSystem.getName());
> >>
> >>
> >>
> >> This code prints file system as file:/// instead of hdfs://.
> >>
> >>
> >>
> >> When I manually set HDFS configuration parameter like below, it works
> well
> >>
> >>
> >>
> >> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
> >>
> >>
> >>
> >> How can I make my code work? Any inputs would be greatly appreciated.
> >>
> >>
> >>
> >> Regards,
> >>
> >> Anand.C
> >>
> >>
> >>
> >>
>