Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Question on adding Hadoop XML to Configuration Object


Copy link to this message
-
Re: Question on adding Hadoop XML to Configuration Object
Also - make sure the fs.default.name parameter is actually in
core-site.xml, not hdfs-site.xml

On 9 August 2012 09:32, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Try this and let me know if it works,
>
>                 Configuration conf = new Configuration();
>                 conf.addResource(new
> Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
>                 conf.addResource(new
> Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
>                 FileSystem fs = FileSystem.get(conf);
> Regards,
>     Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > I have added other xml files too. But the issue is configuration object
> is not getting updated with the xml conents.
> >
> > When I add
> >
> >         System.out.println(configuration.get("fs.default.name"));
> >
> > I am not getting my hdfs url which I have in core-site.xml. When I run
> my code in eclipse, it works. But when I make it as a runnable jar file, it
> outputs file:///
> >
> > Regards,
> > Anand.C
> >
> > -----Original Message-----
> > From: Mohammad Tariq [mailto:[EMAIL PROTECTED]]
> > Sent: Thursday, August 09, 2012 1:28 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: Question on adding Hadoop XML to Configuration Object
> >
> > Hello there,
> >
> >         Add the "conf/core-site.xml" file as well.
> >
> > Regards,
> >     Mohammad Tariq
> >
> >
> > On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
> > <[EMAIL PROTECTED]> wrote:
> >> Hi,
> >>
> >>
> >>
> >> I am trying to add a file to HDFS programmatically.
> >>
> >>
> >>
> >> In my code, I am adding hdfs-site.xml and other xml to Hadoop
> Configuration
> >> object as follows
> >>
> >>
> >>
> >> Configuration configuration = null;
> >>
> >> configuration.addResource(new
> >> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >>
> >> configuration.addResource(new
> >> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
> >>
> >> configuration.reloadConfiguration();
> >>
> >> fileSystem = FileSystem.get(configuration);
> >>
> >> System.out.println(fileSystem.getName());
> >>
> >>
> >>
> >> This code prints file system as file:/// instead of hdfs://.
> >>
> >>
> >>
> >> When I manually set HDFS configuration parameter like below, it works
> well
> >>
> >>
> >>
> >> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
> >>
> >>
> >>
> >> How can I make my code work? Any inputs would be greatly appreciated.
> >>
> >>
> >>
> >> Regards,
> >>
> >> Anand.C
> >>
> >>
> >>
> >>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB