Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Question on adding Hadoop XML to Configuration Object


Copy link to this message
-
Re: Question on adding Hadoop XML to Configuration Object
Also, Hadoop jars must be in the classpath when you run the app from
command line or as a jar.

Regards,
    Mohammad Tariq
On Thu, Aug 9, 2012 at 2:02 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:
> Try this and let me know if it works,
>
>                 Configuration conf = new Configuration();
>                 conf.addResource(new Path("YOUR_HADOOP_HOME/conf/core-site.xml"));
>                 conf.addResource(new Path("YOUR_HADOOP_HOME/conf/hdfs-site.xml"));
>                 FileSystem fs = FileSystem.get(conf);
> Regards,
>     Mohammad Tariq
>
>
> On Thu, Aug 9, 2012 at 1:58 PM, Chandra Mohan, Ananda Vel Murugan
> <[EMAIL PROTECTED]> wrote:
>> Hi,
>>
>> I have added other xml files too. But the issue is configuration object is not getting updated with the xml conents.
>>
>> When I add
>>
>>         System.out.println(configuration.get("fs.default.name"));
>>
>> I am not getting my hdfs url which I have in core-site.xml. When I run my code in eclipse, it works. But when I make it as a runnable jar file, it outputs file:///
>>
>> Regards,
>> Anand.C
>>
>> -----Original Message-----
>> From: Mohammad Tariq [mailto:[EMAIL PROTECTED]]
>> Sent: Thursday, August 09, 2012 1:28 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Question on adding Hadoop XML to Configuration Object
>>
>> Hello there,
>>
>>         Add the "conf/core-site.xml" file as well.
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>> On Thu, Aug 9, 2012 at 12:37 PM, Chandra Mohan, Ananda Vel Murugan
>> <[EMAIL PROTECTED]> wrote:
>>> Hi,
>>>
>>>
>>>
>>> I am trying to add a file to HDFS programmatically.
>>>
>>>
>>>
>>> In my code, I am adding hdfs-site.xml and other xml to Hadoop Configuration
>>> object as follows
>>>
>>>
>>>
>>> Configuration configuration = null;
>>>
>>> configuration.addResource(new
>>> URL("file:///usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.addResource(new
>>> Path("usr/local/hadoop-1.0.2/conf/hdfs-site.xml"));
>>>
>>> configuration.reloadConfiguration();
>>>
>>> fileSystem = FileSystem.get(configuration);
>>>
>>> System.out.println(fileSystem.getName());
>>>
>>>
>>>
>>> This code prints file system as file:/// instead of hdfs://.
>>>
>>>
>>>
>>> When I manually set HDFS configuration parameter like below, it works well
>>>
>>>
>>>
>>> //configuration.set("fs.default.name", "hdfs://10.78.32.252:54310");
>>>
>>>
>>>
>>> How can I make my code work? Any inputs would be greatly appreciated.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Anand.C
>>>
>>>
>>>
>>>