-Re: Feed hdfs with external data.
Hemanth Yamijala 2010-06-23, 09:19
> Ok, thank a lot about this, it worked out by specifying manually the conf...
> as in:
> Configuration config = new Configuration();
> config.set("dfs.name.dir", "/hd/name/");
> config.set("dfs.data.dir", "/hd/data/");
> config.set("fs.default.name", "hdfs://hadoop1/");
> Now, I would like to specify the conf dir location and have all this loaded
You can use --config <path to conf dir> to your bin/hadoop commands. I
think it would also work if you set the HADOOP_CONF_DIR environment
variable to point to this path.
> On Wed, Jun 23, 2010 at 10:52 AM, Hemanth Yamijala <[EMAIL PROTECTED]>wrote:
>> > I have a program that generates the data that's supposed to be treated by
>> > hadoop.
>> > It's a java program that should write right on hdfs.
>> > So as a test, I do this:
>> > Configuration config = new Configuration();
>> > FileSystem dfs = FileSystem.get(config);
>> > dfs.mkdirs(new Path("/test"));
>> > Which of course failed, certainly because it's unaware of the position of
>> > the configuration files.
>> > How should I do this?
>> > Thanks.
>> > P.S. Just in case, here's my error:
>> > 10/06/23 10:04:06 ERROR conf.Configuration: Failed to set
>> > setXIncludeAware(true) for parser
>> > gnu.xml.dom.JAXPFactory@1d6096:java.lang.UnsupportedOperationException:
>> > This parser does not support specification "null" version "null"
>> > java.lang.UnsupportedOperationException: This parser does not support
>> > specification "null" version "null"
>> > at
>> > at
>> This exception seems very similar to the one we faced and fixed using
>> the information here: http://bit.ly/a5zX1e. Could you give it a try ?
> Ego sum quis ego servo
> "Je suis ce que je protège"
> "I am what I protect"