Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Misconfiguration of hdfs-site.xml


+
Jean-Marc Spaggiari 2012-12-18, 20:28
+
Harsh J 2012-12-18, 20:36
Copy link to this message
-
Re: Misconfiguration of hdfs-site.xml
> Updating the config and restarting will result in data loss.

Ok. If that's the case, then I can still re-configure the servers one
by one manually and restart them. And moving to the next server only
when the replication is completed.

> dfs.data.dir presently is /home/hadoop/haddop_drive/${user.name}/dfs/data
> dfs.name.dir presently is /home/hadoop/haddop_drive/${user.name}/dfs/name

Ok. That sound like what I was expecting. I don't really want to move
the data/name directories. My goal is just to fix the lack of
configuration.

I will try to update with the properties above on one of the servers
and restart it. And if it's working fine, I will apply that on all the
others.

Thanks,

JM

2012/12/18, Harsh J <[EMAIL PROTECTED]>:
> Given your previous config is
> "/home/hadoop/haddop_drive/${user.name}", if the defaults are to go
> by, then:
>
> dfs.data.dir presently is /home/hadoop/haddop_drive/${user.name}/dfs/data
> dfs.name.dir presently is /home/hadoop/haddop_drive/${user.name}/dfs/name
>
> If you are specifying explicit paths, it will need to be the full one
> (preferably pre-evaluated for ${user.name}) as above, exactly.
>
> If you wish to move the location somewhere else, you will need to mv
> the {data,name} directories elsewhere and re-point down to that path
> component again.
>
> On Wed, Dec 19, 2012 at 1:58 AM, Jean-Marc Spaggiari
> <[EMAIL PROTECTED]> wrote:
>> Hi,
>>
>> For months now I'm using my hadoop cluster with absolutly nothing
>> related to the drive directory on my hdfs-site.xml file.
>>
>> It seems that it's using the hadoop.tmp.dir directory to store data.
>>
>> My hadoop.tmp.dir is pointing to
>> /home/hadoop/haddop_drive/${user.name} and on my
>> /home/hadoop/haddop_drive directory I can see hadoop and hbase.
>>
>> Now, I want to configure that properly without loosing the data I have.
>>
>> I'm using 1.0.3.
>>
>> Based on the documentation
>> http://hadoop.apache.org/docs/r1.0.3/cluster_setup.html#Site+Configuration
>> it seems I need to setup dfs.data.dir to point to
>> /home/hadoop/haddop_drive. If I add this entry and restart my
>> datanode, will it "simply" continue to point to the same place and
>> find the data he needs?
>>
>> Also, what about dfs.name.dir ? I guess I can simply point it to the
>> same place, right?
>>
>> On  /home/hadoop/haddop_drive/hadoop/dfs I have data and name
>> directories. So I'm not sure if I should point the properties above to
>>  /home/hadoop/haddop_drive or to  /home/hadoop/haddop_drive/dfs.
>>
>> Thanks,
>>
>> JM
>
>
>
> --
> Harsh J
>
+
Bryan Beaudreault 2012-12-18, 20:36
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB