Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> reply: a question about dfs.replication


+
Francis.Hu 2013-07-02, 04:25
+
Azuryy Yu 2013-07-02, 04:39
Copy link to this message
-
Re: reply: a question about dfs.replication
HI HU and Yu

    Aggree with dfs.replication is a client side configuration, not server
side. It make the point in my last mail  sense.

    And the cmd:hdfs dfs -setrep -R -w 2 /  solve the problem that I can
not change the existed file's replication value.
2013/7/2 Azuryy Yu <[EMAIL PROTECTED]>

> It's not HDFS issue.
>
> dfs.replication is a client side configuration, not server side. so you
> need to set it to '2' on your client side( your application running on).
> THEN execute command such as : hdfs dfs -put  or call HDFS API in java
> application.
>
>
>
>
> On Tue, Jul 2, 2013 at 12:25 PM, Francis.Hu <[EMAIL PROTECTED]>wrote:
>
>>  Thanks all of you, I just get the problem fixed through the command: ***
>> *
>>
>> hdfs dfs -setrep -R -w 2 /****
>>
>> ** **
>>
>> Is that an issue of HDFS ? Why do i need to execute manually a command to
>> tell the hadoop the replication factor even it is set in hdfs-site.xml ?*
>> ***
>>
>> ** **
>>
>> Thanks,****
>>
>> Francis.Hu****
>>
>> ** **
>>
>> *发件人:* Francis.Hu [mailto:[EMAIL PROTECTED]]
>> *发送时间:* Tuesday, July 02, 2013 11:30
>> *收件人:* [EMAIL PROTECTED]
>> *主题:* 答复: 答复: a question about dfs.replication****
>>
>> ** **
>>
>> Yes , it returns 2 correctly after "hdfs getconf -confkey
>> dfs.replication"****
>>
>> ** **
>>
>> ****
>>
>> ** **
>>
>> but in web page ,it is 3 as below:****
>>
>> ****
>>
>> ** **
>>
>> *发件人:* yypvsxf19870706 [mailto:[EMAIL PROTECTED]]
>> *发送时间:* Monday, July 01, 2013 23:24
>> *收件人:* [EMAIL PROTECTED]
>> *主题:* Re: 答复: a question about dfs.replication****
>>
>> ** **
>>
>> Hi ****
>>
>> ** **
>>
>>     Could you please get the property value by using : hdfs getconf
>> -confkey dfs.replication.****
>>
>>
>> ������ iPhone****
>>
>>
>> �?2013-7-1锛?5:51锛�rancis.Hu <[EMAIL PROTECTED]> ���锛?br>****
>>
>>   ****
>>
>> Actually, My java client is running with the same configuration as the
>> hadoop's . The dfs.replication is already set as 2 in my hadoop's
>> configuration.****
>>
>> So i think the dfs.replication is already overrided by my configuration
>> in hdfs-site.xml. but seems it doesn't work even i overrided the parameter
>> evidently.****
>>
>>  ****
>>
>>  ****
>>
>> *��欢浜**?span lang="EN-US">:* �屑械谢��薪芯胁 �芯�懈� [mailto:[EMAIL PROTECTED]<[EMAIL PROTECTED]>]
>>
>> *��**�*���**?span lang="EN-US">:* Monday, July 01, 2013 15:18
>> *�朵欢浜?span lang="EN-US">:* [EMAIL PROTECTED]
>> *涓婚**�:* Re: a question about dfs.replication****
>>
>>  ****
>>
>> On 01.07.2013 10:19, Francis.Hu wrote:****
>>
>> Hi, All****
>>
>>  ****
>>
>> I am installing a cluster with Hadoop 2.0.5-alpha. I have one namenode
>> and two datanodes. The dfs.replication is set as 2 in hdfs-site.xml.
>> After all configuration work is done, I started all nodes. Then I saved a
>> file into HDFS through java client. nOW I can access hdfs web page:
>> x.x.x.x:50070,and also see the file is already listed in the hdfs list.**
>> **
>>
>> My question is:  *The replication column in HDFS web page is showing as
>> 3, not 2.  Does anyone know What the problem is?*****
>>
>> * *****
>>
>> ---Actual setting of hdfs-site.xml****
>>
>> <property>****
>>
>> <name>dfs.replication</name>****
>>
>> <value>2</value>****
>>
>> </property>****
>>
>>  ****
>>
>> After that, I typed dfsamdin command to check the file:****
>>
>> hdfs fsck /test3/****
>>
>> The result of above command:****
>>
>> /test3/hello005.txt:  Under replicated
>> BP-609310498-192.168.219.129-1372323727200:blk_-1069303317294683372_1006.
>> Target Replicas is 3 but found 2 replica(s).****
>>
>> Status: HEALTHY****
>>
>>  Total size:    35 B****
>>
>>  Total dirs:    1****
>>
>>  Total files:   1****
>>
>>  Total blocks (validated):      1 (avg. block size 35 B)****
>>
>>  Minimally replicated blocks:   1 (100.0 %)****
>>
>>  Over-replicated blocks:        0 (0.0 %)****
>>
>>  Under-replicated blocks:       1 (100.0 %)****
>>
>>  Mis-replicated blocks:         0 (0.0 %)****
>>
>>  Default replication factor:    2****
+
Francis.Hu 2013-07-02, 06:22
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB