Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - about replication


Copy link to this message
-
Re: about replication
manish dunani 2013-08-06, 06:10
hello,

I think u r newbie.You need to work on to learn from scratch.
Please do not mind.whatever u try to think hadoop should be like this, ur
concept is totally wrong.
but,your effort is positive.

My suggestion is try to learn on ubuntu os.Windows is not good enough for
hadoop.

Follow the link to setup single node cluster on ubuntu.

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/

If u stop anywhere and do not get idea what to do next? just revert me .

Also read this:

https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxtYW5pc2hkdW5hbml8Z3g6NGRiM2JhODhhOGEzMTcyYw
*Thanks & Regards*
*
*
*Manish dunani*

On Tue, Aug 6, 2013 at 10:29 AM, Irfan Sayed <[EMAIL PROTECTED]> wrote:

> thanks.
> i verified, datanode is up and running
>
> i ran the below command:
>
> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
> $ ./hadoop dfs -copyFromLocal
> C:\\Users\\Administrator\\Desktop\\hadoop-1.1.2.tar /wksp
> copyFromLocal: File C:/Users/Administrator/Desktop/hadoop-1.1.2.tar does
> not exist.
>
> it says , file does not exist. as my cluster is windows based , i dont
> know how dir path needs to be used
> i am using cygwin for linux style formatting
>
> i tried below as well :
>
> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
> $ ./hadoop dfs -copyFromLocal
> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar /wksp
> copyFromLocal: File
> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar does not exist.
>
> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
> $ ./hadoop dfs -copyFromLocal
> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz /wksp
> copyFromLocal: File
> /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz does not exist.
>
> please suggest
>
> regards
>
>
>
> On Mon, Aug 5, 2013 at 6:00 PM, manish dunani <[EMAIL PROTECTED]>wrote:
>
>> You can not physically access the datanode.You have to understand it to
>> logically and it really happens.
>>
>> Type "jps" command to check ur datanode was started or not.
>>
>> when user stores the file into hdfs ,the request is goes to datanode and
>> datanode will divide the file into number of blocks.
>> Each block size is 64mb or 128mb.but,default is 64 mb.
>>
>> The default replication factor is 1 that was u already set in ur
>> hdfs-site.xml .If u want to change ur replication factor from 1 to 3 for
>> particular file or particular directory in hdfs then use the below commands
>> appropriately.
>>
>> First load any file to hdfs using command:
>>
>> bin/hadoop  dfs -copyFromLocal /ur/local/directory/path/to/file
>>  /ur/hdfs/directory/path.
>>
>> *commands:*
>>
>> To set replication of an individual file to 4:
>>
>> ./bin/hadoop dfs -setrep -w 4 /path/to/file
>>
>> You can also do this recursively. To change replication of entire HDFS to
>> 1:
>>
>> ./bin/hadoop dfs -setrep -R -w 1 /
>> *
>> *
>>  If the replication factor is 3 then data will divided into 3 blocks and
>> replicated to various data nodes of ur production cluster(multi node).
>>
>> Here Replication means same file copied three times on data nodes for
>> handling hardware failure.
>>
>>
>> On Mon, Aug 5, 2013 at 5:29 PM, Irfan Sayed <[EMAIL PROTECTED]> wrote:
>>
>>> thanks.
>>> please refer below:
>>>
>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>> $ ./hadoop dfs -ls /wksp
>>> Found 1 items
>>> drwxr-xr-x   - Administrator Domain          0 2013-08-05 16:58
>>> /wksp/New folder
>>>
>>> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>> $
>>>
>>>
>>> same command if i run on the datanode then it says:
>>>
>>>  Administrator@DFS-1 /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin
>>> $ ./hadoop dfs -ls /wksp
>>> ls: Cannot access /wksp: No such file or directory.
>>>
>>> does it mean that replication is not yet started ...?
>>>
>>> please suggest
>>>
>>> regards
>>>
>>>
>>>
>>> On Mon, Aug 5, 2013 at 5:18 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
MANISH DUNANI
-THANX
+91 9426881954,+91 8460656443
[EMAIL PROTECTED]