Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Modify the number of map tasks


Copy link to this message
-
Re: Modify the number of map tasks
Mohammad Tariq 2012-12-12, 15:37
No. But the process is pretty simple. You just have to copy the set of conf
file across all the machines after a few minor changes.
The process goes like this :
Add the IPs and hostnames of each machines into the /etc/hosts file of rest
of the files.
Make ssh(passwordless) work across all the machines.
Modify the core-site.xml, hdfs-site.xml and mapred-site.xml(as specified
above).
Add the name of all slave machines to the /conf/slaves file and the machine
name, where you want to run the Secondary NameNode, in the /etc/master file.

Copy these files across all the machines in your cluster and you are good
to go.
For a detailed info you can go here :
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/

Regards,
    Mohammad Tariq

On Wed, Dec 12, 2012 at 8:55 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:

> have you a page in which you explain the steps.
>
>
>
> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>
>> Hi Imen,
>>
>>          I am sorry, I didn't get the question. Are you asking about
>> creating a distributed cluster? Yeah, I have done that.
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>>
>> On Wed, Dec 12, 2012 at 7:45 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:
>>
>>> have you please commented the configuration of hadoop on cluster
>>>
>>> thanks
>>>
>>>
>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>
>>>> You are always welcome. If you still need any help, you can go here :
>>>> http://cloudfront.blogspot.in/2012/07/how-to-configure-hadoop.html
>>>> I have outlined the entire process here along with few small(but
>>>> necessary) explanations.
>>>>
>>>> Regards,
>>>>     Mohammad Tariq
>>>>
>>>>
>>>>
>>>> On Wed, Dec 12, 2012 at 7:31 PM, imen Megdiche <[EMAIL PROTECTED]
>>>> > wrote:
>>>>
>>>>> thank you very much you re awsome.
>>>>>
>>>>> Fixed
>>>>>
>>>>>
>>>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>>>
>>>>>> Uncomment the property in core-site.xml. That is a must. After doing
>>>>>> this  you have to restart the daemons?
>>>>>>
>>>>>> Regards,
>>>>>>     Mohammad Tariq
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Dec 12, 2012 at 7:08 PM, imen Megdiche <
>>>>>> [EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>>> I changed the files
>>>>>>> now when i run i have this response :
>>>>>>>
>>>>>>> 12/12/12 14:37:33 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 0 time(s).
>>>>>>> 12/12/12 14:37:34 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 1 time(s).
>>>>>>> 12/12/12 14:37:35 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 2 time(s).
>>>>>>> 12/12/12 14:37:36 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 3 time(s).
>>>>>>> 12/12/12 14:37:37 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 4 time(s).
>>>>>>> 12/12/12 14:37:38 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 5 time(s).
>>>>>>> 12/12/12 14:37:39 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 6 time(s).
>>>>>>> 12/12/12 14:37:40 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 7 time(s).
>>>>>>> 12/12/12 14:37:41 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 8 time(s).
>>>>>>> 12/12/12 14:37:42 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 9 time(s).
>>>>>>> Exception in thread "main" java.net.ConnectException: Call to
>>>>>>> localhost/127.0.0.1:9001 failed on connection exception:
>>>>>>> java.net.ConnectException: Connexion refusée
>>>>>>>     at org.apache.hadoop.ipc.Client.wrapException(Client.java:1099)
>>>>>>>     at org.apache.hadoop.ipc.Client.call(Client.java:1075)
>>>>>>>     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>>