Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Modify the number of map tasks


Copy link to this message
-
Re: Modify the number of map tasks
No. But the process is pretty simple. You just have to copy the set of conf
file across all the machines after a few minor changes.
The process goes like this :
Add the IPs and hostnames of each machines into the /etc/hosts file of rest
of the files.
Make ssh(passwordless) work across all the machines.
Modify the core-site.xml, hdfs-site.xml and mapred-site.xml(as specified
above).
Add the name of all slave machines to the /conf/slaves file and the machine
name, where you want to run the Secondary NameNode, in the /etc/master file.

Copy these files across all the machines in your cluster and you are good
to go.
For a detailed info you can go here :
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/

Regards,
    Mohammad Tariq

On Wed, Dec 12, 2012 at 8:55 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:

> have you a page in which you explain the steps.
>
>
>
> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>
>> Hi Imen,
>>
>>          I am sorry, I didn't get the question. Are you asking about
>> creating a distributed cluster? Yeah, I have done that.
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>>
>> On Wed, Dec 12, 2012 at 7:45 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:
>>
>>> have you please commented the configuration of hadoop on cluster
>>>
>>> thanks
>>>
>>>
>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>
>>>> You are always welcome. If you still need any help, you can go here :
>>>> http://cloudfront.blogspot.in/2012/07/how-to-configure-hadoop.html
>>>> I have outlined the entire process here along with few small(but
>>>> necessary) explanations.
>>>>
>>>> Regards,
>>>>     Mohammad Tariq
>>>>
>>>>
>>>>
>>>> On Wed, Dec 12, 2012 at 7:31 PM, imen Megdiche <[EMAIL PROTECTED]
>>>> > wrote:
>>>>
>>>>> thank you very much you re awsome.
>>>>>
>>>>> Fixed
>>>>>
>>>>>
>>>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>>>
>>>>>> Uncomment the property in core-site.xml. That is a must. After doing
>>>>>> this  you have to restart the daemons?
>>>>>>
>>>>>> Regards,
>>>>>>     Mohammad Tariq
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Dec 12, 2012 at 7:08 PM, imen Megdiche <
>>>>>> [EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>>> I changed the files
>>>>>>> now when i run i have this response :
>>>>>>>
>>>>>>> 12/12/12 14:37:33 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 0 time(s).
>>>>>>> 12/12/12 14:37:34 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 1 time(s).
>>>>>>> 12/12/12 14:37:35 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 2 time(s).
>>>>>>> 12/12/12 14:37:36 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 3 time(s).
>>>>>>> 12/12/12 14:37:37 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 4 time(s).
>>>>>>> 12/12/12 14:37:38 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 5 time(s).
>>>>>>> 12/12/12 14:37:39 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 6 time(s).
>>>>>>> 12/12/12 14:37:40 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 7 time(s).
>>>>>>> 12/12/12 14:37:41 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 8 time(s).
>>>>>>> 12/12/12 14:37:42 INFO ipc.Client: Retrying connect to server:
>>>>>>> localhost/127.0.0.1:9001. Already tried 9 time(s).
>>>>>>> Exception in thread "main" java.net.ConnectException: Call to
>>>>>>> localhost/127.0.0.1:9001 failed on connection exception:
>>>>>>> java.net.ConnectException: Connexion refusée
>>>>>>>     at org.apache.hadoop.ipc.Client.wrapException(Client.java:1099)
>>>>>>>     at org.apache.hadoop.ipc.Client.call(Client.java:1075)
>>>>>>>     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB