Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - starting Daemons remotely, through ssh


Copy link to this message
-
RE: starting Daemons remotely, through ssh - SOLVED
Kartashov, Andy 2012-11-06, 20:46
Forrest,

I must admit. I am very new myself.  Two months ago I didn't know what "$sudo" was. :) So, do not take my comments too seriously. I could be wrong about many issues.
There are 5 daemon scripts  in /etc/init.d/. I am using MRv1, so mine are:

hadoop-hdfs-datanode
hadoop-hdfs-namenode
hadoop-hdfs-secondarynamenode
<specific to MRv1>
hadoop-0.20-mapreduce-jobtracker
hadoop-0.20-mapreduce-tasktracker

For scalability I chose to:

-           only have NN/JT on my master node so I need to start|stop|restart only hadoop-hdfs-namenode/hadoop-0.20-mapreduce-jobtracker

-          different node for SNN so I need to only start|stop|restart hadoop-hdfs-secondarynamenode

-          all of my DNs - hadoop-hdfs-datanode/hadoop-0.20-mapreduce-tasktracker
My script is VERY simple for now:

#!/bin/bash
# myHadoop.sh
dir=/etc/init.d
if [ $# -lt 1 ]; then
  echo "USAGE: $`basename $0` [start|stop|restart]"
  exit 1
fi
sudo ${dir}/hadoop-hdfs-namenode  $1
sudo ${dir}/hadoop-0.20-mapreduce-jobtracker  $1
exit 0

for SNN I altered script to run instead:
sudo ${dir}/hadoop-hdfs-secondarynamenode  $1

      and for DNs:
sudo ${dir}/hadoop-hdfs-datanode  $1
sudo ${dir}/hadoop-0.20-mapreduce-tasktracker $1

p.s. Criticism on my post is welcomed.

AK47
From: Forrest Aldrich [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, November 06, 2012 3:09 PM
To: [EMAIL PROTECTED]
Subject: Re: starting Daemons remotely, through ssh - SOLVED

Andy, I noticed you are splitting up your /etc/init.d startup -- we're new-ish to Hadoop and found that the supplied init.d script doesn't necessarily stop services properly, so I end up doing it manually.   If you could share your info that would be great... I can see where they can be time-delayed in the startup with chkconfig (RHEL).
Thanks.
On 11/6/12 3:07 PM, Kartashov, Andy wrote:
Harsh/Ravi,

I wrote my own scripts to [start|stop|restart]  [hdfs|mapred] daemons. The content of the script is

$sudo service /etc/init.d/hadoop-[hdfs-*|mapred-*]

I have no problem starting Daemons locally on each node. I have sudo-access granted. I run script and enter password.
I have passwordless ssh enabled. But when I am trying to start slave Daemons from NN I get
"<remothsot>:  sudo: sorry, you must have a tty to run sudo".

I just solved this issue by:

a.       Modifying slaves.sh script to add -t after ssh "-t" <host-ip> <command>
a1. Someone suggested to:

$sudo sudeors and commenting out Defaults requiretty

But a. worked just fine.

Cheers,

AK47

From: Ravi Mutyala [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, November 06, 2012 2:31 PM
To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
Subject: Re: starting Daemons remotely, through ssh

Andy,

Which specific scripts are these? As far as I know, all daemons on the hadoop side can be started with hadoop/hdfs or any non-sudo username. You will require passwordless ssh for that user if you are using slaves/regionservers configured to start the daemons remotely from the NameNode/HBase Master.
On Tue, Nov 6, 2012 at 1:25 PM, Kartashov, Andy <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hadoopers,

How does one start Daemons remotely when scripts normally require root user to start them? Do you modify scripts?

Thanks,
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel