|
yogesh.kumar13@...
2012-07-20, 06:20
Bejoy KS
2012-07-20, 06:33
Mohammad Tariq
2012-07-20, 06:47
yogesh.kumar13@...
2012-07-20, 07:21
Bejoy KS
2012-07-20, 07:26
yogesh.kumar13@...
2012-07-20, 08:29
Mohammad Tariq
2012-07-20, 09:47
yogesh.kumar13@...
2012-07-20, 10:28
Bejoy Ks
2012-07-20, 10:44
Mohammad Tariq
2012-07-20, 11:39
VIGNESH PRAJAPATI
2012-07-20, 06:50
|
-
NameNode failsyogesh.kumar13@... 2012-07-20, 06:20
Hello All :-),
I am new to Hdfs. I have installed single node hdfs and started all nodes, every nodes gets started and work fine. But when I shutdown my system or Restart it, then i try to run all nodes but Namenode doesn't start . to Start it i need to format the namenode and all data gets wash off :-(. Please help me and suggest me regarding this and how can I recover namenode from secondary name node on single node setup Thanks & Regards Yogesh Kumar Dhari Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com +
yogesh.kumar13@... 2012-07-20, 06:20
-
Re: NameNode failsBejoy KS 2012-07-20, 06:33
Hi Yogesh
Is your dfs.name.dir pointing to /tmp dir? If so try changing that to any other dir . The contents of /tmp may get wiped out on OS restarts. Regards Bejoy KS Sent from handheld, please excuse typos. -----Original Message----- From: <[EMAIL PROTECTED]> Date: Fri, 20 Jul 2012 06:20:02 To: <[EMAIL PROTECTED]> Reply-To: [EMAIL PROTECTED] Subject: NameNode fails Hello All :-), I am new to Hdfs. I have installed single node hdfs and started all nodes, every nodes gets started and work fine. But when I shutdown my system or Restart it, then i try to run all nodes but Namenode doesn't start . to Start it i need to format the namenode and all data gets wash off :-(. Please help me and suggest me regarding this and how can I recover namenode from secondary name node on single node setup Thanks & Regards Yogesh Kumar Dhari Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com +
Bejoy KS 2012-07-20, 06:33
-
Re: NameNode failsMohammad Tariq 2012-07-20, 06:47
Hi yogesh,
Do as suggested by bejoy and add the hadoop.tmp.dir in your core-site.xml file.it's value defaults to the /tmp dir which gets emptied at each restart so all the data is lost... also add dfs.name.dir and dfs.data.dir properties into your hdfs-site.xml file to avoid the data loss. On Friday, July 20, 2012, Bejoy KS <[EMAIL PROTECTED]> wrote: > Hi Yogesh > > Is your dfs.name.dir pointing to /tmp dir? If so try changing that to any other dir . The contents of /tmp may get wiped out on OS restarts. > Regards > Bejoy KS > > Sent from handheld, please excuse typos. > ________________________________ > From: <[EMAIL PROTECTED]> > Date: Fri, 20 Jul 2012 06:20:02 +0000 > To: <[EMAIL PROTECTED]> > ReplyTo: [EMAIL PROTECTED] > Subject: NameNode fails > Hello All :-), > > I am new to Hdfs. > > I have installed single node hdfs and started all nodes, every nodes gets started and work fine. > But when I shutdown my system or Restart it, then i try to run all nodes but Namenode doesn't start . > > to Start it i need to format the namenode and all data gets wash off :-(. > > Please help me and suggest me regarding this and how can I recover namenode from secondary name node on single node setup > > > Thanks & Regards > Yogesh Kumar Dhari > > Please do not print this email unless it is absolutely necessary. > > The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. > > www.wipro.com -- Regards, Mohammad Tariq +
Mohammad Tariq 2012-07-20, 06:47
-
RE: NameNode failsyogesh.kumar13@... 2012-07-20, 07:21
Thanks Bejoy, Mohammad and Vignesh :-).
I have done the suggested by you and made thses changes. and formatted the namenode and trying to start cluster. but now name node is not starting :-( hdfs-site.xml ********************************************************** <configuration> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.name.dir</name> <value>/HADOOP/hadoop-0.20.2/hadoop_name_dirr</value> </property> <property> <name>dfs.data.dir</name> <value>/HADOOP/hadoop-0.20.2/hadoop_data_dirr</value> </property> </configuration> ************************************************************** hdfs-core.xml ************************************************************** <configuration> <property> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/HADOOP/hadoop-0.20.2/hadoop_temp_dirr</value> <description>A base for other temporary directories.</description> </property> </configuration> ***************************************************************** Please suggest. Thanks & Regards Yogesh Kumar Dhari ________________________________ From: Mohammad Tariq [[EMAIL PROTECTED]] Sent: Friday, July 20, 2012 12:17 PM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: NameNode fails Hi yogesh, Do as suggested by bejoy and add the hadoop.tmp.dir in your core-site.xml file.it<http://file.it>'s value defaults to the /tmp dir which gets emptied at each restart so all the data is lost... also add dfs.name.dir and dfs.data.dir properties into your hdfs-site.xml file to avoid the data loss. On Friday, July 20, 2012, Bejoy KS <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote: > Hi Yogesh > > Is your dfs.name.dir pointing to /tmp dir? If so try changing that to any other dir . The contents of /tmp may get wiped out on OS restarts. > Regards > Bejoy KS > > Sent from handheld, please excuse typos. > ________________________________ > From: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> > Date: Fri, 20 Jul 2012 06:20:02 +0000 > To: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> > ReplyTo: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]> > Subject: NameNode fails > Hello All :-), > > I am new to Hdfs. > > I have installed single node hdfs and started all nodes, every nodes gets started and work fine. > But when I shutdown my system or Restart it, then i try to run all nodes but Namenode doesn't start . > > to Start it i need to format the namenode and all data gets wash off :-(. > > Please help me and suggest me regarding this and how can I recover namenode from secondary name node on single node setup > > > Thanks & Regards > Yogesh Kumar Dhari > > Please do not print this email unless it is absolutely necessary. > > The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. > > www.wipro.com<http://www.wipro.com> -- Regards, Mohammad Tariq Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com +
yogesh.kumar13@... 2012-07-20, 07:21
-
Re: NameNode failsBejoy KS 2012-07-20, 07:26
Hi Yogesh
Please post in the error logs/messages if you find any. Regards Bejoy KS Sent from handheld, please excuse typos. -----Original Message----- From: <[EMAIL PROTECTED]> Date: Fri, 20 Jul 2012 07:21:24 To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> Subject: RE: NameNode fails Thanks Bejoy, Mohammad and Vignesh :-). I have done the suggested by you and made thses changes. and formatted the namenode and trying to start cluster. but now name node is not starting :-( hdfs-site.xml ********************************************************** <configuration> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.name.dir</name> <value>/HADOOP/hadoop-0.20.2/hadoop_name_dirr</value> </property> <property> <name>dfs.data.dir</name> <value>/HADOOP/hadoop-0.20.2/hadoop_data_dirr</value> </property> </configuration> ************************************************************** hdfs-core.xml ************************************************************** <configuration> <property> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/HADOOP/hadoop-0.20.2/hadoop_temp_dirr</value> <description>A base for other temporary directories.</description> </property> </configuration> ***************************************************************** Please suggest. Thanks & Regards Yogesh Kumar Dhari ________________________________ From: Mohammad Tariq [[EMAIL PROTECTED]] Sent: Friday, July 20, 2012 12:17 PM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: NameNode fails Hi yogesh, Do as suggested by bejoy and add the hadoop.tmp.dir in your core-site.xml file.it<http://file.it>'s value defaults to the /tmp dir which gets emptied at each restart so all the data is lost... also add dfs.name.dir and dfs.data.dir properties into your hdfs-site.xml file to avoid the data loss. On Friday, July 20, 2012, Bejoy KS <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote: > Hi Yogesh > > Is your dfs.name.dir pointing to /tmp dir? If so try changing that to any other dir . The contents of /tmp may get wiped out on OS restarts. > Regards > Bejoy KS > > Sent from handheld, please excuse typos. > ________________________________ > From: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> > Date: Fri, 20 Jul 2012 06:20:02 +0000 > To: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> > ReplyTo: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]> > Subject: NameNode fails > Hello All :-), > > I am new to Hdfs. > > I have installed single node hdfs and started all nodes, every nodes gets started and work fine. > But when I shutdown my system or Restart it, then i try to run all nodes but Namenode doesn't start . > > to Start it i need to format the namenode and all data gets wash off :-(. > > Please help me and suggest me regarding this and how can I recover namenode from secondary name node on single node setup > > > Thanks & Regards > Yogesh Kumar Dhari > > Please do not print this email unless it is absolutely necessary. > > The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. > > www.wipro.com<http://www.wipro.com> -- Regards, Mohammad Tariq Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com +
Bejoy KS 2012-07-20, 07:26
-
RE: NameNode failsyogesh.kumar13@... 2012-07-20, 08:29
Hi Bejoy,
Its done now, Error log was showing namenode is not formated, I closed all previous terminal and restarted it after formatting. its running now. Please suggest me that if in case it gets crashed then how do I recover it from Secondary Namenode. how should I proceed for that Thanks & regards Yogesh Kumar Dhari ________________________________ From: Bejoy KS [[EMAIL PROTECTED]] Sent: Friday, July 20, 2012 12:56 PM To: Yogesh Kumar (WT01 - Communication and Media); [EMAIL PROTECTED] Subject: Re: NameNode fails Hi Yogesh Please post in the error logs/messages if you find any. Regards Bejoy KS Sent from handheld, please excuse typos. ________________________________ From: <[EMAIL PROTECTED]> Date: Fri, 20 Jul 2012 07:21:24 +0000 To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> Subject: RE: NameNode fails Thanks Bejoy, Mohammad and Vignesh :-). I have done the suggested by you and made thses changes. and formatted the namenode and trying to start cluster. but now name node is not starting :-( hdfs-site.xml ********************************************************** <configuration> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.name.dir</name> <value>/HADOOP/hadoop-0.20.2/hadoop_name_dirr</value> </property> <property> <name>dfs.data.dir</name> <value>/HADOOP/hadoop-0.20.2/hadoop_data_dirr</value> </property> </configuration> ************************************************************** hdfs-core.xml ************************************************************** <configuration> <property> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/HADOOP/hadoop-0.20.2/hadoop_temp_dirr</value> <description>A base for other temporary directories.</description> </property> </configuration> ***************************************************************** Please suggest. Thanks & Regards Yogesh Kumar Dhari ________________________________ From: Mohammad Tariq [[EMAIL PROTECTED]] Sent: Friday, July 20, 2012 12:17 PM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: NameNode fails Hi yogesh, Do as suggested by bejoy and add the hadoop.tmp.dir in your core-site.xml file.it<http://file.it>'s value defaults to the /tmp dir which gets emptied at each restart so all the data is lost... also add dfs.name.dir and dfs.data.dir properties into your hdfs-site.xml file to avoid the data loss. On Friday, July 20, 2012, Bejoy KS <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote: > Hi Yogesh > > Is your dfs.name.dir pointing to /tmp dir? If so try changing that to any other dir . The contents of /tmp may get wiped out on OS restarts. > Regards > Bejoy KS > > Sent from handheld, please excuse typos. > ________________________________ > From: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> > Date: Fri, 20 Jul 2012 06:20:02 +0000 > To: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> > ReplyTo: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]> > Subject: NameNode fails > Hello All :-), > > I am new to Hdfs. > > I have installed single node hdfs and started all nodes, every nodes gets started and work fine. > But when I shutdown my system or Restart it, then i try to run all nodes but Namenode doesn't start . > > to Start it i need to format the namenode and all data gets wash off :-(. > > Please help me and suggest me regarding this and how can I recover namenode from secondary name node on single node setup > > > Thanks & Regards > Yogesh Kumar Dhari > > Please do not print this email unless it is absolutely necessary. > > The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. Regards, Mohammad Tariq Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com +
yogesh.kumar13@... 2012-07-20, 08:29
-
Re: NameNode failsMohammad Tariq 2012-07-20, 09:47
Hi yogesh,
First of all, we should always keep it mind that Secondary Namenode is not a backup for the Namenode. By its name, it gives a sense that its a backup for the Namenode, but in reality its not. Namenode stores data in 2 files : 1- fsimage - snapshot of the filesystem when namenode started 2- Edit logs - contain the sequence of changes made to the filesystem after namenode started. The sole purpose of Secondary Namenode is to have a checkpoint in HDFS and it acts as a helper of the Namenode. It basically : 1- gets the edit logs from the namenode in regular intervals and applies to fsimage 2- once it has new fsimage, it copies back to namenode Namenode uses this fsimage for the next restart. Regards, Mohammad Tariq On Fri, Jul 20, 2012 at 1:59 PM, <[EMAIL PROTECTED]> wrote: > Hi Bejoy, > > Its done now, Error log was showing namenode is not formated, > I closed all previous terminal and restarted it after formatting. > > its running now. > > Please suggest me that if in case it gets crashed then how do I recover it > from Secondary Namenode. how should I proceed for that > > > Thanks & regards > Yogesh Kumar Dhari > ________________________________ > From: Bejoy KS [[EMAIL PROTECTED]] > Sent: Friday, July 20, 2012 12:56 PM > To: Yogesh Kumar (WT01 - Communication and Media); > [EMAIL PROTECTED] > Subject: Re: NameNode fails > > Hi Yogesh > > Please post in the error logs/messages if you find any. > > Regards > Bejoy KS > > Sent from handheld, please excuse typos. > ________________________________ > From: <[EMAIL PROTECTED]> > Date: Fri, 20 Jul 2012 07:21:24 +0000 > To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> > Subject: RE: NameNode fails > > Thanks Bejoy, Mohammad and Vignesh :-). > > I have done the suggested by you and made thses changes. and formatted the > namenode and trying to start cluster. > but now name node is not starting :-( > > > hdfs-site.xml > > ********************************************************** > <configuration> > <property> > <name>dfs.replication</name> > <value>1</value> > </property> > > <property> > <name>dfs.name.dir</name> > <value>/HADOOP/hadoop-0.20.2/hadoop_name_dirr</value> > </property> > > <property> > <name>dfs.data.dir</name> > <value>/HADOOP/hadoop-0.20.2/hadoop_data_dirr</value> > </property> > > </configuration> > > ************************************************************** > > > > hdfs-core.xml > > ************************************************************** > > <configuration> > <property> > <name>fs.default.name</name> > <value>hdfs://localhost:9000</value> > </property> > > <property> > <name>hadoop.tmp.dir</name> > <value>/HADOOP/hadoop-0.20.2/hadoop_temp_dirr</value> > <description>A base for other temporary directories.</description> > </property> > > </configuration> > > ***************************************************************** > > Please suggest. > > Thanks & Regards > Yogesh Kumar Dhari > > > ________________________________ > From: Mohammad Tariq [[EMAIL PROTECTED]] > Sent: Friday, July 20, 2012 12:17 PM > To: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: Re: NameNode fails > > Hi yogesh, > Do as suggested by bejoy and add the hadoop.tmp.dir in your > core-site.xml file.it's value defaults to the /tmp dir which gets emptied at > each restart so all the data is lost... also add dfs.name.dir and > dfs.data.dir properties into your hdfs-site.xml file to avoid the data loss. > > On Friday, July 20, 2012, Bejoy KS <[EMAIL PROTECTED]> wrote: >> Hi Yogesh >> >> Is your dfs.name.dir pointing to /tmp dir? If so try changing that to any >> other dir . The contents of /tmp may get wiped out on OS restarts. >> Regards >> Bejoy KS >> >> Sent from handheld, please excuse typos. >> ________________________________ >> From: <[EMAIL PROTECTED]> >> Date: Fri, 20 Jul 2012 06:20:02 +0000 +
Mohammad Tariq 2012-07-20, 09:47
-
RE: NameNode failsyogesh.kumar13@... 2012-07-20, 10:28
Thanks Mohammad :-),
I just read this concept of secondary NameNode. Thank you for your reply. Mohammad I am not finding the way to implement would you please explain me regarding to recover namenode, Iam getting confuse. Thanks & Regards Yogesh Kumar Dhari ________________________________________ From: Mohammad Tariq [[EMAIL PROTECTED]] Sent: Friday, July 20, 2012 3:17 PM To: [EMAIL PROTECTED] Subject: Re: NameNode fails Hi yogesh, First of all, we should always keep it mind that Secondary Namenode is not a backup for the Namenode. By its name, it gives a sense that its a backup for the Namenode, but in reality its not. Namenode stores data in 2 files : 1- fsimage - snapshot of the filesystem when namenode started 2- Edit logs - contain the sequence of changes made to the filesystem after namenode started. The sole purpose of Secondary Namenode is to have a checkpoint in HDFS and it acts as a helper of the Namenode. It basically : 1- gets the edit logs from the namenode in regular intervals and applies to fsimage 2- once it has new fsimage, it copies back to namenode Namenode uses this fsimage for the next restart. Regards, Mohammad Tariq On Fri, Jul 20, 2012 at 1:59 PM, <[EMAIL PROTECTED]> wrote: > Hi Bejoy, > > Its done now, Error log was showing namenode is not formated, > I closed all previous terminal and restarted it after formatting. > > its running now. > > Please suggest me that if in case it gets crashed then how do I recover it > from Secondary Namenode. how should I proceed for that > > > Thanks & regards > Yogesh Kumar Dhari > ________________________________ > From: Bejoy KS [[EMAIL PROTECTED]] > Sent: Friday, July 20, 2012 12:56 PM > To: Yogesh Kumar (WT01 - Communication and Media); > [EMAIL PROTECTED] > Subject: Re: NameNode fails > > Hi Yogesh > > Please post in the error logs/messages if you find any. > > Regards > Bejoy KS > > Sent from handheld, please excuse typos. > ________________________________ > From: <[EMAIL PROTECTED]> > Date: Fri, 20 Jul 2012 07:21:24 +0000 > To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> > Subject: RE: NameNode fails > > Thanks Bejoy, Mohammad and Vignesh :-). > > I have done the suggested by you and made thses changes. and formatted the > namenode and trying to start cluster. > but now name node is not starting :-( > > > hdfs-site.xml > > ********************************************************** > <configuration> > <property> > <name>dfs.replication</name> > <value>1</value> > </property> > > <property> > <name>dfs.name.dir</name> > <value>/HADOOP/hadoop-0.20.2/hadoop_name_dirr</value> > </property> > > <property> > <name>dfs.data.dir</name> > <value>/HADOOP/hadoop-0.20.2/hadoop_data_dirr</value> > </property> > > </configuration> > > ************************************************************** > > > > hdfs-core.xml > > ************************************************************** > > <configuration> > <property> > <name>fs.default.name</name> > <value>hdfs://localhost:9000</value> > </property> > > <property> > <name>hadoop.tmp.dir</name> > <value>/HADOOP/hadoop-0.20.2/hadoop_temp_dirr</value> > <description>A base for other temporary directories.</description> > </property> > > </configuration> > > ***************************************************************** > > Please suggest. > > Thanks & Regards > Yogesh Kumar Dhari > > > ________________________________ > From: Mohammad Tariq [[EMAIL PROTECTED]] > Sent: Friday, July 20, 2012 12:17 PM > To: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: Re: NameNode fails > > Hi yogesh, > Do as suggested by bejoy and add the hadoop.tmp.dir in your > core-site.xml file.it's value defaults to the /tmp dir which gets emptied at > each restart so all the data is lost... also add dfs.name.dir and > dfs.data.dir properties into your hdfs-site.xml file to avoid the data loss. Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com +
yogesh.kumar13@... 2012-07-20, 10:28
-
Re: NameNode failsBejoy Ks 2012-07-20, 10:44
Hi Yogesh
Just treat SecondaryName Node as a check pointing node. Name recovery is mostly done by writing the fsimage and edit log to a remote nfs mount other than the local fs. So that if the copy on local disk gets corrupted or lost on a disk failure or machine failure the one from remote mount can be used. You can read more here http://wiki.apache.org/hadoop/NameNodeFailover Regards Bejoy KS On Fri, Jul 20, 2012 at 3:58 PM, <[EMAIL PROTECTED]> wrote: > Thanks Mohammad :-), > > I just read this concept of secondary NameNode. Thank you for your reply. > Mohammad I am not finding the way to implement would you please explain me regarding to recover namenode, Iam getting confuse. > > Thanks & Regards > Yogesh Kumar Dhari > > ________________________________________ > From: Mohammad Tariq [[EMAIL PROTECTED]] > Sent: Friday, July 20, 2012 3:17 PM > To: [EMAIL PROTECTED] > Subject: Re: NameNode fails > > Hi yogesh, > > First of all, we should always keep it mind that Secondary > Namenode is not a backup for the Namenode. By its name, it gives a > sense that its a backup for the Namenode, but in reality its not. > Namenode stores data in 2 files : > 1- fsimage - snapshot of the filesystem when namenode started > 2- Edit logs - contain the sequence of changes made to the filesystem > after namenode started. > > The sole purpose of Secondary Namenode is to have a checkpoint in HDFS > and it acts as a helper of the Namenode. It basically : > 1- gets the edit logs from the namenode in regular intervals and > applies to fsimage > 2- once it has new fsimage, it copies back to namenode > > Namenode uses this fsimage for the next restart. > > Regards, > Mohammad Tariq > > > On Fri, Jul 20, 2012 at 1:59 PM, <[EMAIL PROTECTED]> wrote: >> Hi Bejoy, >> >> Its done now, Error log was showing namenode is not formated, >> I closed all previous terminal and restarted it after formatting. >> >> its running now. >> >> Please suggest me that if in case it gets crashed then how do I recover it >> from Secondary Namenode. how should I proceed for that >> >> >> Thanks & regards >> Yogesh Kumar Dhari >> ________________________________ >> From: Bejoy KS [[EMAIL PROTECTED]] >> Sent: Friday, July 20, 2012 12:56 PM >> To: Yogesh Kumar (WT01 - Communication and Media); >> [EMAIL PROTECTED] >> Subject: Re: NameNode fails >> >> Hi Yogesh >> >> Please post in the error logs/messages if you find any. >> >> Regards >> Bejoy KS >> >> Sent from handheld, please excuse typos. >> ________________________________ >> From: <[EMAIL PROTECTED]> >> Date: Fri, 20 Jul 2012 07:21:24 +0000 >> To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> >> Subject: RE: NameNode fails >> >> Thanks Bejoy, Mohammad and Vignesh :-). >> >> I have done the suggested by you and made thses changes. and formatted the >> namenode and trying to start cluster. >> but now name node is not starting :-( >> >> >> hdfs-site.xml >> >> ********************************************************** >> <configuration> >> <property> >> <name>dfs.replication</name> >> <value>1</value> >> </property> >> >> <property> >> <name>dfs.name.dir</name> >> <value>/HADOOP/hadoop-0.20.2/hadoop_name_dirr</value> >> </property> >> >> <property> >> <name>dfs.data.dir</name> >> <value>/HADOOP/hadoop-0.20.2/hadoop_data_dirr</value> >> </property> >> >> </configuration> >> >> ************************************************************** >> >> >> >> hdfs-core.xml >> >> ************************************************************** >> >> <configuration> >> <property> >> <name>fs.default.name</name> >> <value>hdfs://localhost:9000</value> >> </property> >> >> <property> >> <name>hadoop.tmp.dir</name> >> <value>/HADOOP/hadoop-0.20.2/hadoop_temp_dirr</value> >> <description>A base for other temporary directories.</description> >> </property> >> >> </configuration> +
Bejoy Ks 2012-07-20, 10:44
-
Re: NameNode failsMohammad Tariq 2012-07-20, 11:39
Hi Yogesh,
Follow the link specified by Bejoy. It shows all the necessary steps. Regards, Mohammad Tariq On Fri, Jul 20, 2012 at 4:14 PM, Bejoy Ks <[EMAIL PROTECTED]> wrote: > Hi Yogesh > > Just treat SecondaryName Node as a check pointing node. Name recovery > is mostly done by writing the fsimage and edit log to a remote nfs > mount other than the local fs. So that if the copy on local disk gets > corrupted or lost on a disk failure or machine failure the one from > remote mount can be used. > > You can read more here > http://wiki.apache.org/hadoop/NameNodeFailover > > Regards > Bejoy KS > > > On Fri, Jul 20, 2012 at 3:58 PM, <[EMAIL PROTECTED]> wrote: >> Thanks Mohammad :-), >> >> I just read this concept of secondary NameNode. Thank you for your reply. >> Mohammad I am not finding the way to implement would you please explain me regarding to recover namenode, Iam getting confuse. >> >> Thanks & Regards >> Yogesh Kumar Dhari >> >> ________________________________________ >> From: Mohammad Tariq [[EMAIL PROTECTED]] >> Sent: Friday, July 20, 2012 3:17 PM >> To: [EMAIL PROTECTED] >> Subject: Re: NameNode fails >> >> Hi yogesh, >> >> First of all, we should always keep it mind that Secondary >> Namenode is not a backup for the Namenode. By its name, it gives a >> sense that its a backup for the Namenode, but in reality its not. >> Namenode stores data in 2 files : >> 1- fsimage - snapshot of the filesystem when namenode started >> 2- Edit logs - contain the sequence of changes made to the filesystem >> after namenode started. >> >> The sole purpose of Secondary Namenode is to have a checkpoint in HDFS >> and it acts as a helper of the Namenode. It basically : >> 1- gets the edit logs from the namenode in regular intervals and >> applies to fsimage >> 2- once it has new fsimage, it copies back to namenode >> >> Namenode uses this fsimage for the next restart. >> >> Regards, >> Mohammad Tariq >> >> >> On Fri, Jul 20, 2012 at 1:59 PM, <[EMAIL PROTECTED]> wrote: >>> Hi Bejoy, >>> >>> Its done now, Error log was showing namenode is not formated, >>> I closed all previous terminal and restarted it after formatting. >>> >>> its running now. >>> >>> Please suggest me that if in case it gets crashed then how do I recover it >>> from Secondary Namenode. how should I proceed for that >>> >>> >>> Thanks & regards >>> Yogesh Kumar Dhari >>> ________________________________ >>> From: Bejoy KS [[EMAIL PROTECTED]] >>> Sent: Friday, July 20, 2012 12:56 PM >>> To: Yogesh Kumar (WT01 - Communication and Media); >>> [EMAIL PROTECTED] >>> Subject: Re: NameNode fails >>> >>> Hi Yogesh >>> >>> Please post in the error logs/messages if you find any. >>> >>> Regards >>> Bejoy KS >>> >>> Sent from handheld, please excuse typos. >>> ________________________________ >>> From: <[EMAIL PROTECTED]> >>> Date: Fri, 20 Jul 2012 07:21:24 +0000 >>> To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> >>> Subject: RE: NameNode fails >>> >>> Thanks Bejoy, Mohammad and Vignesh :-). >>> >>> I have done the suggested by you and made thses changes. and formatted the >>> namenode and trying to start cluster. >>> but now name node is not starting :-( >>> >>> >>> hdfs-site.xml >>> >>> ********************************************************** >>> <configuration> >>> <property> >>> <name>dfs.replication</name> >>> <value>1</value> >>> </property> >>> >>> <property> >>> <name>dfs.name.dir</name> >>> <value>/HADOOP/hadoop-0.20.2/hadoop_name_dirr</value> >>> </property> >>> >>> <property> >>> <name>dfs.data.dir</name> >>> <value>/HADOOP/hadoop-0.20.2/hadoop_data_dirr</value> >>> </property> >>> >>> </configuration> >>> >>> ************************************************************** >>> >>> >>> >>> hdfs-core.xml >>> >>> ************************************************************** >>> >>> <configuration> >>> <property> +
Mohammad Tariq 2012-07-20, 11:39
-
Re: NameNode failsVIGNESH PRAJAPATI 2012-07-20, 06:50
And avoid formatting namenode while your cluster is running for safety of
data in hdfs. On Fri, Jul 20, 2012 at 2:47 AM, Mohammad Tariq <[EMAIL PROTECTED]> wrote: > Hi yogesh, > Do as suggested by bejoy and add the hadoop.tmp.dir in your > core-site.xml file.it's value defaults to the /tmp dir which gets emptied > at each restart so all the data is lost... also add dfs.name.dir and > dfs.data.dir properties into your hdfs-site.xml file to avoid the data loss. > > > On Friday, July 20, 2012, Bejoy KS <[EMAIL PROTECTED]> wrote: > > Hi Yogesh > > > > Is your dfs.name.dir pointing to /tmp dir? If so try changing that to > any other dir . The contents of /tmp may get wiped out on OS restarts. > > Regards > > Bejoy KS > > > > Sent from handheld, please excuse typos. > > ________________________________ > > From: <[EMAIL PROTECTED]> > > Date: Fri, 20 Jul 2012 06:20:02 +0000 > > To: <[EMAIL PROTECTED]> > > ReplyTo: [EMAIL PROTECTED] > > Subject: NameNode fails > > Hello All :-), > > > > I am new to Hdfs. > > > > I have installed single node hdfs and started all nodes, every nodes > gets started and work fine. > > But when I shutdown my system or Restart it, then i try to run all nodes > but Namenode doesn't start . > > > > to Start it i need to format the namenode and all data gets wash off :-(. > > > > Please help me and suggest me regarding this and how can I recover > namenode from secondary name node on single node setup > > > > > > Thanks & Regards > > Yogesh Kumar Dhari > > > > Please do not print this email unless it is absolutely necessary. > > > > The information contained in this electronic message and any attachments > to this message are intended for the exclusive use of the addressee(s) and > may contain proprietary, confidential or privileged information. If you are > not the intended recipient, you should not disseminate, distribute or copy > this e-mail. Please notify the sender immediately and destroy all copies of > this message and any attachments. > > > > WARNING: Computer viruses can be transmitted via email. The recipient > should check this email and any attachments for the presence of viruses. > The company accepts no liability for any damage caused by any virus > transmitted by this email. > > > > www.wipro.com > > -- > Regards, > Mohammad Tariq > > -- *Vignesh Prajapati* Tel: 9427415949 | [EMAIL PROTECTED] | www.vipras.com.co.in MYTK [image: Facebook] <https://www.facebook.com/vigs143> [image: Twitter]<http://twitter.com/#%21/vigs143> [image: LinkedIn] <http://www.linkedin.com/pub/vignesh-prajapati/37/756/46a> [image: about.me] <http://www.way4fun.tk><http://r1.wisestamp.com/r/landing?promo=7&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_7> +
VIGNESH PRAJAPATI 2012-07-20, 06:50
|