Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - formatting hdfs without user interaction


Copy link to this message
-
Re: formatting hdfs without user interaction
Arpit Gupta 2011-09-22, 22:17
The reason you are getting multiple prompts is that you have multiple dir's defined in the dfs.name.dir.

A simple expect script would take care of this.

#!/usr/bin/expect -f
spawn /bin/hadoop  namenode -format
expect "Re-format filesystem in"
send Y\n
expect "Re-format filesystem in"
send Y\n
interact
You might have to put the above in a loop based on how many dirs are defined in the dfs.name.dir

--
Arpit
[EMAIL PROTECTED]
On Sep 22, 2011, at 2:59 PM, <[EMAIL PROTECTED]> wrote:

> Hi Raj and Arpit,
>
> I think the fact that there are 2 confirmations is causing it not to work:
>
> echo "Y" | hadoop namenode -format
> 11/09/22 14:57:10 INFO namenode.NameNode: STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting NameNode
> STARTUP_MSG:   host = hdm1/172.28.9.250
> STARTUP_MSG:   args = [-format]
> STARTUP_MSG:   version = 0.20.204.0
> STARTUP_MSG:   build = git://hrt8n35.cc1.ygridcore.net/ on branch
> branch-0.20-security-204 -r 65e258bf0813ac2b15bb4c954660eaf9e8fba141;
> compiled by 'hortonow' on Thu Aug 25 23:25:52 UTC 2011
> ************************************************************/
> Re-format filesystem in /data/local/hdfs/name ? (Y or N) Re-format
> filesystem in /data/remote/hdfs/name ? (Y or N) Format aborted in
> /data/remote/hdfs/name
> 11/09/22 14:57:10 INFO namenode.NameNode: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at hdm1/172.28.9.250
> ************************************************************/
> Heap
> par new generation   total 2831168K, used 150996K [0x00002aaaae210000,
> 0x00002aab6e210000, 0x00002aab6e210000)
>  eden space 2516608K,   6% used [0x00002aaaae210000, 0x00002aaab75852d0,
> 0x00002aab47bb0000)
>  from space 314560K,   0% used [0x00002aab47bb0000, 0x00002aab47bb0000,
> 0x00002aab5aee0000)
>  to   space 314560K,   0% used [0x00002aab5aee0000, 0x00002aab5aee0000,
> 0x00002aab6e210000)
> concurrent mark-sweep generation total 30408704K, used 0K
> [0x00002aab6e210000, 0x00002ab2ae210000, 0x00002ab2ae210000)
> concurrent-mark-sweep perm gen total 21248K, used 7333K
> [0x00002ab2ae210000, 0x00002ab2af6d0000, 0x00002ab2b3610000)
>
>
> Cheers,
> Ivan
>
> On 9/22/11 2:52 PM, "Raj V" <[EMAIL PROTECTED]> wrote:
>
>> Ian
>> "echo 'Y' | hadoop namenode -format"
>>
>> should work.,
>>
>> Raj
>>
>>
>>> ________________________________
>>> From: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>>> To: [EMAIL PROTECTED]
>>> Sent: Thursday, September 22, 2011 2:43 PM
>>> Subject: formatting hdfs without user interaction
>>>
>>> Hello,
>>>
>>> I am trying to automate formatting an HDFS volume.  Is there any way to
>>> do this without the interaction (and using expect)?
>>>
>>> Cheers,
>>> Ivan
>>>
>>>
>