Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> formatting hdfs without user interaction


Copy link to this message
-
Re: formatting hdfs without user interaction
The reason you are getting multiple prompts is that you have multiple dir's defined in the dfs.name.dir.

A simple expect script would take care of this.

#!/usr/bin/expect -f
spawn /bin/hadoop  namenode -format
expect "Re-format filesystem in"
send Y\n
expect "Re-format filesystem in"
send Y\n
interact
You might have to put the above in a loop based on how many dirs are defined in the dfs.name.dir

--
Arpit
[EMAIL PROTECTED]
On Sep 22, 2011, at 2:59 PM, <[EMAIL PROTECTED]> wrote:

> Hi Raj and Arpit,
>
> I think the fact that there are 2 confirmations is causing it not to work:
>
> echo "Y" | hadoop namenode -format
> 11/09/22 14:57:10 INFO namenode.NameNode: STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting NameNode
> STARTUP_MSG:   host = hdm1/172.28.9.250
> STARTUP_MSG:   args = [-format]
> STARTUP_MSG:   version = 0.20.204.0
> STARTUP_MSG:   build = git://hrt8n35.cc1.ygridcore.net/ on branch
> branch-0.20-security-204 -r 65e258bf0813ac2b15bb4c954660eaf9e8fba141;
> compiled by 'hortonow' on Thu Aug 25 23:25:52 UTC 2011
> ************************************************************/
> Re-format filesystem in /data/local/hdfs/name ? (Y or N) Re-format
> filesystem in /data/remote/hdfs/name ? (Y or N) Format aborted in
> /data/remote/hdfs/name
> 11/09/22 14:57:10 INFO namenode.NameNode: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at hdm1/172.28.9.250
> ************************************************************/
> Heap
> par new generation   total 2831168K, used 150996K [0x00002aaaae210000,
> 0x00002aab6e210000, 0x00002aab6e210000)
>  eden space 2516608K,   6% used [0x00002aaaae210000, 0x00002aaab75852d0,
> 0x00002aab47bb0000)
>  from space 314560K,   0% used [0x00002aab47bb0000, 0x00002aab47bb0000,
> 0x00002aab5aee0000)
>  to   space 314560K,   0% used [0x00002aab5aee0000, 0x00002aab5aee0000,
> 0x00002aab6e210000)
> concurrent mark-sweep generation total 30408704K, used 0K
> [0x00002aab6e210000, 0x00002ab2ae210000, 0x00002ab2ae210000)
> concurrent-mark-sweep perm gen total 21248K, used 7333K
> [0x00002ab2ae210000, 0x00002ab2af6d0000, 0x00002ab2b3610000)
>
>
> Cheers,
> Ivan
>
> On 9/22/11 2:52 PM, "Raj V" <[EMAIL PROTECTED]> wrote:
>
>> Ian
>> "echo 'Y' | hadoop namenode -format"
>>
>> should work.,
>>
>> Raj
>>
>>
>>> ________________________________
>>> From: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>>> To: [EMAIL PROTECTED]
>>> Sent: Thursday, September 22, 2011 2:43 PM
>>> Subject: formatting hdfs without user interaction
>>>
>>> Hello,
>>>
>>> I am trying to automate formatting an HDFS volume.  Is there any way to
>>> do this without the interaction (and using expect)?
>>>
>>> Cheers,
>>> Ivan
>>>
>>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB