Hello!

I'm trying to install Hadoop 1.1.2.21 on CentOS 6.3.

I've configured dfs.name.dir in /etc/hadoop/conf/hdfs-site.xml file

<name>dfs.name.dir</name>
<value>/mnt/ext/hadoop/hdfs/namenode</value>

But when I run "hadoop namenode -format" command, it formats
/tmp/hadoop-hadoop/dfs/name directory instead. It seems like default
directory.

What am I missing?

have you set the variable HADOOP_HOME ?

i just tested it and its working fine on my hdfs
On Wed, Feb 6, 2013 at 8:37 PM, Andrey V. Romanchev <
[EMAIL PROTECTED]> wrote:

> Hello!
>
> I'm trying to install Hadoop 1.1.2.21 on CentOS 6.3.
>
> I've configured dfs.name.dir in /etc/hadoop/conf/hdfs-site.xml file
>
> <name>dfs.name.dir</name>
> <value>/mnt/ext/hadoop/hdfs/namenode</value>
>
> But when I run "hadoop namenode -format" command, it formats
> /tmp/hadoop-hadoop/dfs/name directory instead. It seems like default
> directory.
>
> What am I missing?
>

--
Nitin Pawar

you can try to use the new parameter "dfs.namenode.name.dir" to
specify the directory.

2013/2/6, Andrey V. Romanchev <[EMAIL PROTECTED]>:
> Hello!
>
> I'm trying to install Hadoop 1.1.2.21 on CentOS 6.3.
>
> I've configured dfs.name.dir in /etc/hadoop/conf/hdfs-site.xml file
>
> <name>dfs.name.dir</name>
> <value>/mnt/ext/hadoop/hdfs/namenode</value>
>
> But when I run "hadoop namenode -format" command, it formats
> /tmp/hadoop-hadoop/dfs/name directory instead. It seems like default
> directory.
>
> What am I missing?
>

No.
Is it necessary? What directory it should point?

Just tested
export HADOOP_HOME=/mnt/ext/hadoop
the same problem, /tmp/hadoop-hadoop used
2013/2/6 Nitin Pawar <[EMAIL PROTECTED]>

> have you set the variable HADOOP_HOME ?
>
> i just tested it and its working fine on my hdfs
>
>
> On Wed, Feb 6, 2013 at 8:37 PM, Andrey V. Romanchev <
> [EMAIL PROTECTED]> wrote:
>
>> Hello!
>>
>> I'm trying to install Hadoop 1.1.2.21 on CentOS 6.3.
>>
>> I've configured dfs.name.dir in /etc/hadoop/conf/hdfs-site.xml file
>>
>> <name>dfs.name.dir</name>
>> <value>/mnt/ext/hadoop/hdfs/namenode</value>
>>
>> But when I run "hadoop namenode -format" command, it formats
>> /tmp/hadoop-hadoop/dfs/name directory instead. It seems like default
>> directory.
>>
>> What am I missing?
>>
>
>
>
> --
> Nitin Pawar
>

Hi, Andrey

Did you try <final>true</final> in your config ? It should be like this:

<name>dfs.name.dir</name>
<value>/mnt/ext/hadoop/hdfs/namenode</value>
<final>true</final>

Cheers
On Wed, Feb 6, 2013 at 4:07 PM, Andrey V. Romanchev
<[EMAIL PROTECTED]> wrote:
>
> Hello!
>
> I'm trying to install Hadoop 1.1.2.21 on CentOS 6.3.
>
> I've configured dfs.name.dir in /etc/hadoop/conf/hdfs-site.xml file
>
> <name>dfs.name.dir</name>
> <value>/mnt/ext/hadoop/hdfs/namenode</value>
>
> But when I run "hadoop namenode -format" command, it formats /tmp/hadoop-hadoop/dfs/name directory instead. It seems like default directory.
>
> What am I missing?

Added to hdfs-site.xml
  <property>
    <name>dfs.namenode.name.dir</name>
    <value>/mnt/ext/hadoop/hdfs/namenode</value>
    <final>true</final>
  </property>

The same problem exists :(
2013/2/6 Yanbo Liang <[EMAIL PROTECTED]>

> you can try to use the new parameter "dfs.namenode.name.dir" to
> specify the directory.
>
> 2013/2/6, Andrey V. Romanchev <[EMAIL PROTECTED]>:
> > Hello!
> >
> > I'm trying to install Hadoop 1.1.2.21 on CentOS 6.3.
> >
> > I've configured dfs.name.dir in /etc/hadoop/conf/hdfs-site.xml file
> >
> > <name>dfs.name.dir</name>
> > <value>/mnt/ext/hadoop/hdfs/namenode</value>
> >
> > But when I run "hadoop namenode -format" command, it formats
> > /tmp/hadoop-hadoop/dfs/name directory instead. It seems like default
> > directory.
> >
> > What am I missing?
> >
>

Hello Samir
Yes. Here is full section from my hdfs-site.xml

  <property>
    <name>dfs.name.dir</name>
    <value>/mnt/ext/hadoop/hdfs/namenode</value>
    <description>
        Determines where on the local filesystem the DFS name node
        should store the name table.  If this is a comma-delimited list
        of directories then the name table is replicated in all of the
        directories, for redundancy.
    </description>
    <final>true</final>
  </property>
2013/2/6 Samir Ahmic <[EMAIL PROTECTED]>

> Hi, Andrey
>
> Did you try <final>true</final> in your config ? It should be like this:
>
> <name>dfs.name.dir</name>
> <value>/mnt/ext/hadoop/hdfs/namenode</value>
> <final>true</final>
>
> Cheers
>
>
> On Wed, Feb 6, 2013 at 4:07 PM, Andrey V. Romanchev
> <[EMAIL PROTECTED]> wrote:
> >
> > Hello!
> >
> > I'm trying to install Hadoop 1.1.2.21 on CentOS 6.3.
> >
> > I've configured dfs.name.dir in /etc/hadoop/conf/hdfs-site.xml file
> >
> > <name>dfs.name.dir</name>
> > <value>/mnt/ext/hadoop/hdfs/namenode</value>
> >
> > But when I run "hadoop namenode -format" command, it formats
> /tmp/hadoop-hadoop/dfs/name directory instead. It seems like default
> directory.
> >
> > What am I missing?
>

HADOOP_HOME is the directory where you installed hadoop

a directory above your conf directory (in other words parent directory of
conf)
On Wed, Feb 6, 2013 at 8:55 PM, Andrey V. Romanchev <
[EMAIL PROTECTED]> wrote:

> Hello Samir
> Yes. Here is full section from my hdfs-site.xml
>
>   <property>
>
>     <name>dfs.name.dir</name>
>     <value>/mnt/ext/hadoop/hdfs/namenode</value>
>     <description>
>         Determines where on the local filesystem the DFS name node
>         should store the name table.  If this is a comma-delimited list
>         of directories then the name table is replicated in all of the
>         directories, for redundancy.
>     </description>
>     <final>true</final>
>   </property>
>
>
> 2013/2/6 Samir Ahmic <[EMAIL PROTECTED]>
>
>> Hi, Andrey
>>
>> Did you try <final>true</final> in your config ? It should be like this:
>>
>> <name>dfs.name.dir</name>
>> <value>/mnt/ext/hadoop/hdfs/namenode</value>
>> <final>true</final>
>>
>> Cheers
>>
>>
>> On Wed, Feb 6, 2013 at 4:07 PM, Andrey V. Romanchev
>> <[EMAIL PROTECTED]> wrote:
>> >
>> > Hello!
>> >
>> > I'm trying to install Hadoop 1.1.2.21 on CentOS 6.3.
>> >
>> > I've configured dfs.name.dir in /etc/hadoop/conf/hdfs-site.xml file
>> >
>> > <name>dfs.name.dir</name>
>> > <value>/mnt/ext/hadoop/hdfs/namenode</value>
>> >
>> > But when I run "hadoop namenode -format" command, it formats
>> /tmp/hadoop-hadoop/dfs/name directory instead. It seems like default
>> directory.
>> >
>> > What am I missing?
>>
>
>
--
Nitin Pawar

Done.
$ echo $HADOOP_HOME
/usr/lib/hadoop/
$ ll /usr/lib/hadoop/conf
lrwxrwxrwx. 1 root root 16 Фев  6 16:09 /usr/lib/hadoop/conf ->
/etc/hadoop/conf

Sadly, it did not help.
2013/2/6 Nitin Pawar <[EMAIL PROTECTED]>

> HADOOP_HOME is the directory where you installed hadoop
>
> a directory above your conf directory (in other words parent directory of
> conf)
>
>
> On Wed, Feb 6, 2013 at 8:55 PM, Andrey V. Romanchev <
> [EMAIL PROTECTED]> wrote:
>
>> Hello Samir
>> Yes. Here is full section from my hdfs-site.xml
>>
>>   <property>
>>
>>     <name>dfs.name.dir</name>
>>     <value>/mnt/ext/hadoop/hdfs/namenode</value>
>>      <description>
>>         Determines where on the local filesystem the DFS name node
>>         should store the name table.  If this is a comma-delimited list
>>         of directories then the name table is replicated in all of the
>>         directories, for redundancy.
>>     </description>
>>     <final>true</final>
>>   </property>
>>
>>
>> 2013/2/6 Samir Ahmic <[EMAIL PROTECTED]>
>>
>>> Hi, Andrey
>>>
>>> Did you try <final>true</final> in your config ? It should be like this:
>>>
>>> <name>dfs.name.dir</name>
>>> <value>/mnt/ext/hadoop/hdfs/namenode</value>
>>> <final>true</final>
>>>
>>> Cheers
>>>
>>>
>>> On Wed, Feb 6, 2013 at 4:07 PM, Andrey V. Romanchev
>>> <[EMAIL PROTECTED]> wrote:
>>> >
>>> > Hello!
>>> >
>>> > I'm trying to install Hadoop 1.1.2.21 on CentOS 6.3.
>>> >
>>> > I've configured dfs.name.dir in /etc/hadoop/conf/hdfs-site.xml file
>>> >
>>> > <name>dfs.name.dir</name>
>>> > <value>/mnt/ext/hadoop/hdfs/namenode</value>
>>> >
>>> > But when I run "hadoop namenode -format" command, it formats
>>> /tmp/hadoop-hadoop/dfs/name directory instead. It seems like default
>>> directory.
>>> >
>>> > What am I missing?
>>>
>>
>>
>
>
> --
> Nitin Pawar
>

Hi,
Can you try to execute format command like this:

hadoop --config  /etc/hadoop/conf namenode -format

Look like hadoop script is unable to pickup your config dir.

Cheers
On Wed, Feb 6, 2013 at 4:32 PM, Andrey V. Romanchev
<[EMAIL PROTECTED]> wrote:
> Done.
> $ echo $HADOOP_HOME
> /usr/lib/hadoop/
> $ ll /usr/lib/hadoop/conf
> lrwxrwxrwx. 1 root root 16 Фев  6 16:09 /usr/lib/hadoop/conf ->
> /etc/hadoop/conf
>
> Sadly, it did not help.
>
>
> 2013/2/6 Nitin Pawar <[EMAIL PROTECTED]>
>>
>> HADOOP_HOME is the directory where you installed hadoop
>>
>> a directory above your conf directory (in other words parent directory of
>> conf)
>>
>>
>> On Wed, Feb 6, 2013 at 8:55 PM, Andrey V. Romanchev
>> <[EMAIL PROTECTED]> wrote:
>>>
>>> Hello Samir
>>> Yes. Here is full section from my hdfs-site.xml
>>>
>>>   <property>
>>>
>>>     <name>dfs.name.dir</name>
>>>     <value>/mnt/ext/hadoop/hdfs/namenode</value>
>>>     <description>
>>>         Determines where on the local filesystem the DFS name node
>>>         should store the name table.  If this is a comma-delimited list
>>>         of directories then the name table is replicated in all of the
>>>         directories, for redundancy.
>>>     </description>
>>>     <final>true</final>
>>>   </property>
>>>
>>>
>>> 2013/2/6 Samir Ahmic <[EMAIL PROTECTED]>
>>>>
>>>> Hi, Andrey
>>>>
>>>> Did you try <final>true</final> in your config ? It should be like this:
>>>>
>>>> <name>dfs.name.dir</name>
>>>> <value>/mnt/ext/hadoop/hdfs/namenode</value>
>>>> <final>true</final>
>>>>
>>>> Cheers
>>>>
>>>>
>>>> On Wed, Feb 6, 2013 at 4:07 PM, Andrey V. Romanchev
>>>> <[EMAIL PROTECTED]> wrote:
>>>> >
>>>> > Hello!
>>>> >
>>>> > I'm trying to install Hadoop 1.1.2.21 on CentOS 6.3.
>>>> >
>>>> > I've configured dfs.name.dir in /etc/hadoop/conf/hdfs-site.xml file
>>>> >
>>>> > <name>dfs.name.dir</name>
>>>> > <value>/mnt/ext/hadoop/hdfs/namenode</value>
>>>> >
>>>> > But when I run "hadoop namenode -format" command, it formats
>>>> > /tmp/hadoop-hadoop/dfs/name directory instead. It seems like default
>>>> > directory.
>>>> >
>>>> > What am I missing?
>>>
>>>
>>
>>
>>
>> --
>> Nitin Pawar
>
>

That's worked! Thank you!
Can't find, which setting should I change to show hadoop my conf directory?
2013/2/6 Samir Ahmic <[EMAIL PROTECTED]>

> Hi,
> Can you try to execute format command like this:
>
> hadoop --config  /etc/hadoop/conf namenode -format
>
> Look like hadoop script is unable to pickup your config dir.
>
> Cheers
>
>
> On Wed, Feb 6, 2013 at 4:32 PM, Andrey V. Romanchev
> <[EMAIL PROTECTED]> wrote:
> > Done.
> > $ echo $HADOOP_HOME
> > /usr/lib/hadoop/
> > $ ll /usr/lib/hadoop/conf
> > lrwxrwxrwx. 1 root root 16 Фев  6 16:09 /usr/lib/hadoop/conf ->
> > /etc/hadoop/conf
> >
> > Sadly, it did not help.
> >
> >
> > 2013/2/6 Nitin Pawar <[EMAIL PROTECTED]>
> >>
> >> HADOOP_HOME is the directory where you installed hadoop
> >>
> >> a directory above your conf directory (in other words parent directory
> of
> >> conf)
> >>
> >>
> >> On Wed, Feb 6, 2013 at 8:55 PM, Andrey V. Romanchev
> >> <[EMAIL PROTECTED]> wrote:
> >>>
> >>> Hello Samir
> >>> Yes. Here is full section from my hdfs-site.xml
> >>>
> >>>   <property>
> >>>
> >>>     <name>dfs.name.dir</name>
> >>>     <value>/mnt/ext/hadoop/hdfs/namenode</value>
> >>>     <description>
> >>>         Determines where on the local filesystem the DFS name node
> >>>         should store the name table.  If this is a comma-delimited list
> >>>         of directories then the name table is replicated in all of the
> >>>         directories, for redundancy.
> >>>     </description>
> >>>     <final>true</final>
> >>>   </property>
> >>>
> >>>
> >>> 2013/2/6 Samir Ahmic <[EMAIL PROTECTED]>
> >>>>
> >>>> Hi, Andrey
> >>>>
> >>>> Did you try <final>true</final> in your config ? It should be like
> this:
> >>>>
> >>>> <name>dfs.name.dir</name>
> >>>> <value>/mnt/ext/hadoop/hdfs/namenode</value>
> >>>> <final>true</final>
> >>>>
> >>>> Cheers
> >>>>
> >>>>
> >>>> On Wed, Feb 6, 2013 at 4:07 PM, Andrey V. Romanchev
> >>>> <[EMAIL PROTECTED]> wrote:
> >>>> >
> >>>> > Hello!
> >>>> >
> >>>> > I'm trying to install Hadoop 1.1.2.21 on CentOS 6.3.
> >>>> >
> >>>> > I've configured dfs.name.dir in /etc/hadoop/conf/hdfs-site.xml file
> >>>> >
> >>>> > <name>dfs.name.dir</name>
> >>>> > <value>/mnt/ext/hadoop/hdfs/namenode</value>
> >>>> >
> >>>> > But when I run "hadoop namenode -format" command, it formats
> >>>> > /tmp/hadoop-hadoop/dfs/name directory instead. It seems like default
> >>>> > directory.
> >>>> >
> >>>> > What am I missing?
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> Nitin Pawar
> >
> >
>

Try running the command using "hadoop --config /etc/hadoop/conf" to make sure it is looking at the right conf dir.  

It would help to understand how you installed hadoop - local build/rpm, etc .. to figure out which config dir is being looked at by default.

-- Hitesh

On Feb 6, 2013, at 7:25 AM, Andrey V. Romanchev wrote:

> Hello Samir
> Yes. Here is full section from my hdfs-site.xml
>
>   <property>
>     <name>dfs.name.dir</name>
>     <value>/mnt/ext/hadoop/hdfs/namenode</value>
>     <description>
>         Determines where on the local filesystem the DFS name node
>         should store the name table.  If this is a comma-delimited list
>         of directories then the name table is replicated in all of the
>         directories, for redundancy.
>     </description>
>     <final>true</final>
>   </property>
>
>
> 2013/2/6 Samir Ahmic <[EMAIL PROTECTED]>
> Hi, Andrey
>
> Did you try <final>true</final> in your config ? It should be like this:
>
> <name>dfs.name.dir</name>
> <value>/mnt/ext/hadoop/hdfs/namenode</value>
> <final>true</final>
>
> Cheers
>
>
> On Wed, Feb 6, 2013 at 4:07 PM, Andrey V. Romanchev
> <[EMAIL PROTECTED]> wrote:
> >
> > Hello!
> >
> > I'm trying to install Hadoop 1.1.2.21 on CentOS 6.3.
> >
> > I've configured dfs.name.dir in /etc/hadoop/conf/hdfs-site.xml file
> >
> > <name>dfs.name.dir</name>
> > <value>/mnt/ext/hadoop/hdfs/namenode</value>
> >
> > But when I run "hadoop namenode -format" command, it formats /tmp/hadoop-hadoop/dfs/name directory instead. It seems like default directory.
> >
> > What am I missing?
>