HDFS, mail # user - Re: problem configuring hadoop with s3 bucket

problem configuring hadoop with s3 bucket

Alok Kumar 2012-07-23, 08:26

Hello Group,

I've hadoop setup locally running.

Now I want to use Amazon s3://<mybucket> as my data store,
so i changed like " dfs.data.dir=s3://<mybucket>/hadoop/ " in my
hdfs-site.xml, Is it the correct way?
I'm getting error :

WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Invalid directory in
dfs.data.dir: can not create directory: s3://<mybucket>/hadoop
2012-07-23 13:15:06,260 ERROR
org.apache.hadoop.hdfs.server.datanode.DataNode: All directories in
dfs.data.dir are invalid.

and
when i changed like " dfs.data.dir=s3://<mybucket>/ "
I got error :
ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:
java.lang.IllegalArgumentException: Wrong FS: s3://<mybucket>/, expected:
file:///
   at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
   at
org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
   at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
   at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
   at
org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:146)
   at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:162)
   at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1574)
   at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
   at
org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
   at
org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
   at
org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)

Also,
When I'm changing fs.default.name=s3://<<mybucket> , Namenode is not coming
up with error : ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
java.net.BindException: (Any way I want to run namenode locally, so I
reverted it back to hdfs://localhost:9000 )

Your help is highly appreciated!
Thanks
--
Alok Kumar

Re: problem configuring hadoop with s3 bucket

Yanbo Liang 2012-07-24, 10:41

I think you have made confusion about the integration of hadoop and S3.
1) If you set "dfs.data.dir=s3://******", it means that you set S3 as the
DataNode local storage.
You are still using HDFS as the under storage layer.
As far as I know, it is not supported at present.
2) The right way of using S3 integrated with Hadoop is to replace the HDFS
with S3 just like you have tried.
But I think you have missed some configuration parameters.
" fs.default.name=s3://<<mybucket> " is the most important parameter when
you are using S3 to replace HDFS, but it is not enough. The detail
configuration can be obtained from here
http://wiki.apache.org/hadoop/AmazonS3

Yanbo

2012/7/23 Alok Kumar <[EMAIL PROTECTED]>

> Hello Group,
>
> I've hadoop setup locally running.
>
> Now I want to use Amazon s3://<mybucket> as my data store,
> so i changed like " dfs.data.dir=s3://<mybucket>/hadoop/ " in my
> hdfs-site.xml, Is it the correct way?
> I'm getting error :
>
> WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Invalid directory in
> dfs.data.dir: can not create directory: s3://<mybucket>/hadoop
> 2012-07-23 13:15:06,260 ERROR
> org.apache.hadoop.hdfs.server.datanode.DataNode: All directories in
> dfs.data.dir are invalid.
>
> and
> when i changed like " dfs.data.dir=s3://<mybucket>/ "
> I got error :
> ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:
> java.lang.IllegalArgumentException: Wrong FS: s3://<mybucket>/, expected:
> file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> at
> org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:146)
> at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:162)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1574)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
>
> Also,
> When I'm changing fs.default.name=s3://<<mybucket> , Namenode is not
> coming up with error : ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.net.BindException:
> (Any way I want to run namenode locally, so I reverted it back to
> hdfs://localhost:9000 )
>
> Your help is highly appreciated!
> Thanks
> --
> Alok Kumar
>

Re: problem configuring hadoop with s3 bucket

Alok Kumar 2012-07-24, 12:16

Hi Yanbo,

Thank you for your reply..

Now I've made changes exactly , present in this link
http://wiki.apache.org/hadoop/AmazonS3

But my namenode is not coming up with exception (I tried it both locally
and inside EC2)

ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
java.net.BindException: Problem binding to <bucket-name>.
s3.amazonaws.com/207.171.163.14:8020 : Cannot assign requested address

fs.default.name = s3://<mybucket> // Error : UnknownHost Exception in
Namenode!
or
fs.default.name = s3://<mybucket>.s3.amazonaws.com // Error BindException
in Namenode log!

Also I can't see any dir created inside my bucket. ( I'm able to execute
command $ bin/hadoop dfs -ls s3://<bucket>/ )

"bin/hadoop namenode -format " is saying succesfully formatted namenode dir
S3://bucket/hadoop/namenode , when it is not even existing there!
any suggestion?

Thanks again.

On Tue, Jul 24, 2012 at 4:11 PM, Yanbo Liang <[EMAIL PROTECTED]> wrote:

> I think you have made confusion about the integration of hadoop and S3.
> 1) If you set "dfs.data.dir=s3://******", it means that you set S3 as the
> DataNode local storage.
> You are still using HDFS as the under storage layer.
> As far as I know, it is not supported at present.
> 2) The right way of using S3 integrated with Hadoop is to replace the
> HDFS with S3 just like you have tried.
> But I think you have missed some configuration parameters.
> " fs.default.name=s3://<<mybucket> " is the most important parameter
> when you are using S3 to replace HDFS, but it is not enough. The detail
> configuration can be obtained from here
> http://wiki.apache.org/hadoop/AmazonS3
>
> Yanbo
>
>
> 2012/7/23 Alok Kumar <[EMAIL PROTECTED]>
>
>> Hello Group,
>>
>> I've hadoop setup locally running.
>>
>> Now I want to use Amazon s3://<mybucket> as my data store,
>> so i changed like " dfs.data.dir=s3://<mybucket>/hadoop/ " in my
>> hdfs-site.xml, Is it the correct way?
>> I'm getting error :
>>
>> WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Invalid directory
>> in dfs.data.dir: can not create directory: s3://<mybucket>/hadoop
>> 2012-07-23 13:15:06,260 ERROR
>> org.apache.hadoop.hdfs.server.datanode.DataNode: All directories in
>> dfs.data.dir are invalid.
>>
>> and
>> when i changed like " dfs.data.dir=s3://<mybucket>/ "
>> I got error :
>> ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:
>> java.lang.IllegalArgumentException: Wrong FS: s3://<mybucket>/, expected:
>> file:///
>> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
>> at
>> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>> at
>> org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:146)
>> at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:162)
>> at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1574)
>> at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
>> at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
>> at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
>> at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
>>
>> Also,
>> When I'm changing fs.default.name=s3://<<mybucket> , Namenode is not
>> coming up with error : ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.net.BindException:
>> (Any way I want to run namenode locally, so I reverted it back to
>> hdfs://localhost:9000 )
>>
>> Your help is highly appreciated!
>> Thanks
>> --
>> Alok Kumar
>>
>
>
--
Alok Kumar

Re: problem configuring hadoop with s3 bucket

Yanbo Liang 2012-07-25, 06:35

Could you provide your execute environment, operation procedure and the
detail log information?

2012/7/24 Alok Kumar <[EMAIL PROTECTED]>

> Hi Yanbo,
>
> Thank you for your reply..
>
> Now I've made changes exactly , present in this link
> http://wiki.apache.org/hadoop/AmazonS3
>
> But my namenode is not coming up with exception (I tried it both locally
> and inside EC2)
>
> ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
> java.net.BindException: Problem binding to <bucket-name>.
> s3.amazonaws.com/207.171.163.14:8020 : Cannot assign requested address
>
> fs.default.name = s3://<mybucket> // Error : UnknownHost Exception in
> Namenode!
> or
> fs.default.name = s3://<mybucket>.s3.amazonaws.com // Error BindException
> in Namenode log!
>
> Also I can't see any dir created inside my bucket. ( I'm able to execute
> command $ bin/hadoop dfs -ls s3://<bucket>/ )
>
> "bin/hadoop namenode -format " is saying succesfully formatted namenode
> dir S3://bucket/hadoop/namenode , when it is not even existing there!
>
>
> any suggestion?
>
> Thanks again.
>
>
> On Tue, Jul 24, 2012 at 4:11 PM, Yanbo Liang <[EMAIL PROTECTED]> wrote:
>
>> I think you have made confusion about the integration of hadoop and S3.
>> 1) If you set "dfs.data.dir=s3://******", it means that you set S3 as the
>> DataNode local storage.
>> You are still using HDFS as the under storage layer.
>> As far as I know, it is not supported at present.
>> 2) The right way of using S3 integrated with Hadoop is to replace the
>> HDFS with S3 just like you have tried.
>> But I think you have missed some configuration parameters.
>> " fs.default.name=s3://<<mybucket> " is the most important parameter
>> when you are using S3 to replace HDFS, but it is not enough. The detail
>> configuration can be obtained from here
>> http://wiki.apache.org/hadoop/AmazonS3
>>
>> Yanbo
>>
>>
>> 2012/7/23 Alok Kumar <[EMAIL PROTECTED]>
>>
>>> Hello Group,
>>>
>>> I've hadoop setup locally running.
>>>
>>> Now I want to use Amazon s3://<mybucket> as my data store,
>>> so i changed like " dfs.data.dir=s3://<mybucket>/hadoop/ " in my
>>> hdfs-site.xml, Is it the correct way?
>>> I'm getting error :
>>>
>>> WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Invalid directory
>>> in dfs.data.dir: can not create directory: s3://<mybucket>/hadoop
>>> 2012-07-23 13:15:06,260 ERROR
>>> org.apache.hadoop.hdfs.server.datanode.DataNode: All directories in
>>> dfs.data.dir are invalid.
>>>
>>> and
>>> when i changed like " dfs.data.dir=s3://<mybucket>/ "
>>> I got error :
>>> ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:
>>> java.lang.IllegalArgumentException: Wrong FS: s3://<mybucket>/, expected:
>>> file:///
>>> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
>>> at
>>> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
>>> at
>>> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393)
>>> at
>>> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>>> at
>>> org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:146)
>>> at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:162)
>>> at
>>> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1574)
>>> at
>>> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
>>> at
>>> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
>>> at
>>> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
>>> at
>>> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
>>>
>>> Also,
>>> When I'm changing fs.default.name=s3://<<mybucket> , Namenode is not
>>> coming up with error : ERROR
>>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.net.BindException:
>>> (Any way I want to run namenode locally, so I reverted it back to
>>> hdfs://localhost:9000 )

Re: problem configuring hadoop with s3 bucket

Alok Kumar 2012-07-25, 07:55

Hi Yanbo,

I'm using hadoop-1.0.1 on local machine (not EC2)

conf/master : localhost
conf/slave : localhost
conf/core-site.xml :
<property>
   <name>fs.default.name</name>
   <value>s3://bucketname</value>
  </property>

<property>
<name>fs.s3.awsAccessKeyId</name>
<value>***********</value>
</property>

<property>
<name>fs.s3.awsSecretAccessKey</name>
<value>**************</value>
</property>

/etc/hosts :
192.168.2.24 DRPM4 # Added by NetworkManager
127.0.0.1 localhost.localdomain localhost
$ bin/hadoop namenode -format : successful

$ bin/start-dfs.sh

Namenode log :

2012-07-25 13:19:32,896 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = DRPM4/192.168.2.24
STARTUP_MSG: args = []
STARTUP_MSG: version = 1.0.1
STARTUP_MSG: build https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r
1243785; compiled by 'hortonfo' on Tue Feb 14 08:15:38 UTC 2012
************************************************************/
2012-07-25 13:19:33,037 INFO org.apache.hadoop.metrics2.impl.MetricsConfig:
loaded properties from hadoop-metrics2.properties
2012-07-25 13:19:33,049 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
MetricsSystem,sub=Stats registered.
2012-07-25 13:19:33,050 INFO
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
period at 10 second(s).
2012-07-25 13:19:33,050 INFO
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system
started
2012-07-25 13:19:33,889 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi
registered.
2012-07-25 13:19:33,896 WARN
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
exists!
2012-07-25 13:19:33,904 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm
registered.
2012-07-25 13:19:33,906 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
NameNode registered.
2012-07-25 13:19:33,946 INFO org.apache.hadoop.hdfs.util.GSet: VM
type = 32-bit
2012-07-25 13:19:33,946 INFO org.apache.hadoop.hdfs.util.GSet: 2% max
memory = 17.77875 MB
2012-07-25 13:19:33,946 INFO org.apache.hadoop.hdfs.util.GSet:
capacity = 2^22 = 4194304 entries
2012-07-25 13:19:33,946 INFO org.apache.hadoop.hdfs.util.GSet:
recommended=4194304, actual=4194304
2012-07-25 13:19:33,986 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=alok
2012-07-25 13:19:33,986 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup
2012-07-25 13:19:33,986 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
isPermissionEnabled=true
2012-07-25 13:19:33,995 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
dfs.block.invalidate.limit=100
2012-07-25 13:19:33,995 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s),
accessTokenLifetime=0 min(s)
2012-07-25 13:19:34,396 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered
FSNamesystemStateMBean and NameNodeMXBean
2012-07-25 13:19:34,416 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names
occuring more than 10 times
2012-07-25 13:19:34,442 INFO org.apache.hadoop.hdfs.server.common.Storage:
Number of files = 1
2012-07-25 13:19:34,449 INFO org.apache.hadoop.hdfs.server.common.Storage:
Number of files under construction = 0
2012-07-25 13:19:34,449 INFO org.apache.hadoop.hdfs.server.common.Storage:
Image file of size 110 loaded in 0 seconds.
2012-07-25 13:19:34,449 INFO org.apache.hadoop.hdfs.server.common.Storage:
Edits file /home/alok/work/s3s/hadoop/namenode/current/edits of size 4
edits # 0 loaded in 0 seconds.
2012-07-25 13:19:34,451 INFO org.apache.hadoop.hdfs.server.common.Storage:
Image file of size 110 saved in 0 seconds.
2012-07-25 13:19:35,161 INFO org.apache.hadoop.hdfs.server.common.Storage:
Image file of size 110 saved in 0 seconds.
2012-07-25 13:19:35,873 INFO
org.apache.hadoop.hdfs.server.namenode.NameCache: initialized with 0
entries 0 lookups
2012-07-25 13:19:35,873 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Finished loading
FSImage in 1900 msecs
2012-07-25 13:19:35,888 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Total number of blocks
= 0
2012-07-25 13:19:35,888 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of invalid
blocks = 0
2012-07-25 13:19:35,888 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
under-replicated blocks = 0
2012-07-25 13:19:35,888 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
over-replicated blocks = 0
2012-07-25 13:19:35,889 INFO org.apache.hadoop.hdfs.StateChange: STATE*
Safe mode termination scan for invalid, over- and under-replicated blocks
completed in 15 msec
2012-07-25 13:19:35,889 INFO org.apache.hadoop.hdfs.StateChange: STATE*
Leaving safe mode after 1 secs.
2012-07-25 13:19:35,889 INFO org.apache.hadoop.hdfs.StateChange: STATE*
Network topology has 0 racks and 0 datanodes
2012-07-25 13:19:35,889 INFO org.apache.hadoop.hdfs.StateChange: STATE*
UnderReplicatedBlocks has 0 blocks
2012-07-25 13:19:35,897 INFO org.apache.hadoop.util.HostsFileReader:
Refreshing hosts (include/exclude) list
2012-07-25 13:19:35,905 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: ReplicateQueue
QueueProcessingStatistics: First cycle completed 0 blocks in 7 msec
2012-07-25 13:19:35,905 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: ReplicateQueue
QueueProcessingStatistics: Queue flush completed 0 blocks in 7 msec
processing time, 7 msec clock time, 1 cycles
2012-07-25 13:19:35,905 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
FSNamesystemMetrics registered.
2012-07-25 13:19:35,906 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: InvalidateQueue
QueueProcessingStatistics: First cycle completed 0 blocks in 1 msec
20

Re: problem configuring hadoop with s3 bucket

Yanbo Liang 2012-07-26, 07:20

>From the log, I guess that you have made wrong configurations or operations.
The S3 storage layer is at the same level with HDFS.
So if you configure to use S3, it's irrational to occur the HDFS related
logs such as "org.apache.hadoop.hdfs.server.namenode.FSNamesystem".
The HDFS related logs only occur at the condition that you use HDFS as the
underlying storage layer.

I infer that you should not format the file system, because S3 is already
service delivery and the Hadoop framework just use it as the underlying
file system.
Don't execute " bin/hadoop namenode -format ", it is used only when your
underlying file system is HDFS.

Yanbo

2012/7/25 Alok Kumar <[EMAIL PROTECTED]>

> Hi Yanbo,
>
> I'm using hadoop-1.0.1 on local machine (not EC2)
>
> conf/master : localhost
> conf/slave : localhost
> conf/core-site.xml :
> <property>
> <name>fs.default.name</name>
> <value>s3://bucketname</value>
> </property>
>
> <property>
> <name>fs.s3.awsAccessKeyId</name>
> <value>***********</value>
> </property>
>
> <property>
> <name>fs.s3.awsSecretAccessKey</name>
> <value>**************</value>
> </property>
>
> /etc/hosts :
> 192.168.2.24 DRPM4 # Added by NetworkManager
> 127.0.0.1 localhost.localdomain localhost
>
>
> $ bin/hadoop namenode -format : successful
>
> $ bin/start-dfs.sh
>
> Namenode log :
>
> 2012-07-25 13:19:32,896 INFO
> org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting NameNode
> STARTUP_MSG: host = DRPM4/192.168.2.24
> STARTUP_MSG: args = []
> STARTUP_MSG: version = 1.0.1
> STARTUP_MSG: build > https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r
> 1243785; compiled by 'hortonfo' on Tue Feb 14 08:15:38 UTC 2012
> ************************************************************/
> 2012-07-25 13:19:33,037 INFO
> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
> hadoop-metrics2.properties
> 2012-07-25 13:19:33,049 INFO
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
> MetricsSystem,sub=Stats registered.
> 2012-07-25 13:19:33,050 INFO
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
> period at 10 second(s).
> 2012-07-25 13:19:33,050 INFO
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system
> started
> 2012-07-25 13:19:33,889 INFO
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi
> registered.
> 2012-07-25 13:19:33,896 WARN
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
> exists!
> 2012-07-25 13:19:33,904 INFO
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm
> registered.
> 2012-07-25 13:19:33,906 INFO
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
> NameNode registered.
> 2012-07-25 13:19:33,946 INFO org.apache.hadoop.hdfs.util.GSet: VM
> type = 32-bit
> 2012-07-25 13:19:33,946 INFO org.apache.hadoop.hdfs.util.GSet: 2% max
> memory = 17.77875 MB
> 2012-07-25 13:19:33,946 INFO org.apache.hadoop.hdfs.util.GSet:
> capacity = 2^22 = 4194304 entries
> 2012-07-25 13:19:33,946 INFO org.apache.hadoop.hdfs.util.GSet:
> recommended=4194304, actual=4194304
> 2012-07-25 13:19:33,986 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=alok
> 2012-07-25 13:19:33,986 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup
> 2012-07-25 13:19:33,986 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
> isPermissionEnabled=true
> 2012-07-25 13:19:33,995 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
> dfs.block.invalidate.limit=100
> 2012-07-25 13:19:33,995 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
> isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s),
> accessTokenLifetime=0 min(s)
> 2012-07-25 13:19:34,396 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered
> FSNamesystemStateMBean and NameNodeMXBean