Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> one minute delay in running a simple ls command on hadoop (maybe near security groups..): hadoop 0.23.5


Copy link to this message
-
one minute delay in running a simple ls command on hadoop (maybe near security groups..): hadoop 0.23.5
Hi..

I have installed hadoop 0.23.5 and is working fine on two of my
installations.  In a new installation on Windows Azure VMs, I am seeing an
inordinate delay (of 1 minute) when doing a simple dfs ls command.

This is a 3 node cluster, running 64 bit Ubuntu 12.04.1 LTS, Sun jre

command : bin/hadoop dfs -ls /

With log set to DEBUG, following is the output on the console.  The logs in
bold show the sleep/timeout/delay of *1 minute*.  Any pointers on what to
check for / what is the reason for this delay?

13/03/31 05:48:55 DEBUG lib.MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
with annotation @org.apache.
hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of
successful kerberos logins
and latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)
13/03/31 05:48:55 DEBUG lib.MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
with annotation @org.apache.
hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of
failed kerberos logins and
latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)
13/03/31 05:48:55 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group
related metrics
13/03/31 05:49:55 DEBUG security.Groups:  Creating new Groups object
*13/03/31 05:49:55 DEBUG security.Groups: Group mapping
impl=org.apache.hadoop.security.ShellBasedUni*
*xGroupsMapping; cacheTimeout=300000*
*13/03/31 05:50:55 DEBUG security.UserGroupInformation: hadoop login*
13/03/31 05:50:55 DEBUG security.UserGroupInformation: hadoop login commit
13/03/31 05:50:55 DEBUG security.UserGroupInformation: using local
user:UnixPrincipal: hduser
13/03/31 05:50:55 DEBUG security.UserGroupInformation: UGI loginUser:hduser
(auth:SIMPLE)
13/03/31 05:50:56 DEBUG hdfs.DFSClient: Short circuit read is false
13/03/31 05:50:56 DEBUG ipc.Client: The ping interval is 60000 ms.
13/03/31 05:50:56 DEBUG ipc.Client: Use SIMPLE authentication for protocol
ClientProtocol
13/03/31 05:50:56 DEBUG ipc.Client: Connecting to XXXXXX:9000
Thanks
Gopi
+
Daryn Sharp 2013-04-04, 12:59
+
Gopi Krishna M 2013-04-04, 13:09
+
Gopi Krishna M 2013-04-04, 12:33
+
Gopi Krishna M 2013-04-04, 15:17
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB