Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> one minute delay in running a simple ls command on hadoop (maybe near security groups..): hadoop 0.23.5


Copy link to this message
-
Re: one minute delay in running a simple ls command on hadoop (maybe near security groups..): hadoop 0.23.5
Hi Gopi,

Check if you can run "groups -F hduser".  I think it's causing the delay.

Daryn

On Mar 31, 2013, at 1:17 AM, Gopi Krishna M wrote:

Hi..

I have installed hadoop 0.23.5 and is working fine on two of my installations.  In a new installation on Windows Azure VMs, I am seeing an inordinate delay (of 1 minute) when doing a simple dfs ls command.

This is a 3 node cluster, running 64 bit Ubuntu 12.04.1 LTS, Sun jre

command : bin/hadoop dfs -ls /

With log set to DEBUG, following is the output on the console.  The logs in bold show the sleep/timeout/delay of 1 minute.  Any pointers on what to check for / what is the reason for this delay?

13/03/31 05:48:55 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.
hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of successful kerberos logins
and latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)
13/03/31 05:48:55 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.
hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of failed kerberos logins and
latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)
13/03/31 05:48:55 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
13/03/31 05:49:55 DEBUG security.Groups:  Creating new Groups object
13/03/31 05:49:55 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.ShellBasedUni
xGroupsMapping; cacheTimeout=300000
13/03/31 05:50:55 DEBUG security.UserGroupInformation: hadoop login
13/03/31 05:50:55 DEBUG security.UserGroupInformation: hadoop login commit
13/03/31 05:50:55 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: hduser
13/03/31 05:50:55 DEBUG security.UserGroupInformation: UGI loginUser:hduser (auth:SIMPLE)
13/03/31 05:50:56 DEBUG hdfs.DFSClient: Short circuit read is false
13/03/31 05:50:56 DEBUG ipc.Client: The ping interval is 60000 ms.
13/03/31 05:50:56 DEBUG ipc.Client: Use SIMPLE authentication for protocol ClientProtocol
13/03/31 05:50:56 DEBUG ipc.Client: Connecting to XXXXXX:9000
Thanks
Gopi