Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> one minute delay in running a simple ls command on hadoop (maybe near security groups..): hadoop 0.23.5


+
Gopi Krishna M 2013-03-31, 06:17
+
Daryn Sharp 2013-04-04, 12:59
Copy link to this message
-
Re: one minute delay in running a simple ls command on hadoop (maybe near security groups..): hadoop 0.23.5
just tried the command.  returns instantaneously, no delay

~/installations/hadoop-0.23.5$ groups hduser
hduser : hduser

ps: -F option is not working.

On Thu, Apr 4, 2013 at 6:29 PM, Daryn Sharp <[EMAIL PROTECTED]> wrote:

>  Hi Gopi,
>
>  Check if you can run "groups -F hduser".  I think it's causing the delay.
>
>  Daryn
>
>  On Mar 31, 2013, at 1:17 AM, Gopi Krishna M wrote:
>
>  Hi..
>
>  I have installed hadoop 0.23.5 and is working fine on two of my
> installations.  In a new installation on Windows Azure VMs, I am seeing an
> inordinate delay (of 1 minute) when doing a simple dfs ls command.
>
>  This is a 3 node cluster, running 64 bit Ubuntu 12.04.1 LTS, Sun jre
>
>  command : bin/hadoop dfs -ls /
>
>  With log set to DEBUG, following is the output on the console.  The logs
> in bold show the sleep/timeout/delay of *1 minute*.  Any pointers on what
> to check for / what is the reason for this delay?
>
>  13/03/31 05:48:55 DEBUG lib.MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
> with annotation @org.apache.
> hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of
> successful kerberos logins
> and latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)
> 13/03/31 05:48:55 DEBUG lib.MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
> with annotation @org.apache.
> hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of
> failed kerberos logins and
> latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)
> 13/03/31 05:48:55 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group
> related metrics
> 13/03/31 05:49:55 DEBUG security.Groups:  Creating new Groups object
> *13/03/31 05:49:55 DEBUG security.Groups: Group mapping
> impl=org.apache.hadoop.security.ShellBasedUni*
> *xGroupsMapping; cacheTimeout=300000*
> *13/03/31 05:50:55 DEBUG security.UserGroupInformation: hadoop login*
> 13/03/31 05:50:55 DEBUG security.UserGroupInformation: hadoop login commit
> 13/03/31 05:50:55 DEBUG security.UserGroupInformation: using local
> user:UnixPrincipal: hduser
> 13/03/31 05:50:55 DEBUG security.UserGroupInformation: UGI
> loginUser:hduser (auth:SIMPLE)
> 13/03/31 05:50:56 DEBUG hdfs.DFSClient: Short circuit read is false
> 13/03/31 05:50:56 DEBUG ipc.Client: The ping interval is 60000 ms.
> 13/03/31 05:50:56 DEBUG ipc.Client: Use SIMPLE authentication for protocol
> ClientProtocol
> 13/03/31 05:50:56 DEBUG ipc.Client: Connecting to XXXXXX:9000
>
>
>  Thanks
> Gopi
>
>
>
+
Gopi Krishna M 2013-04-04, 12:33
+
Gopi Krishna M 2013-04-04, 15:17