Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> one minute delay in running a simple ls command on hadoop (maybe near security groups..): hadoop 0.23.5


+
Gopi Krishna M 2013-03-31, 06:17
+
Daryn Sharp 2013-04-04, 12:59
Copy link to this message
-
Re: one minute delay in running a simple ls command on hadoop (maybe near security groups..): hadoop 0.23.5
just tried the command.  returns instantaneously, no delay

~/installations/hadoop-0.23.5$ groups hduser
hduser : hduser

ps: -F option is not working.

On Thu, Apr 4, 2013 at 6:29 PM, Daryn Sharp <[EMAIL PROTECTED]> wrote:

>  Hi Gopi,
>
>  Check if you can run "groups -F hduser".  I think it's causing the delay.
>
>  Daryn
>
>  On Mar 31, 2013, at 1:17 AM, Gopi Krishna M wrote:
>
>  Hi..
>
>  I have installed hadoop 0.23.5 and is working fine on two of my
> installations.  In a new installation on Windows Azure VMs, I am seeing an
> inordinate delay (of 1 minute) when doing a simple dfs ls command.
>
>  This is a 3 node cluster, running 64 bit Ubuntu 12.04.1 LTS, Sun jre
>
>  command : bin/hadoop dfs -ls /
>
>  With log set to DEBUG, following is the output on the console.  The logs
> in bold show the sleep/timeout/delay of *1 minute*.  Any pointers on what
> to check for / what is the reason for this delay?
>
>  13/03/31 05:48:55 DEBUG lib.MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
> with annotation @org.apache.
> hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of
> successful kerberos logins
> and latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)
> 13/03/31 05:48:55 DEBUG lib.MutableMetricsFactory: field
> org.apache.hadoop.metrics2.lib.MutableRate
> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
> with annotation @org.apache.
> hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of
> failed kerberos logins and
> latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)
> 13/03/31 05:48:55 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group
> related metrics
> 13/03/31 05:49:55 DEBUG security.Groups:  Creating new Groups object
> *13/03/31 05:49:55 DEBUG security.Groups: Group mapping
> impl=org.apache.hadoop.security.ShellBasedUni*
> *xGroupsMapping; cacheTimeout=300000*
> *13/03/31 05:50:55 DEBUG security.UserGroupInformation: hadoop login*
> 13/03/31 05:50:55 DEBUG security.UserGroupInformation: hadoop login commit
> 13/03/31 05:50:55 DEBUG security.UserGroupInformation: using local
> user:UnixPrincipal: hduser
> 13/03/31 05:50:55 DEBUG security.UserGroupInformation: UGI
> loginUser:hduser (auth:SIMPLE)
> 13/03/31 05:50:56 DEBUG hdfs.DFSClient: Short circuit read is false
> 13/03/31 05:50:56 DEBUG ipc.Client: The ping interval is 60000 ms.
> 13/03/31 05:50:56 DEBUG ipc.Client: Use SIMPLE authentication for protocol
> ClientProtocol
> 13/03/31 05:50:56 DEBUG ipc.Client: Connecting to XXXXXX:9000
>
>
>  Thanks
> Gopi
>
>
>
+
Gopi Krishna M 2013-04-04, 12:33
+
Gopi Krishna M 2013-04-04, 15:17
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB