Jane Wayne 2013-05-14, 14:41
Niels Basjes 2013-05-14, 15:38
Jane Wayne 2013-05-14, 15:46
Niels Basjes 2013-05-14, 21:09
Jane Wayne 2013-05-16, 14:01
Niels Basjes 2013-05-16, 15:34
Michael Segel 2013-05-16, 16:46
Jane Wayne 2013-05-17, 11:51
Niels Basjes 2013-05-17, 13:17
Bertrand Dechoux 2013-05-17, 14:02
Jane Wayne 2013-05-17, 14:10
-Re: how to get the time of a hadoop cluster, v0.20.2
Jane Wayne 2013-05-17, 14:13
and please remember, i stated that although the hadoop cluster uses NTP,
the "server" (the machine that is not a part of the hadoop cluster) cannot
assume to be using NTP (and in fact, doesn't).
On Fri, May 17, 2013 at 10:10 AM, Jane Wayne <[EMAIL PROTECTED]>wrote:
> "if NTP is correclty used"
> that's the key statement. in several of our clusters, NTP setup is kludgy.
> note that the professionals administering the cluster are different from
> "us" the engineers. so, there's a lot of red tape to go through to get
> something trivial or not fixed. we have noticed that NTP is not setup
> correctly (using default GMT timezone, for example). without explaining all
> the tedious details, this mismatch of date/time (of the hadoop cluster to
> the server machine) is causing some pains.
> i'm not sure i agree with "the local OS time from your server machine will
> be the best estimation." that doesn't make sense.
> but what i want to achieve is very simple. as stated before, i just want
> to ask the namenode or jobtracker, "hey, what date/time do you have?"
> unfortunately for me, as niels pointed out, this query is not possible via
> the hadoop api.
> thanks for helping, though.
> On Fri, May 17, 2013 at 10:02 AM, Bertrand Dechoux <[EMAIL PROTECTED]>wrote:
>> For hadoop, 'cluster time' is the local OS time. You might want to get the
>> time of the namenode machine but indeed if NTP is correctly used, the
>> OS time from your server machine will be the best estimation. If you
>> request the time from the namenode machine, you will be penalized by the
>> delay of your request.
>> On Fri, May 17, 2013 at 3:17 PM, Niels Basjes <[EMAIL PROTECTED]> wrote:
>> > Hi,
>> > > i have another computer (which i have referred to as a server, since
>> > is
>> > > running tomcat), and this computer is NOT a part of the hadoop cluster
>> > (it
>> > > doesn't run any of the hadoop daemons), but does submit jobs to the
>> > hadoop
>> > > cluster via a JEE webapp interface. i need to check that the time on
>> > > computer is in sync with the time on the hadoop cluster. when i say
>> > "check
>> > > that the time is in sync", there is a defined tolerance/threshold
>> > > difference in date/time that i am willing to accept (e.g. the
>> > > should be the same down to the minute).
>> > If you ensure (using NTP) that all your servers have the same time then
>> > can simply query your local server for the time and you have the correct
>> > answer to your question.
>> > You are searching for a solution in the Hadoop API (where this does not
>> > exist) when the solution is present at a different level.
>> > --
>> > Best regards / Met vriendelijke groeten,
>> > Niels Basjes
Michael Segel 2013-05-18, 15:54
Jane Wayne 2013-05-17, 14:01