Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> A requirement to change time of the Hbase cluster.


+
Gaojinchao 2011-10-25, 04:33
+
Gaojinchao 2011-10-25, 10:14
+
Michel Segel 2011-10-25, 19:54
+
Gaojinchao 2011-10-26, 02:36
+
Doug Meil 2011-10-26, 13:22
+
Stack 2011-10-26, 16:02
Copy link to this message
-
Re: A requirement to change time of the Hbase cluster.
At the same time, it might be simpler to get your customers/operators
to fix their ntp setups.

Not having synchronized clocks throughout the cluster will cause
problems in other areas as well.  It will make it very difficult to
correlate events in different server logs when troubleshooting
problems (you can't rely on the timestamp as a rough guideline).  The
Kerberos infrastructure used by Hadoop and HBase security also
requires synchronized clocks to operate correctly.  By default, if the
clock skew between two machines is greater than 5 minutes, it will
reject messages as invalid.  So you're likely to experience other
headaches even if you can coax HBase into operating the way you'd
like.
On Wed, Oct 26, 2011 at 9:02 AM, Stack <[EMAIL PROTECTED]> wrote:
> On Tue, Oct 25, 2011 at 7:36 PM, Gaojinchao <[EMAIL PROTECTED]> wrote:
>> So we hope to add a choice about metadata is not time-dependent. Just like use data can use a number as a timestamp .
>> If we can do this, the effect for time will be smaller. We don't use the ntp server, the cluster also can work normal ?
>> Can I open a file? I will try to make a patch and share my mind.
>>
>
> I suppose you could set an attribute on a table that says "use always
> increasing version rather than timestamp".  You'd have to then on a
> per region basis keep note of the most recent version and rather than
> use system time, do a +1 per edit coming in.
>
> I think hfile already records the version of the last edit added to
> the file.  On open of a region, you'd look at all hfiles and figure
> the highest verison and then set your version machine to start at
> highest-version +1.
>
> It might not be that hard to add.  You'd have to check the code but a
> while back we made it so we indirectly got version by going to an
> EnvironmentEdge class.  You could add your 'always increasing version'
> as an atomic long or something and then it would be available
> throughout.
>
> St.Ack
>
+
Doug Meil 2011-10-26, 19:00
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB