Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Some questions on HLog


+
Wukang Lin 2013-10-24, 15:49
Copy link to this message
-
Re: Some questions on HLog
For #2, see HBASE-5954

For #1, see the following comment in SequenceFileLogWriter:

+      // reflection for a version of SequenceFile.createWriter that doesn't
+      // automatically create the parent directory (see HBASE-2312)
+      this.writer = (SequenceFile.Writer) SequenceFile.class

On Thu, Oct 24, 2013 at 8:49 AM, Wukang Lin <[EMAIL PROTECTED]> wrote:

> Hi all,
>     Recently, i read the source of HBase's HLog, and i got some questions
> that puzzled me a lot. here there are:
>     1 why use reflection to init a SequenceFile.Writer
> in SequenceFileLogWriter? i read HBASE-2312 but still can't catch the
> point.
>      2 It seems that hlog use SequenceFile.Writer's append method to sync
> the WAL logs to DataNode, not FSDataOutputStream.hflush(), for each
> mutation(or batch mutations), so may it lose data when HDFS crash while WAL
> logs were 'sync' to DataNode but not flush to disk? or are there something
> i misunderstanded?
>
>       Thank you.
>
+
Wukang Lin 2013-10-24, 16:27
+
Ted Yu 2013-10-24, 16:33
+
Wukang Lin 2013-10-24, 17:30
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB