Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Re: [Shadow Regions / Read Replicas ] Wal per region?


Copy link to this message
-
Re: [Shadow Regions / Read Replicas ] Wal per region?
Jonathan Hsieh 2013-12-03, 22:03
On Tue, Dec 3, 2013 at 11:42 AM, Enis Söztutar <[EMAIL PROTECTED]> wrote:

> On Mon, Dec 2, 2013 at 10:20 PM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
>
> > > Deveraj:
> > > Jonathan Hsieh, WAL per region (WALpr) would give you the locality (and
> > hence HDFS short
> > > circuit) of reads if you were to couple it with the favored nodes. The
> > cost is of course more WAL
> > > files... In the current situation (no WALpr) it would create quite some
> > traffic cross machine, no?
> >
> > I think we all agree that wal per region isn't efficient on today's
> > spinning hard drive world where we are limited to a relatively low budget
> > or seeks (though may be better in the future with SSD's).
> >
>
> WALpr makes sense in fully SSD world and if hdfs had journaling for writes.
> I don't think anybody
> is working on this yet.
what do you mean by journaling for writes?  do you mean where sync
operations update length at the nn on every call?
> Full SSD clusters are already in place (pinterest
> for example), so I
> think having WALpr as a pluggable implementation makes sense. HBase should
> work with both
> WAL-per-regionserver (or multi) or WAL-per-region.
>
>
> I agree here.
> >
> > With this in mind, I actually I making the case that we would group the
> all
> > the regions from RS-A onto the same set of preferred regions servers.
>  This
> > way we only need to have one or two other RS's tailing the RS.
> >
> > So for example, if region X, Y and Z were on RS-A and its hlog, the
> shadow
> > region memstores for X, Y, and Z would be assigned to the same one or two
> > other RSs.  Ideally this would be where the HLog files replicas have
> > locality (helped by favored nodes/block affinity).  Doing this, we hold
> the
> > number of readers on the active hlogs to a constant number, do not add
> any
> > new cross machine traffic (though tailing currently has costs on the NN).
> >
> > One inefficiency we have is that if there is a single log per RS, we end
> up
> > reading all the logs to tables that may not have the shadow feature
> > enabled.  However, with HBase multi-wals coming, one strategy is to shard
> > wals to a number on the order of the number of disks on a machine (12-24
> > these days).  I think the a wal per namespaces (this could be used to
> have
> > a wal per table) of the hlog would make sense.  This way of shardind the
> > hlog would reduce the amount of reading of irrelevant log entries on a
> log
> > tailing scheme. It would have the added benefit of reducing the log
> > splitting work reducing MTTR and allowing for recovery priorities if the
> > primaries and shadows also go down.  (this is an generalization of the
> > separate out the META into a separate log idea).
> >
> > Jon.
> >
> > --
> > // Jonathan Hsieh (shay)
> > // Software Engineer, Cloudera
> > // [EMAIL PROTECTED]
> >
>

--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// [EMAIL PROTECTED]