Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> reason to do major compaction after split


Copy link to this message
-
Re: reason to do major compaction after split
On Thu, Mar 7, 2013 at 2:28 PM, Matteo Bertozzi <[EMAIL PROTECTED]>wrote:

> This is seems to going in a super messy direction.
>

Smile. I was thinking you'd show up on this thread.

Agree.

> With HBASE-7806 the ideas was to cleanup all this crazy stuff (HFileLink,
> References, ...)
>
> unfortunately the initial decision of tight together the fs layout
> and the tables/regions/families is bringing to all this workaround to have
> something cool.
>
> If you put the files in one place, and the association in another  you can
> avoid all this complexity.
>
> /hbase/data/[file1, file 2, file 3, file N]
>
> table 1/region 1: [file 2]
> table 1/region 2: [file 1 (from 0 to 50)]
> table 1/region 3: [file 1 (from 50 to 100)]
> table 2/region 1: [file 1, file 2]
>
>
Any ideas on what migration from current format to the above would be like
Matteo?  We'd read current layout, use it to populate a files table, new
files would be written to a the new /hbase/data/ dir, and for a while we'd
span the old and new locations?

St.Ack
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB