Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> reason to do major compaction after split


Copy link to this message
-
Re: reason to do major compaction after split
On Thu, Mar 7, 2013 at 2:28 PM, Matteo Bertozzi <[EMAIL PROTECTED]>wrote:

> This is seems to going in a super messy direction.
>

Smile. I was thinking you'd show up on this thread.

Agree.

> With HBASE-7806 the ideas was to cleanup all this crazy stuff (HFileLink,
> References, ...)
>
> unfortunately the initial decision of tight together the fs layout
> and the tables/regions/families is bringing to all this workaround to have
> something cool.
>
> If you put the files in one place, and the association in another  you can
> avoid all this complexity.
>
> /hbase/data/[file1, file 2, file 3, file N]
>
> table 1/region 1: [file 2]
> table 1/region 2: [file 1 (from 0 to 50)]
> table 1/region 3: [file 1 (from 50 to 100)]
> table 2/region 1: [file 1, file 2]
>
>
Any ideas on what migration from current format to the above would be like
Matteo?  We'd read current layout, use it to populate a files table, new
files would be written to a the new /hbase/data/ dir, and for a while we'd
span the old and new locations?

St.Ack