Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Re: 【region compaction check key-bound or not】


Copy link to this message
-
Re: 【region compaction check key-bound or not】
Yeah, this is not going to work... moving storefiles manually between
regions is generally not a good idea.
Do you want to move entire table into new table? Then the best thing is
probably to upgrade to 0.94.6 (rolling restart is supported from 0.94.3)
and use HBase snapshots. You can use export table w/o upgrading.
If you want to merge into existing table I'm not really sure what is the
best way to do that. You might have to rewrite data with target region
boundaries, or manually split and merge regions accordingly and run
compactions. Then you can disable the table, copy the files and bulk load
them into target cluster. See
http://hbase.apache.org/book/arch.bulk.load.html
Note that bulk load requires boundaries that fit within one region
(matching or sub-range). There may be a tool that solves this but I'm not
aware of it...
On Wed, Jul 10, 2013 at 5:40 PM, Bing Jiang <[EMAIL PROTECTED]>wrote:

> Thanks,Sergey.
> These days, I want to move a table from one hbase cluster to another hbase
> cluster, and there are the same table's schema.
> So I want to move a region's storefile to another table's region
> corresponding directory, and the key bound of regions are overlapped.
> For example:
> Cluster   |  Table | Region's key bound
> cluster1  |  dat    |  [1ffff,2ffff]
> cluster2  |  dat    |  [2bfff,31fff]
>
> In my opinion , if region files in cluster2 are move into cluster1's
> regions, and make compaction(Minor && Major) upon the first region, it will
> prune the improper key in (2ffff,31fff].
> However I found hbase compaction cannot support that.
>
> Any idea would be thankful.
>
>
> 2013/7/11 Sergey Shelukhin <[EMAIL PROTECTED]>
>
> > You should not have to manually take care of region bounds in normal
> > circumstances (unless you are reading the file from coprocessor in some
> > special way, or something like that). Please tell us if you are seeing
> any
> > strange behavior :)
> > See HalfStoreFileReader for the code that is used to read the referenced
> > file and constrains the keys.
> >
> > On Wed, Jul 10, 2013 at 4:25 AM, Bing Jiang <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Hi,all
> > >
> > > If Region process splits, it will make a reference.After executing
> child
> > > region makes a compaction that absorbs all the reference. And I have a
> > > question that how to make differences when executes scanner.
> > > As we know that Compaction uses the scanner as well, so whether to set
> > the
> > > startKey and endKey of child region's bound, in order to guarantee that
> > the
> > > storefile in child region will not contain outlier keys? I cannot find
> > code
> > > to prove it, BTW, we use 0.94.3.
> > >
> > > Thanks.
> > >
> > > --
> > > Bing Jiang
> > > weibo: http://weibo.com/jiangbinglover
> > > BLOG: http://blog.sina.com.cn/jiangbinglover
> > > National Research Center for Intelligent Computing Systems
> > > Institute of Computing technology
> > > Graduate University of Chinese Academy of Science
> > >
> >
>
>
>
> --
> Bing Jiang
> Tel:(86)134-2619-1361
> weibo: http://weibo.com/jiangbinglover
> BLOG: http://blog.sina.com.cn/jiangbinglover
> National Research Center for Intelligent Computing Systems
> Institute of Computing technology
> Graduate University of Chinese Academy of Science
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB