Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Merge and HMerge


Copy link to this message
-
Re: Merge and HMerge
Hi Ted,

The log is from an earlier attempt, I tried this a few times. This is all local, after rm'ing the /hbase. So the files are all pretty empty, but since I put data in I was assuming it should work. Once you gotten into this state, you also get funny error messages in the shell:

hbase(main):001:0> list
TABLE                                                                                                                                        
11/07/03 09:36:21 INFO ipc.HBaseRPC: Using org.apache.hadoop.hbase.ipc.WritableRpcEngine for org.apache.hadoop.hbase.ipc.HMasterInterface

ERROR: undefined method `map' for nil:NilClass

Here is some help for this command:
List all tables in hbase. Optional regular expression parameter could
be used to filter the output. Examples:

  hbase> list
  hbase> list 'abc.*'
hbase(main):002:0>

I am assuming this is collateral, but why? The UI works but the table is gone too.

Lars

On Jul 2, 2011, at 10:55 PM, Ted Yu wrote:

> There is TestMergeTool which tests Merge.
>
> From the log you provided, I got a little confused as why
> 'testtable,row-20,1309613053987.23a35ac696bdf4a8023dcc4c5b8419e0.' didn't
> appear in your command line or the output from .META. scanning.
>
> On Sat, Jul 2, 2011 at 10:36 AM, Lars George <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> These two seem both in a bit of a weird state: HMerge is scoped package
>> local, therefore no one but the package can call the merge() functions...
>> and no one does that but the unit test. But it would be good to have this on
>> the CLI and shell as a command (and in the shell maybe with a confirmation
>> message?), but it is not available AFAIK.
>>
>> HMerge can merge regions of tables that are disabled. It also merges all
>> that qualify, i.e. where the merged region is less than or equal of half the
>> configured max file size.
>>
>> Merge on the other hand does have a main(), so can be invoked:
>>
>> $ hbase org.apache.hadoop.hbase.util.Merge
>> Usage: bin/hbase merge <table-name> <region-1> <region-2>
>>
>> Note how the help insinuates that you can use it as a tool, but that is not
>> correct. Also, it only merges two given regions, and the cluster must be
>> shut down (only the HBase daemons). So that is a step back.
>>
>> What is worse is that I cannot get it to work. I tried in the shell:
>>
>> hbase(main):001:0> create 'testtable', 'colfam1',  {SPLITS =>
>> ['row-10','row-20','row-30','row-40','row-50']}
>> 0 row(s) in 0.2640 seconds
>>
>> hbase(main):002:0> for i in '0'..'9' do for j in '0'..'9' do put
>> 'testtable', "row-#{i}#{j}", "colfam1:#{j}", "#{j}" end end
>> 0 row(s) in 1.0450 seconds
>>
>> hbase(main):003:0> flush 'testtable'
>> 0 row(s) in 0.2000 seconds
>>
>> hbase(main):004:0> scan '.META.', { COLUMNS => ['info:regioninfo']}
>> ROW                                  COLUMN+CELL
>> testtable,,1309614509037.612d1e0112 column=info:regioninfo,
>> timestamp=130...
>> 406e6c2bb482eeaec57322.             STARTKEY => '', ENDKEY => 'row-10'
>> testtable,row-10,1309614509040.2fba column=info:regioninfo,
>> timestamp=130...
>> fcc9bc6afac94c465ce5dcabc5d1.       STARTKEY => 'row-10', ENDKEY =>
>> 'row-20'
>> testtable,row-20,1309614509041.e7c1 column=info:regioninfo,
>> timestamp=130...
>> 6267eb30e147e5d988c63d40f982.       STARTKEY => 'row-20', ENDKEY =>
>> 'row-30'
>> testtable,row-30,1309614509041.a9cd column=info:regioninfo,
>> timestamp=130...
>> e1cbc7d1a21b1aca2ac7fda30ad8.       STARTKEY => 'row-30', ENDKEY =>
>> 'row-40'
>> testtable,row-40,1309614509041.d458 column=info:regioninfo,
>> timestamp=130...
>> 236feae097efcf33477e7acc51d4.       STARTKEY => 'row-40', ENDKEY =>
>> 'row-50'
>> testtable,row-50,1309614509041.74a5 column=info:regioninfo,
>> timestamp=130...
>> 7dc7e3e9602d9229b15d4c0357d1.       STARTKEY => 'row-50', ENDKEY => ''
>> 6 row(s) in 0.0440 seconds
>>
>> hbase(main):005:0> exit
>>
>> $ ./bin/stop-hbase.sh
>>
>> $ hbase org.apache.hadoop.hbase.util.Merge testtable \
>> testtable,row-20,1309614509041.e7c16267eb30e147e5d988c63d40f982. \