Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Merge and HMerge


Copy link to this message
-
Re: Merge and HMerge
Hi Ted,

The log is from an earlier attempt, I tried this a few times. This is all local, after rm'ing the /hbase. So the files are all pretty empty, but since I put data in I was assuming it should work. Once you gotten into this state, you also get funny error messages in the shell:

hbase(main):001:0> list
TABLE                                                                                                                                        
11/07/03 09:36:21 INFO ipc.HBaseRPC: Using org.apache.hadoop.hbase.ipc.WritableRpcEngine for org.apache.hadoop.hbase.ipc.HMasterInterface

ERROR: undefined method `map' for nil:NilClass

Here is some help for this command:
List all tables in hbase. Optional regular expression parameter could
be used to filter the output. Examples:

  hbase> list
  hbase> list 'abc.*'
hbase(main):002:0>

I am assuming this is collateral, but why? The UI works but the table is gone too.

Lars

On Jul 2, 2011, at 10:55 PM, Ted Yu wrote:

> There is TestMergeTool which tests Merge.
>
> From the log you provided, I got a little confused as why
> 'testtable,row-20,1309613053987.23a35ac696bdf4a8023dcc4c5b8419e0.' didn't
> appear in your command line or the output from .META. scanning.
>
> On Sat, Jul 2, 2011 at 10:36 AM, Lars George <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> These two seem both in a bit of a weird state: HMerge is scoped package
>> local, therefore no one but the package can call the merge() functions...
>> and no one does that but the unit test. But it would be good to have this on
>> the CLI and shell as a command (and in the shell maybe with a confirmation
>> message?), but it is not available AFAIK.
>>
>> HMerge can merge regions of tables that are disabled. It also merges all
>> that qualify, i.e. where the merged region is less than or equal of half the
>> configured max file size.
>>
>> Merge on the other hand does have a main(), so can be invoked:
>>
>> $ hbase org.apache.hadoop.hbase.util.Merge
>> Usage: bin/hbase merge <table-name> <region-1> <region-2>
>>
>> Note how the help insinuates that you can use it as a tool, but that is not
>> correct. Also, it only merges two given regions, and the cluster must be
>> shut down (only the HBase daemons). So that is a step back.
>>
>> What is worse is that I cannot get it to work. I tried in the shell:
>>
>> hbase(main):001:0> create 'testtable', 'colfam1',  {SPLITS =>
>> ['row-10','row-20','row-30','row-40','row-50']}
>> 0 row(s) in 0.2640 seconds
>>
>> hbase(main):002:0> for i in '0'..'9' do for j in '0'..'9' do put
>> 'testtable', "row-#{i}#{j}", "colfam1:#{j}", "#{j}" end end
>> 0 row(s) in 1.0450 seconds
>>
>> hbase(main):003:0> flush 'testtable'
>> 0 row(s) in 0.2000 seconds
>>
>> hbase(main):004:0> scan '.META.', { COLUMNS => ['info:regioninfo']}
>> ROW                                  COLUMN+CELL
>> testtable,,1309614509037.612d1e0112 column=info:regioninfo,
>> timestamp=130...
>> 406e6c2bb482eeaec57322.             STARTKEY => '', ENDKEY => 'row-10'
>> testtable,row-10,1309614509040.2fba column=info:regioninfo,
>> timestamp=130...
>> fcc9bc6afac94c465ce5dcabc5d1.       STARTKEY => 'row-10', ENDKEY =>
>> 'row-20'
>> testtable,row-20,1309614509041.e7c1 column=info:regioninfo,
>> timestamp=130...
>> 6267eb30e147e5d988c63d40f982.       STARTKEY => 'row-20', ENDKEY =>
>> 'row-30'
>> testtable,row-30,1309614509041.a9cd column=info:regioninfo,
>> timestamp=130...
>> e1cbc7d1a21b1aca2ac7fda30ad8.       STARTKEY => 'row-30', ENDKEY =>
>> 'row-40'
>> testtable,row-40,1309614509041.d458 column=info:regioninfo,
>> timestamp=130...
>> 236feae097efcf33477e7acc51d4.       STARTKEY => 'row-40', ENDKEY =>
>> 'row-50'
>> testtable,row-50,1309614509041.74a5 column=info:regioninfo,
>> timestamp=130...
>> 7dc7e3e9602d9229b15d4c0357d1.       STARTKEY => 'row-50', ENDKEY => ''
>> 6 row(s) in 0.0440 seconds
>>
>> hbase(main):005:0> exit
>>
>> $ ./bin/stop-hbase.sh
>>
>> $ hbase org.apache.hadoop.hbase.util.Merge testtable \
>> testtable,row-20,1309614509041.e7c16267eb30e147e5d988c63d40f982. \
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB