Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Merging regions


Copy link to this message
-
Re: Merging regions
Jean-Marc Spaggiari 2012-11-19, 02:31
I don't have a core-site :(

But I was able to figure that I had to add:

    <property>
      <name>fs.default.name</name>
      <value>hdfs://node3:9000/</value>
      <description>The name of the default file system. A URI whose scheme and
      authority determine the FileSystem implementation. The uri's
scheme determines
      the config property (fs.SCHEME.impl) naming the FileSystem implementation
      class. The uri's authority is used to determine the host, port,
etc. for a filesystem.
      </description>
    </property>

With that it's working fine. I was able to merge 2 regions. Now I
"just" need  to build a script to merge all the regions into a single
one.

Thanks,

JM

2012/11/18, Kevin O'dell <[EMAIL PROTECTED]>:
> You should be able to grab that from the core-site and add it into you
> hbase-site.
> On Nov 18, 2012 6:34 PM, "Jean-Marc Spaggiari" <[EMAIL PROTECTED]>
> wrote:
>
>> Seems it has been discussed a year ago ;)
>>
>> http://www.mail-archive.com/[EMAIL PROTECTED]/msg11383.html
>>
>> It looks like I need to setup fs.default.name ... I tried few
>> different values, but even so it's not working. What should this value
>> look like? I used hdfs://localhost/ and the same with the master (also
>> hosting the namenode) with no success. Any idea what it should be?
>>
>> Thanks,
>>
>> JM
>>
>> 2012/11/18, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
>> > Hum
>> >
>> > I tried the --config parameters but I donc think bin/hbase is reading
>> > it.
>> >
>> > I have exported HBASE_CONFIG but  same result.
>> >
>> > Does anyone have succeed in a merge recently?
>> >
>> > I will try to google that to see if there is something I missed
>> >
>> > 2012/11/18, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
>> >> And that's the issue.
>> >>
>> >> I have an 8 nodes cluster, with 3 ZK servers, everything working fine.
>> >> I just ran a MP last week which has generated a 400M rows table. But I
>> >> hae wrongly pre-split the table. so I want to merge and re-split it.
>> >>
>> >> I tried with the cluster on and it has detected it and stopped the
>> merge.
>> >> I stoped de cluster but it's trying to access the file locally? I ran
>> >> the merge on the master server directly from the HBase directory.
>> >>
>> >> I will take a look at see if there is any other parameter I should
>> >> pass, or if I missed something...
>> >>
>> >> JM
>> >>
>> >> 2012/11/18, Kevin O'dell <[EMAIL PROTECTED]>:
>> >>> Rather it is expecting local:
>> >>>
>> >>> 12/11/18 08:54:05 FATAL util.Merge: Merge failed
>> >>> java.lang.IllegalArgumentException: Wrong FS:
>> >>> hdfs://node3:9000/hbase/-ROOT-/70236052/.regioninfo, expected:
>> >>> file:///at
>> >>> org.apache.hadoop.fs. FileSystem.checkPath(FileSystem.java:381)
>> >>> On Nov 18, 2012 10:37 AM, "Kevin O'dell" <[EMAIL PROTECTED]>
>> >>> wrote:
>> >>>
>> >>>> Looks like your configuration is pointing local.
>> >>>> On Nov 18, 2012 8:59 AM, "Jean-Marc Spaggiari"
>> >>>> <[EMAIL PROTECTED]>
>> >>>> wrote:
>> >>>>
>> >>>>> Hi,
>> >>>>>
>> >>>>> I'm trying to merge 2 regions based on the book documentation and
>> >>>>> I'm
>> >>>>> getting the exception at the bottom of this message. Any idea why?
>> >>>>>
>> >>>>> I got the regions names from the HTML UI. I tried also to scan the
>> >>>>> .META. table like in Lars' book. Both are giving me the same info
>> >>>>> (a
>> >>>>> chance!), but it's still not working.
>> >>>>>
>> >>>>> Also, the HBase online book is not giving the right class info for
>> the
>> >>>>> merge. I will open a Jira and propose a patch.
>> >>>>>
>> >>>>> Thanks,
>> >>>>>
>> >>>>> JM
>> >>>>>
>> >>>>> hbase@node3:~/hbase-0.94.2$ bin/hbase
>> >>>>> org.apache.hadoop.hbase.util.Merge work_proposed
>> >>>>> work_proposed,\x0A,1342541226467.19929d1e6b6ecb3beae91e316b790378.
>> >>>>> work_proposed,\x14,1342541226467.622a3e62c4cf3f3b59655bdaf915908d.
>> >>>>> 12/11/18 08:54:05 INFO util.Merge: Verifying that file system is
>> >>>>> available...