Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Can't insert data into hbase, OOME.


Copy link to this message
-
Re: Can't insert data into hbase, OOME.
Against what version this patch.  I can't find "Only 1 KV, does" in
any stumbleupon or
upstream repositories
2010/9/29 Jean-Daniel Cryans <[EMAIL PROTECTED]>:
> The fix is here: http://pastebin.com/zuL23e0U
>
> We're going to do a push to github later today, along with other
> patches that require more testing.
>
> J-D
>
> On Wed, Sep 29, 2010 at 10:54 AM, Andrey Stepachev <[EMAIL PROTECTED]> wrote:
>> wow. i'll wait. thanks for reply.
>>
>> 2010/9/29 Jean-Daniel Cryans <[EMAIL PROTECTED]>:
>>> Ok I found the bug, I think it's only in our distro.
>>>
>>> Stay tuned!
>>>
>>> J-D
>>>
>>> On Wed, Sep 29, 2010 at 9:26 AM, Jean-Daniel Cryans <[EMAIL PROTECTED]> wrote:
>>>> Weird indeed, even after the WAL was rolled 4 times (theoretically
>>>> 256MB of data) I don't even see a flush request... although you're
>>>> running at INFO level instead of DEBUG. Could you switch that and send
>>>> us just the full log.
>>>>
>>>> Thanks a lot!
>>>>
>>>> J-D
>>>>
>>>> On Wed, Sep 29, 2010 at 4:25 AM, Andrey Stepachev <[EMAIL PROTECTED]> wrote:
>>>>> Hi all,
>>>>>
>>>>> I'm stuck. I can't insert any valuable peace of data into hbase.
>>>>>
>>>>> Data is something around ~20mil rows (20G). I try to insert them into
>>>>> nondistributed hbase with 4 parallel jobs.
>>>>> MR job run until all memory given to hbase is exhaused and then
>>>>> hbase produces hprof file. As profiler shows, all memory accumulated
>>>>> in MemStore.kvset.
>>>>> I don't understand, why hbase doesn't block untill flush memstore.
>>>>> The same if I give hbase 6Gb or RAM.
>>>>>
>>>>> 6GB gc log http://paste.ubuntu.com/502577/
>>>>>
>>>>> hadoop: 0.20.2+320
>>>>> hbase: stumbleupon-20100830
>>>>>
>>>>> nothing in hbase-site.xml (except hbase.rootdir and zookeeper.quorum).
>>>>>
>>>>>
>>>>>
>>>>> Andrey.
>>>>>
>>>>
>>>
>>
>