Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Can't insert data into hbase, OOME.


Copy link to this message
-
Re: Can't insert data into hbase, OOME.
ohh... sorry. I find it. Thanks for patch.

2010/9/30 Andrey Stepachev <[EMAIL PROTECTED]>:
> Against what version this patch.  I can't find "Only 1 KV, does" in
> any stumbleupon or
> upstream repositories
>
>
> 2010/9/29 Jean-Daniel Cryans <[EMAIL PROTECTED]>:
>> The fix is here: http://pastebin.com/zuL23e0U
>>
>> We're going to do a push to github later today, along with other
>> patches that require more testing.
>>
>> J-D
>>
>> On Wed, Sep 29, 2010 at 10:54 AM, Andrey Stepachev <[EMAIL PROTECTED]> wrote:
>>> wow. i'll wait. thanks for reply.
>>>
>>> 2010/9/29 Jean-Daniel Cryans <[EMAIL PROTECTED]>:
>>>> Ok I found the bug, I think it's only in our distro.
>>>>
>>>> Stay tuned!
>>>>
>>>> J-D
>>>>
>>>> On Wed, Sep 29, 2010 at 9:26 AM, Jean-Daniel Cryans <[EMAIL PROTECTED]> wrote:
>>>>> Weird indeed, even after the WAL was rolled 4 times (theoretically
>>>>> 256MB of data) I don't even see a flush request... although you're
>>>>> running at INFO level instead of DEBUG. Could you switch that and send
>>>>> us just the full log.
>>>>>
>>>>> Thanks a lot!
>>>>>
>>>>> J-D
>>>>>
>>>>> On Wed, Sep 29, 2010 at 4:25 AM, Andrey Stepachev <[EMAIL PROTECTED]> wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I'm stuck. I can't insert any valuable peace of data into hbase.
>>>>>>
>>>>>> Data is something around ~20mil rows (20G). I try to insert them into
>>>>>> nondistributed hbase with 4 parallel jobs.
>>>>>> MR job run until all memory given to hbase is exhaused and then
>>>>>> hbase produces hprof file. As profiler shows, all memory accumulated
>>>>>> in MemStore.kvset.
>>>>>> I don't understand, why hbase doesn't block untill flush memstore.
>>>>>> The same if I give hbase 6Gb or RAM.
>>>>>>
>>>>>> 6GB gc log http://paste.ubuntu.com/502577/
>>>>>>
>>>>>> hadoop: 0.20.2+320
>>>>>> hbase: stumbleupon-20100830
>>>>>>
>>>>>> nothing in hbase-site.xml (except hbase.rootdir and zookeeper.quorum).
>>>>>>
>>>>>>
>>>>>>
>>>>>> Andrey.
>>>>>>
>>>>>
>>>>
>>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB