Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> HBase Region Server crash if column size become to big


+
John 2013-09-11, 11:07
+
Jean-Marc Spaggiari 2013-09-11, 11:34
+
John 2013-09-11, 11:42
Copy link to this message
-
Re: HBase Region Server crash if column size become to big
Take a look at http://hbase.apache.org/0.94/apidocs/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.html

Cheers

On Sep 11, 2013, at 4:42 AM, John <[EMAIL PROTECTED]> wrote:

> Hi,
>
> thanks for your fast answer! with size becoming too big I mean I have one
> row with thousands of columns. For example:
>
> myrowkey1 -> column1, column2, column3 ... columnN
>
> What do you mean with "change the batch size"? I try to create a little
> java test code to reproduce the problem. It will take a moment
>
>
>
>
> 2013/9/11 Jean-Marc Spaggiari <[EMAIL PROTECTED]>
>
>> Hi John,
>>
>> Just to be sure. What is " the size become too big"? The size of a single
>> column within this row? Or the number of columns?
>>
>> If it's the number of columns, you can change the batch size to get less
>> columns in a single call? Can you share the relevant piece of code doing
>> the call?
>>
>> JM
>>
>>
>> 2013/9/11 John <[EMAIL PROTECTED]>
>>
>>> Hi,
>>>
>>> I store a lot of columns for one row key and if the size become to big
>> the
>>> relevant Region Server crashs if I try to get or scan the row. For
>> example
>>> if I try to get the relevant row I got this error:
>>>
>>> 2013-09-11 12:46:43,696 WARN org.apache.hadoop.ipc.HBaseServer:
>>> (operationTooLarge): {"processingtimems":3091,"client":"
>> 192.168.0.34:52488
>>> ","ti$
>>>
>>> If I try to load the relevant row via Apache Pig and the HBaseStorage
>>> Loader (use the scan operation) I got this message and after that the
>>> Region Servers crashs:
>>>
>>> 2013-09-11 10:30:23,542 WARN org.apache.hadoop.ipc.HBaseServer:
>>> (responseTooLarge):
>>> {"processingtimems":1851,"call":"next(-588368116791418695,
>>> 1), rpc version=1, client version=29,$
>>>
>>> I'm using Cloudera 4.4.0 with 0.94.6-cdh4.4.0
>>>
>>> Any clues?
>>>
>>> regards
>>
+
John 2013-09-11, 12:38
+
Bing Jiang 2013-09-11, 13:41
+
John 2013-09-11, 14:46
+
John 2013-09-11, 14:47
+
Michael Segel 2013-09-11, 14:53
+
Kevin Odell 2013-09-11, 15:02
+
John 2013-09-11, 15:08
+
Kevin Odell 2013-09-11, 15:15
+
Dhaval Shah 2013-09-11, 15:15
+
Kevin Odell 2013-09-11, 15:20
+
John 2013-09-11, 15:26
+
Dhaval Shah 2013-09-11, 15:33
+
Kevin Odell 2013-09-11, 15:30
+
Michael Segel 2013-09-11, 15:31
+
Dhaval Shah 2013-09-11, 15:38
+
Michael Segel 2013-09-11, 18:12
+
Dhaval Shah 2013-09-11, 18:13
+
Ted Yu 2013-09-11, 13:19
+
Jean-Marc Spaggiari 2013-09-11, 11:48
+
John 2013-09-11, 15:58
+
Bryan Beaudreault 2013-09-11, 16:15
+
John 2013-09-11, 17:03
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB