Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Is there a setting to cap row size?


Copy link to this message
-
Is there a setting to cap row size?
I have a wide table schema for an HBase table, where I model a one-to-many relationship of purchase orders and line items. Each row is a purchase order, and I add columns for each line item. Under normal circumstances I don't expect more than a few thousand columns per row, totalling less than 1mb per row in general.

In one of my stress tests, I was inserting many line items into the same row. Eventually, the row's region server shut down. In the log there was an IOException with an error about the wite-ahead log failing to close, with the subsequent regionserver shutdown.

Once in this state, the only way I could manage to get my system functional again was to wipe the /hbase directory in HDFS and start from scratch.

To avoid having my system susceptible to total data loss due to some bad import data or the like, I'd like to limit the size of the row so it will simply throw an exception if it reaches a certain size (either in bytes or in number of columns). Does such a setting exist?
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB