Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> querying salted Hbase tables


Copy link to this message
-
querying salted Hbase tables
Folks -- we have a timeseries-based table we recently converted to a salted
key schema [1] in order to avoid region hotspotting.  The rowkey format is:

salt-timestamp-sessionid-eventtype, where:

salt has the form 00..13, and the timestamp is a Unix timestamp (epoch
based).

With the version 0.10.0 HBaseStorage, what's the recommended way to LOAD a
salted schema from Pig?  Initially, I thought we'd just fire off multiple
LOADs, one for each region (in our case, up to 14), but we're hitting
frequently ScannerTimeoutExceptions with this approach, even on a sample
script that does nothing but LOADs.

Is there a better way?

Thanks,
Norbert

[1]
http://ofps.oreilly.com/titles/9781449396107/advanced.html#ch09_id2336987
+
Dmitriy Ryaboy 2011-09-13, 14:43
+
Norbert Burger 2011-09-13, 15:08
+
Dmitriy Ryaboy 2011-09-13, 16:35