Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Bulk loading HFiles via LoadIncrementalHFiles fails at a region that is being compacted, a bug?


Copy link to this message
-
Re: Bulk loading HFiles via LoadIncrementalHFiles fails at a region that is being compacted, a bug?
File a bug Stan please.  Paste your log snippet and surrounding what is
going on at the time.  It looks broke that a bulk load would be kept out of
a lock for ten minutes or more.

Hope all is well,
St.Ack
On Mon, Jul 8, 2013 at 9:53 AM, Stanislav Barton <[EMAIL PROTECTED]
> wrote:

> Hello Michael,
>
> looking in the code, it seems to me that the 60s is hardcoded, however it
> retries for, on default, 10 times, so in total 10 minutes wait time, I
> upped that to 20 times, so now it is 20 minutes for me, but still, we have
> some pretty big regions whose compaction (which was the case in particular)
> can take more than 40 minutes. I have split the big regions to alleviate
> this, so getting a thread dump now will be difficult (this is in production
> so avoiding the problems is the point). Anyway, looking on the code, for me
> its hard to figure out which actions will block the lock from succeeding on
> the region at the place I indicated, so was hoping for an answer from an
> expert. If the (major) compaction blocks the lock, it might be, that at
> unit testing the compactions are faster than 10 minutes so the problem
> never exhibits.
>
> Stan
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB