Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Bulk loading HFiles via LoadIncrementalHFiles fails at a region that is being compacted, a bug?


Copy link to this message
-
Re: Bulk loading HFiles via LoadIncrementalHFiles fails at a region that is being compacted, a bug?
File a bug Stan please.  Paste your log snippet and surrounding what is
going on at the time.  It looks broke that a bulk load would be kept out of
a lock for ten minutes or more.

Hope all is well,
St.Ack
On Mon, Jul 8, 2013 at 9:53 AM, Stanislav Barton <[EMAIL PROTECTED]
> wrote:

> Hello Michael,
>
> looking in the code, it seems to me that the 60s is hardcoded, however it
> retries for, on default, 10 times, so in total 10 minutes wait time, I
> upped that to 20 times, so now it is 20 minutes for me, but still, we have
> some pretty big regions whose compaction (which was the case in particular)
> can take more than 40 minutes. I have split the big regions to alleviate
> this, so getting a thread dump now will be difficult (this is in production
> so avoiding the problems is the point). Anyway, looking on the code, for me
> its hard to figure out which actions will block the lock from succeeding on
> the region at the place I indicated, so was hoping for an answer from an
> expert. If the (major) compaction blocks the lock, it might be, that at
> unit testing the compactions are faster than 10 minutes so the problem
> never exhibits.
>
> Stan