Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Bulk loading HFiles via LoadIncrementalHFiles fails at a region that is being compacted, a bug?


+
Stanislav Barton 2013-07-05, 17:02
+
Stack 2013-07-05, 20:19
+
Stanislav Barton 2013-07-08, 09:05
+
Stanislav Barton 2013-07-08, 16:53
+
Stack 2013-07-10, 05:43
Copy link to this message
-
Re: Bulk loading HFiles via LoadIncrementalHFiles fails at a region that is being compacted, a bug?
Hello Stan,
                  Your bulk load trying to load data to multiple column
families?

-Anoop-

On Wed, Jul 10, 2013 at 11:13 AM, Stack <[EMAIL PROTECTED]> wrote:

> File a bug Stan please.  Paste your log snippet and surrounding what is
> going on at the time.  It looks broke that a bulk load would be kept out of
> a lock for ten minutes or more.
>
> Hope all is well,
> St.Ack
>
>
> On Mon, Jul 8, 2013 at 9:53 AM, Stanislav Barton <
> [EMAIL PROTECTED]
> > wrote:
>
> > Hello Michael,
> >
> > looking in the code, it seems to me that the 60s is hardcoded, however it
> > retries for, on default, 10 times, so in total 10 minutes wait time, I
> > upped that to 20 times, so now it is 20 minutes for me, but still, we
> have
> > some pretty big regions whose compaction (which was the case in
> particular)
> > can take more than 40 minutes. I have split the big regions to alleviate
> > this, so getting a thread dump now will be difficult (this is in
> production
> > so avoiding the problems is the point). Anyway, looking on the code, for
> me
> > its hard to figure out which actions will block the lock from succeeding
> on
> > the region at the place I indicated, so was hoping for an answer from an
> > expert. If the (major) compaction blocks the lock, it might be, that at
> > unit testing the compactions are faster than 10 minutes so the problem
> > never exhibits.
> >
> > Stan
>
+
Stanislav Barton 2013-07-10, 10:02