Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Bulk loading HFiles via LoadIncrementalHFiles fails at a region that is being compacted, a bug?


+
Stanislav Barton 2013-07-05, 17:02
+
Stack 2013-07-05, 20:19
+
Stanislav Barton 2013-07-08, 09:05
+
Stanislav Barton 2013-07-08, 16:53
+
Stack 2013-07-10, 05:43
Copy link to this message
-
Re: Bulk loading HFiles via LoadIncrementalHFiles fails at a region that is being compacted, a bug?
Hello Stan,
                  Your bulk load trying to load data to multiple column
families?

-Anoop-

On Wed, Jul 10, 2013 at 11:13 AM, Stack <[EMAIL PROTECTED]> wrote:

> File a bug Stan please.  Paste your log snippet and surrounding what is
> going on at the time.  It looks broke that a bulk load would be kept out of
> a lock for ten minutes or more.
>
> Hope all is well,
> St.Ack
>
>
> On Mon, Jul 8, 2013 at 9:53 AM, Stanislav Barton <
> [EMAIL PROTECTED]
> > wrote:
>
> > Hello Michael,
> >
> > looking in the code, it seems to me that the 60s is hardcoded, however it
> > retries for, on default, 10 times, so in total 10 minutes wait time, I
> > upped that to 20 times, so now it is 20 minutes for me, but still, we
> have
> > some pretty big regions whose compaction (which was the case in
> particular)
> > can take more than 40 minutes. I have split the big regions to alleviate
> > this, so getting a thread dump now will be difficult (this is in
> production
> > so avoiding the problems is the point). Anyway, looking on the code, for
> me
> > its hard to figure out which actions will block the lock from succeeding
> on
> > the region at the place I indicated, so was hoping for an answer from an
> > expert. If the (major) compaction blocks the lock, it might be, that at
> > unit testing the compactions are faster than 10 minutes so the problem
> > never exhibits.
> >
> > Stan
>
+
Stanislav Barton 2013-07-10, 10:02
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB