Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - No minor compactions on a table built only on bulk loads


Copy link to this message
-
RE: No minor compactions on a table built only on bulk loads
Jonathan Gray 2011-01-11, 00:38
It's not really a bug.

I think the assumption is that if you are at the level of doing your own bulk loads, you should also manage when you want to compact and split.  I know in cases where I've done this, I would usually know at certain points I would want to trigger major compactions.

At some level, bulk loads are about avoiding compactions so I don't think it would make sense that they automatically be triggered if solely bulk loads.

JG

> -----Original Message-----
> From: Vidhyashankar Venkataraman [mailto:[EMAIL PROTECTED]]
> Sent: Monday, January 10, 2011 4:33 PM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: No minor compactions on a table built only on bulk loads
>
> I have a table (in 0.90 candidate) whose writes are solely through bulk
> incremental loads. I had been running over a period of time and noticed that
> the Storefiles were not minor compacting or splitting. I eyeballed the code
> and observed that the (minor) compaction check is made at the time of
> deploying the region and during every commit (after a flush?) but never
> while bulk loading.
>
> Can you let me know if this was deliberate/an existing bug?
>
> Thank you
> Vidhya