Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> No minor compactions on a table built only on bulk loads


Copy link to this message
-
RE: No minor compactions on a table built only on bulk loads
It's not really a bug.

I think the assumption is that if you are at the level of doing your own bulk loads, you should also manage when you want to compact and split.  I know in cases where I've done this, I would usually know at certain points I would want to trigger major compactions.

At some level, bulk loads are about avoiding compactions so I don't think it would make sense that they automatically be triggered if solely bulk loads.

JG

> -----Original Message-----
> From: Vidhyashankar Venkataraman [mailto:[EMAIL PROTECTED]]
> Sent: Monday, January 10, 2011 4:33 PM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: No minor compactions on a table built only on bulk loads
>
> I have a table (in 0.90 candidate) whose writes are solely through bulk
> incremental loads. I had been running over a period of time and noticed that
> the Storefiles were not minor compacting or splitting. I eyeballed the code
> and observed that the (minor) compaction check is made at the time of
> deploying the region and during every commit (after a flush?) but never
> while bulk loading.
>
> Can you let me know if this was deliberate/an existing bug?
>
> Thank you
> Vidhya
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB