Jean-Daniel Cryans 2013-03-21, 18:03
ramkrishna vasudevan 2013-03-21, 18:05
Nicolas Seyvet 2013-03-21, 17:52
-Re: How to prevent major compaction when doing bulk load provisioning?
Nicolas Seyvet 2013-03-21, 19:06
@Ram: You are entirely correct, I made the exact same mistakes of mixing up
Large and minor compaction. By looking closely, what I see is that at
around 200 HFiles per region it starts minor compacting files per group of
10 HFiles. The "problem" seems that this minor compacting never stops even
when there are about 20 HFiles left. It just keep on going and on taking
more and more time (I guess because the files to compact are getting
Of course in parallel we keep on adding more and more data.
@J-D: "It seems to me that it would be better if you were able to do a
single load for all your files." Yes, I agree.. but that is not what we are
testing, our use case is to use 1min batch files.
Nicolas Seyvet 2013-03-22, 07:12
Jean-Daniel Cryans 2013-03-22, 16:32
Jean-Daniel Cryans 2013-03-21, 20:21
Ted Yu 2013-03-21, 20:05
Ted Yu 2013-03-21, 17:10
Amit Sela 2013-03-21, 16:47