I think the answer to this is "no", but I am hoping someone with more experience can confirm this… we are on hbase 0.90.4 (from cdh3u2). Some of our storefiles have grown into the 3-4GB range (we have 100GB max region size). Ignoring compactions, do large storefiles like this have a negative impact on random reads? We only recently started doing a large number of random gets so I have no history to go on in terms of correlating storefile size with lackluster performance for random reads.

Thanks,
Paul

re:  "with lackluster performance for random reads"

You want to be on CDH3u3 for sure if you want to boost random read
performance.
On 5/2/12 5:29 PM, "Paul Mackles" <[EMAIL PROTECTED]> wrote:

>I think the answer to this is "no", but I am hoping someone with more
>experience can confirm thisŠ we are on hbase 0.90.4 (from cdh3u2). Some
>of our storefiles have grown into the 3-4GB range (we have 100GB max
>region size). Ignoring compactions, do large storefiles like this have a
>negative impact on random reads? We only recently started doing a large
>number of random gets so I have no history to go on in terms of
>correlating storefile size with lackluster performance for random reads.
>
>Thanks,
>Paul

Thanks for the tip Doug. Does that boost come largely from the HDFS
improvements?

On 5/2/12 7:52 PM, "Doug Meil" <[EMAIL PROTECTED]> wrote:

>
>re:  "with lackluster performance for random reads"
>
>You want to be on CDH3u3 for sure if you want to boost random read
>performance.
>
>
>
>
>On 5/2/12 5:29 PM, "Paul Mackles" <[EMAIL PROTECTED]> wrote:
>
>>I think the answer to this is "no", but I am hoping someone with more
>>experience can confirm thisŠ we are on hbase 0.90.4 (from cdh3u2). Some
>>of our storefiles have grown into the 3-4GB range (we have 100GB max
>>region size). Ignoring compactions, do large storefiles like this have a
>>negative impact on random reads? We only recently started doing a large
>>number of random gets so I have no history to go on in terms of
>>correlating storefile size with lackluster performance for random reads.
>>
>>Thanks,
>>Paul
>
>

On Wed, May 2, 2012 at 6:00 PM, Paul Mackles <[EMAIL PROTECTED]> wrote:
> Thanks for the tip Doug. Does that boost come largely from the HDFS
> improvements?
>

Yeah, unless you install 0.92.x hbase (or if you want more
improvement, install 0.94.x RC).

St.Ack

No, it's a direct read using a block index which is in memory.

J-D

On Wed, May 2, 2012 at 2:29 PM, Paul Mackles <[EMAIL PROTECTED]> wrote:
> I think the answer to this is "no", but I am hoping someone with more experience can confirm this… we are on hbase 0.90.4 (from cdh3u2). Some of our storefiles have grown into the 3-4GB range (we have 100GB max region size). Ignoring compactions, do large storefiles like this have a negative impact on random reads? We only recently started doing a large number of random gets so I have no history to go on in terms of correlating storefile size with lackluster performance for random reads.
>
> Thanks,
> Paul