Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Short-circuit reads


Copy link to this message
-
Re: Short-circuit reads
Jean-Daniel Cryans 2013-01-27, 22:45
Inline.

J-D

On Sun, Jan 27, 2013 at 2:29 PM, David Koch <[EMAIL PROTECTED]> wrote:
> Hello,
>
> I read about "short circuit reads" in the HBase documentation's performance
> section[1] and was wondering what people's experiences were using this in a
> production setting.
>
> Also,
>
> 1. Since only one dedicated user can take advantage of the feature do you
> launch all jobs as this user?

That's the big limitation right now. Running everything as the same
user can make managing jobs difficult, also that user would need to be
the same as HBase's.

FWIW, HDFS-347 should fix those limitions but it's not committed yet
(getting close tho).

> 2. Can dfs.client.read.shortcircuit be set to false for jobs wich are not
> launched by the short-circuit user in order to avoid exceptions? In other
> words - can this setting be overriden by the client configuration's
> hbase-site.xml?

Yes, but those exceptions are really harmless.

> 3. In the same context, it is suggested to enable HBase internal
> checksums[2]. Is this a feature which can be enabled in HBase 0.92.1 which
> is part of the Cloudera 4.1.x release?

Yes on the first question, no on the second one (what Ted said)

>
> Thank you,
>
> /David
>
> [1] http://hbase.apache.org/book/perf.hdfs.html#ftn.d2145e7370
> [2] https://issues.apache.org/jira/browse/HBASE-5074