Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Short-circuit reads


+
David Koch 2013-01-27, 22:29
+
Jean-Daniel Cryans 2013-01-27, 22:45
+
Ted 2013-01-27, 22:34
+
Jean-Marc Spaggiari 2013-01-27, 22:38
+
Jean-Daniel Cryans 2013-01-27, 22:50
+
Jean-Marc Spaggiari 2013-01-27, 22:55
+
David Koch 2013-01-27, 23:16
Copy link to this message
-
Re: Short-circuit reads
It's in the region server metrics and also published through JMX.

J-D

On Sun, Jan 27, 2013 at 2:55 PM, Jean-Marc Spaggiari
<[EMAIL PROTECTED]> wrote:
> Hi J-D,
>
> In your presentation, where are you getting the
> hdfsBlocksLocalityIndex value? I'm not able to find it in my UI...
>
> Thanks,
>
> JM
>
> 2013/1/27, Jean-Daniel Cryans <[EMAIL PROTECTED]>:
>> On Sun, Jan 27, 2013 at 2:38 PM, Jean-Marc Spaggiari
>> <[EMAIL PROTECTED]> wrote:
>>> Does this have a significant increase on HBase performances?
>>
>> Ted beats me to linking to my own presentation :P
>>
>>> And what
>>> are the "risks" associated with short-circuit activation (if any)? No
>>> risks of corrupting data?
>>
>> No, nothing, apart.
>>
>>>
>>> Can this be activated after tables are already populated?
>>
>> Not related at all, this is a region server configuration.
>>
>>>
>>> JM
>>>
>>> 2013/1/27, Ted <[EMAIL PROTECTED]>:
>>>> For hbase internal checksum, it is not in hbase 0.92.x release.
>>>>
>>>> Please use 0.94.2 or newer release.
>>>>
>>>> Thanks
>>>>
>>>> On Jan 27, 2013, at 2:29 PM, David Koch <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I read about "short circuit reads" in the HBase documentation's
>>>>> performance
>>>>> section[1] and was wondering what people's experiences were using this
>>>>> in
>>>>> a
>>>>> production setting.
>>>>>
>>>>> Also,
>>>>>
>>>>> 1. Since only one dedicated user can take advantage of the feature do
>>>>> you
>>>>> launch all jobs as this user?
>>>>> 2. Can dfs.client.read.shortcircuit be set to false for jobs wich are
>>>>> not
>>>>> launched by the short-circuit user in order to avoid exceptions? In
>>>>> other
>>>>> words - can this setting be overriden by the client configuration's
>>>>> hbase-site.xml?
>>>>> 3. In the same context, it is suggested to enable HBase internal
>>>>> checksums[2]. Is this a feature which can be enabled in HBase 0.92.1
>>>>> which
>>>>> is part of the Cloudera 4.1.x release?
>>>>>
>>>>> Thank you,
>>>>>
>>>>> /David
>>>>>
>>>>> [1] http://hbase.apache.org/book/perf.hdfs.html#ftn.d2145e7370
>>>>> [2] https://issues.apache.org/jira/browse/HBASE-5074
>>>>
>>
+
Jean-Marc Spaggiari 2013-01-28, 12:45
+
Ted Yu 2013-01-27, 22:45
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB