Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Odd WARN in hbase 0.94.2


Copy link to this message
-
Re: Odd WARN in hbase 0.94.2
The datanode data directory permission settings applies to Apache HDFS
0.22, 0.23 and 2.x, is unrelated and unnecessary for HDFS-347 but is
necessary for the older SCR. If not mentioned in the book yet, they
ought to go in I think as even some other distributions (other than
HDFS 1.x) included the specific security settings in their
past-version releases.

The other settings (the domain socket options), which remove the need
to expose the data directory permissions are part of HDFS-347, which
are in a branch upstream right now, awaiting integration into an
upcoming HDFS release. We could document these better options once
such a release is out, as its a further gain and is also secure.

On Thu, Mar 7, 2013 at 2:38 PM, Anoop Sam John <[EMAIL PROTECTED]> wrote:
> Hi Byran,
>                           This change is needed with usage of any of the open src HDFS release or is it only in CDH? Is this related with HDFS-347?
> In such a case forget abt my previous mail abt adding in book  :)
>
> -Anoop-
> ________________________________________
> From: Kevin O'dell [[EMAIL PROTECTED]]
> Sent: Thursday, March 07, 2013 2:13 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Odd WARN in hbase 0.94.2
>
> Hi Byran,
>
> One of engineers wanted me to pass this recommendation along:
>
> The issue here is mostly likely a missing configuration.
>
> We need to have this configuration on both the DataNode and RegionServer:
> <property>
>   <name>dfs.domain.socket.path</name>
>   <value>/var/run/hadoop-hdfs/dn._PORT</value>
> </property>
> <property>
>   <name>dfs.client.read.shortcircuit</name>
>   <value>true</value>
> </property>
>
> In CDH4.1 and earlier, the DN didn't need to be configured to use SCR,
> so most likely he does not have dfs.client.read.shortcircuit set in
> hdfs-site.xml.  Adding that should fix it
>
> On Tue, Mar 5, 2013 at 5:17 PM, Bryan Beaudreault
> <[EMAIL PROTECTED]>wrote:
>
>> Yep we do have that property set to that value.  The file does not seem to
>> exist when I try ls'ing it myself.  I'm not sure where it comes from or how
>> it should be created.
>>
>>
>> On Tue, Mar 5, 2013 at 4:35 PM, Kevin O'dell <[EMAIL PROTECTED]
>> >wrote:
>>
>> > Bryan,
>> >
>> >   What permissions did you set for the SCRs?  The DNs need
>> >
>> > <property>
>> > <name>dfs.datanode.data.dir.perm</name>
>> > <value>755</name>
>> > </property>
>> >
>> >
>> > On Tue, Mar 5, 2013 at 4:28 PM, Bryan Beaudreault
>> > <[EMAIL PROTECTED]>wrote:
>> >
>> > > We are running hbase 0.94.2, cdh4.2 edition.  We have the local read
>> > > shortcut enabled.  Since updating to this version I've seen a bunch of
>> > WARN
>> > > messages corresponding to exceptions trying to connect to what looks
>> like
>> > > the local DN.
>> > >
>> > > As far as I can tell it doesn't appear to be having much affect on the
>> RS
>> > > but it'd be nice to clear it up and know for sure that it doesn't
>> > indicate
>> > > some problem. Has anyone ever seen this and know what it is or how to
>> fix
>> > > it?
>> > >
>> > > http://pastebin.com/PA6Y9pJN
>> > >
>> > > Thanks!
>> > >
>> >
>> >
>> >
>> > --
>> > Kevin O'Dell
>> > Customer Operations Engineer, Cloudera
>> >
>>
>
>
>
> --
> Kevin O'Dell
> Customer Operations Engineer, Cloudera

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB