Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Odd WARN in hbase 0.94.2


+
Bryan Beaudreault 2013-03-05, 21:28
+
Kevin Odell 2013-03-05, 21:35
+
Bryan Beaudreault 2013-03-05, 22:17
+
Kevin Odell 2013-03-06, 20:43
+
Bryan Beaudreault 2013-03-06, 21:04
+
Anoop Sam John 2013-03-07, 03:45
+
Bryan Beaudreault 2013-03-07, 03:56
+
Anoop Sam John 2013-03-07, 09:08
Copy link to this message
-
Re: Odd WARN in hbase 0.94.2
Harsh J 2013-03-07, 11:06
The datanode data directory permission settings applies to Apache HDFS
0.22, 0.23 and 2.x, is unrelated and unnecessary for HDFS-347 but is
necessary for the older SCR. If not mentioned in the book yet, they
ought to go in I think as even some other distributions (other than
HDFS 1.x) included the specific security settings in their
past-version releases.

The other settings (the domain socket options), which remove the need
to expose the data directory permissions are part of HDFS-347, which
are in a branch upstream right now, awaiting integration into an
upcoming HDFS release. We could document these better options once
such a release is out, as its a further gain and is also secure.

On Thu, Mar 7, 2013 at 2:38 PM, Anoop Sam John <[EMAIL PROTECTED]> wrote:
> Hi Byran,
>                           This change is needed with usage of any of the open src HDFS release or is it only in CDH? Is this related with HDFS-347?
> In such a case forget abt my previous mail abt adding in book  :)
>
> -Anoop-
> ________________________________________
> From: Kevin O'dell [[EMAIL PROTECTED]]
> Sent: Thursday, March 07, 2013 2:13 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Odd WARN in hbase 0.94.2
>
> Hi Byran,
>
> One of engineers wanted me to pass this recommendation along:
>
> The issue here is mostly likely a missing configuration.
>
> We need to have this configuration on both the DataNode and RegionServer:
> <property>
>   <name>dfs.domain.socket.path</name>
>   <value>/var/run/hadoop-hdfs/dn._PORT</value>
> </property>
> <property>
>   <name>dfs.client.read.shortcircuit</name>
>   <value>true</value>
> </property>
>
> In CDH4.1 and earlier, the DN didn't need to be configured to use SCR,
> so most likely he does not have dfs.client.read.shortcircuit set in
> hdfs-site.xml.  Adding that should fix it
>
> On Tue, Mar 5, 2013 at 5:17 PM, Bryan Beaudreault
> <[EMAIL PROTECTED]>wrote:
>
>> Yep we do have that property set to that value.  The file does not seem to
>> exist when I try ls'ing it myself.  I'm not sure where it comes from or how
>> it should be created.
>>
>>
>> On Tue, Mar 5, 2013 at 4:35 PM, Kevin O'dell <[EMAIL PROTECTED]
>> >wrote:
>>
>> > Bryan,
>> >
>> >   What permissions did you set for the SCRs?  The DNs need
>> >
>> > <property>
>> > <name>dfs.datanode.data.dir.perm</name>
>> > <value>755</name>
>> > </property>
>> >
>> >
>> > On Tue, Mar 5, 2013 at 4:28 PM, Bryan Beaudreault
>> > <[EMAIL PROTECTED]>wrote:
>> >
>> > > We are running hbase 0.94.2, cdh4.2 edition.  We have the local read
>> > > shortcut enabled.  Since updating to this version I've seen a bunch of
>> > WARN
>> > > messages corresponding to exceptions trying to connect to what looks
>> like
>> > > the local DN.
>> > >
>> > > As far as I can tell it doesn't appear to be having much affect on the
>> RS
>> > > but it'd be nice to clear it up and know for sure that it doesn't
>> > indicate
>> > > some problem. Has anyone ever seen this and know what it is or how to
>> fix
>> > > it?
>> > >
>> > > http://pastebin.com/PA6Y9pJN
>> > >
>> > > Thanks!
>> > >
>> >
>> >
>> >
>> > --
>> > Kevin O'Dell
>> > Customer Operations Engineer, Cloudera
>> >
>>
>
>
>
> --
> Kevin O'Dell
> Customer Operations Engineer, Cloudera

--
Harsh J